Test data and schema tests

Test data and schema tests

tests/data/README.md sorts the example data into valid/, for data that conforms to the schema, invalid/, for data that must fail validation, and problem/, for data the current schema does not yet handle correctly.¹ A file name has the form ClassName-###.yaml: the part before the first hyphen is the name of a class as the schema writes it, and ### is a number or other characters a file name allows. CONTRIBUTING.md asks for an invalid example for each constraint.

The examples

tests/data/valid/ holds 9 files: 2 for GhaWorkflow, 1 for GhaReusableWorkflow, 2 for GhaAction, 1 for GhaRepositoryRuleset, 2 for GhaRequiredStatusCheck and 1 for GhaDocument. GhaDocument-service.yaml holds one repository’s workflows, its reusable workflow, an action, its ruleset and the check the ruleset requires, which names the workflow that reports it. tests/data/invalid/ holds 34 counter-examples: 12 for GhaWorkflow, 5 for GhaReusableWorkflow, 5 for GhaAction, 7 for GhaRepositoryRuleset and 5 for GhaRequiredStatusCheck. The two problem/ folders hold no example yet. Every example names an example-org repository or an example.org address.

tests/test_data.py

The first test loads each valid file into the generated dataclass of the class its name gives. The other two validate every file with linkml.validator against src/github_actions/schema/github_actions.yaml: a valid example must pass, and a counter-example must fail.² A comment says that the JSON Schema the validator generates is the one check that applies every constraint.

tests/test_schema_rules.py

Six tests compare the schema with the SchemaStore copy and with the upstream watch, and two check the adapt copy:³

• the schema header’s pins equal those in scripts/check_upstream_release.py, and the reading date equals the header’s version;
• the permission pattern admits every scope and level SchemaStore lists;
• the runtime enum differs from SchemaStore’s only by the retired node12 and node16;
• the input types are SchemaStore’s;
• the trigger pattern admits every event SchemaStore names;
• the key and uses patterns are SchemaStore’s;
• no class, enum, type or subset name in .lokf/github-actions.yaml appears in .lokf/lokf.yaml or tests/neighbours/slsa.yaml, and a slot or attribute name they share carries the same IRI;
• every mapping in .lokf/github-actions.yaml is a CURIE whose prefix the copy binds, because a full address that no prefix covers makes every lokf convert --schema over the copy warn that it has no namespace.

tests/neighbours/slsa.yaml is lmodel/slsa’s lokf adapt copy. The test file’s comment says that it is the schema-v0.7.0-slsa.yaml asset of slsa’s v0.7.0 release, checked against its .sha256 and attestation.

tests/test_derive.py

tests/test_derive.py holds the twenty-four tests of scripts/derive.py. They run the script against the fixture repository in tests/derive/repo/: six workflow files and two action files that show each of its rules.⁴ They also read tests/derive/rulesets.md and tests/derive/check-runs.md, which hold what scripts/github_export.py prints. tests/test_github_export.py runs that script against fixed answers in place of the REST API.⁵ Derive and check concepts from workflow files describes the script and its tests.

How they run

just test runs three recipes.⁶ _test-schema generates the project into tmp/. _test-python regenerates the Python models and runs pytest. _test-examples runs linkml-run-examples over tests/data/valid/ and tests/data/invalid/ and writes what it converts into examples/output/, which git ignores. Build and test runs just test on every pull request.

Footnotes

1. tests/data/README.md, the whole page ↩

2. tests/test_data.py, the three test functions ↩

3. tests/test_schema_rules.py, the eight test functions and the comment above COPY ↩

4. tests/test_derive.py, the docstring and the fixture path ↩

5. tests/test_github_export.py, the docstring ↩

6. justfile, the test, _test-schema, _test-python and _test-examples recipes ↩