Test data

Test data

tests/data/README.md says that each file is named ClassName-###.yaml, where the part before the first - is the name of a class as the schema writes it, and sorts the files into three subfolders.¹

• valid/ holds 32 files² of data that conforms to the schema:
  • statements for each kind of predicate: SlsaStatement files for build provenance and Tekton build provenance, and InTotoStatement files for a verification summary, source provenance, source verification summaries for a git and an SVN revision, and a build environment attestation;
  • files for InTotoDigestSet, InTotoResourceDescriptor and SlsaDocument, and for the build provenance and verification summary predicates on their own;
  • a file for each role a party can act in, which the schema models as the subclasses of SlsaParty: SlsaProducer, SlsaConsumer, SlsaVerifier, SlsaInfrastructureProvider and SlsaBuildImageProducer;
  • files for the other Slsa classes a document holds: the build platform and its control plane, the package and its registry, the source control system, repository and revision, a branch and a tag, the build image, the dependency inventory and the adoption metadata.
• invalid/ holds 36 files of data that must fail, and the first line of each says why. Among them are:
  • an InTotoStatement with no subject, a subject with no digest, the wrong statement type or a predicate of the wrong kind;
  • an InTotoDigestSet that is empty or whose SHA-256 digest is in uppercase hexadecimal;
  • an InTotoResourceDescriptor with no URI, digest or content;
  • a SlsaBuildPlatform with an unknown level;
  • a SlsaBuildProvenance whose start time carries a +01:00 offset, where an in-toto timestamp ends in Z;
  • a SlsaVerificationSummary with an unknown result, a policy with no URI, or a verifier id whose host is in uppercase.
• problem/ holds data the current schema does not handle correctly yet, split into valid/ and invalid/. Both are empty today.

tests/test_data.py loads every file in valid/ with the generated dataclass model. tests/test_schema_rules.py checks rules the schema states in two places: the digest set’s named algorithms against its at-least-one rule and its reserved names, and the slsa_result_value pattern against SlsaResult. just test also runs linkml-run-examples over valid/ and invalid/, writing its output to the git-ignored examples/output/.

Footnotes

1. tests/data/README.md, the naming rule and the list of subfolders ↩

2. tests/data/, the YAML files in valid/, invalid/ and problem/ ↩