Test data and tests

tests/data/valid/ holds 21 examples, one of each ATT&CK class and more of a technique, and tests/data/invalid/ holds 72 counter-examples. Each example is an object of MITRE’s v20.0 data written as YAML. Each counter-example is a valid example with one thing broken. Its first line, # Fails at:, names where the validator must report it.¹ tests/test_data.py validates them against the JSON Schema the validator generates, and checks that every ATT&CK class has an example.²

tests/data/attack-stix-data/extract/ holds 97 objects copied unchanged from the three bundles: 37 from Enterprise, 31 from Mobile and 29 from ICS.³ The default tests validate each one.

tests/attack_rules.py holds the ADM’s rules that no validator checks, and the tests apply them to the examples and the extract.⁴ tests/test_upstream.py lists the six places where the data departs from the ADM, and the 44 places where it breaks a statement of the ADM’s documentation across a bundle, each group with its reason. It also holds the schema header to the pins.⁵ tests/test_attack_data.py runs only under just test-attack-data and validates every object of the full bundles, one at a time.⁶

tests/neighbours/ holds the adapt copies of nine released lmodel schemas and of lmodel/stix, which tests/test_schema_rules.py pins by SHA-256 and holds this schema’s copy apart from.⁷

Footnotes

Section titled “Footnotes”

1. tests/data/README.md ↩

2. tests/test_data.py, the docstring and the tests ↩

3. tests/data/attack-stix-data/extract/, the three files ↩

4. tests/attack_rules.py, the docstring ↩

5. tests/test_upstream.py, KNOWN_DEPARTURES and KNOWN_BUNDLE_DEPARTURES ↩

6. tests/test_attack_data.py, the docstring ↩

7. tests/test_schema_rules.py, NEIGHBOURS and STIX ↩