Improve negative tests for JSON schema
We use negative tests to check that the JSON schema invalidates the data we want to invalidate. However, the tests are currently set up such that multiple test cases in one file are not actually checked, since the validator will invalidate the file once it finds just one error.
See also this comment !95469 (comment 1124770149):
[Negative tests] will not cover cases as they should, the reason of this is that "negative tests are validated as a whole file against schema" - so validator should fail on very first error, as the result second/third/etc errors in test-file will not be checked
Possible Solutions
- Restructure current tests in
spec/frontend/editor/schema/ci/yaml_tests/negative_tests
so each test is in a separate file (not ideal, but easy to do) - Restructure test set up to allow validator to check ALL test cases and make sure each one of them fails (requires digging more into the code and possibly the ajv docs to see how we can do this)
Edited by Mireya Andres