Skip to content

Automatically detect Jest tests to run upon backend changes

Rémy Coutable requested to merge 339343-execute-related-jests-specs-for-mrs-with-backend-changes into master

What does this MR do and why?

This implements a first backend file -> (frontend fixture RSpec file) -> frontend test strategy based on frontend fixtures RSpec files.

The main work is done in generate-frontend-fixtures-mapping and detect-tests:

generate-frontend-fixtures-mapping

generate-frontend-fixtures-mapping will run on the 2-hour scheduled pipeline, similarly to the update-tests-metadata's rules.

We generate a test file -> JSON frontend fixture mapping file by running the spec/frontend/fixtures/ RSpec files with GENERATE_FRONTEND_FIXTURES_MAPPING="true" set, which skips the test from running, and only generates the crystalball/frontend_fixtures_mapping.json mapping file.

The mapping file is saved as artifact, and then retrieved by update-tests-metadata so that the pages job can publish the file via Pages (for now we don't retrieve tests metadata file from Pages, but we're able too and might rely on that in the future).

See what the file looks like at https://gitlab-org.gitlab.io/-/gitlab/-/jobs/1772757819/artifacts/crystalball/frontend_fixtures_mapping.json.

detect-tests

  1. The crystalball/frontend_fixtures_mapping.json file is retrieved from the latest successful generate-frontend-fixtures-mapping job from a scheduled pipeline (similarly to how the crystalball/packed-mapping.json file is retrieved).
  2. We get the list of changed files from the MR as usual from the API (with tooling/bin/find_changes). For example, let's assume we changed app/controllers/projects/autocomplete_sources_controller.rb.
  3. We get the list of minimal tests to run based on the list of changed files and the Crystalball & custom mapping (tests.yml file) (with tooling/bin/find_tests).
    • Given the example change from above, we know thanks to Crystalball that spec/frontend/fixtures/autocomplete_sources.rb needs to be run when app/controllers/projects/autocomplete_sources_controller.rb is changed.
    • This can be seen in https://gitlab.com/gitlab-org/gitlab/-/jobs/1772821884#L56.
  4. We re-run tooling/bin/find_changes but this time we pass it ${CHANGES_FILE} ${MATCHED_TESTS_FILE} ${FRONTEND_FIXTURES_MAPPING_PATH}.
    • This time, the script adds frontend fixture files (e.g. tmp/tests/frontend/fixtures-ee/autocomplete_sources/labels.json) that are generated by frontend fixture RSpec files (e.g. spec/frontend/fixtures/autocomplete_sources.rb), to the list of changed files (the tmp/changed_files.txt artifact contains this list of files).
    • The frontend fixture files are mapped from the crystalball/frontend_fixtures_mapping.json with as frontend fixture RSpec file -> JSON frontend fixture.
    • In our example, crystalball/frontend_fixtures_mapping.json contains "spec/frontend/fixtures/autocomplete_sources.rb": ["/builds/gitlab-org/gitlab/tmp/tests/frontend/fixtures-ee/autocomplete_sources/labels.json"], so /builds/gitlab-org/gitlab/tmp/tests/frontend/fixtures-ee/autocomplete_sources/labels.json is added to the list of changed files.
    • This can be seen in https://gitlab.com/gitlab-org/gitlab/-/jobs/1772821884#L55.

jest minimal

The jest:ci:minimal Yarn task uses the --findRelatedTests $(cat tmp/changed_files.txt) (with tmp/changed_files.txt containing /builds/gitlab-org/gitlab/tmp/tests/frontend/fixtures-ee/autocomplete_sources/labels.json), and Jest will be able to run the dependent test.

In our example, that would be spec/frontend/gfm_auto_complete_spec.js, as can be seen in https://gitlab.com/gitlab-org/gitlab/-/jobs/1772757913#L89.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #339343 (closed).

Edited by Rémy Coutable

Merge request reports