Skip to content

Improve test resources delete process

Tiffany Rea requested to merge write-failed-suite-resource-separately into master

What does this MR do and why?

Describe in detail what your merge request does and why.

First half of QA team task https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1161 . The next part of the work is here gitlab-org/quality/pipeline-common!139 (merged)

  • Add logic to write resources from failed suites to a separate file. I.e: append failed to file name.
  • Change test_resources_delete.rb to test_resources_handler.rb because it can now: delete resources, upload (failed) test resources to GCS bucket, and download and remove these files for deletion at a later time.
  • Add new rake tasks and group them under a common namespace test_resources.
  • Using google-cloud-storage gem to connect to GCS since it is used in GCS documentation, should be easier for future maintenance.

My plan is once this is merged:

  • Create another job under clean up stage on live environment pipelines called upload-failed-test-resources to upload files from failed suite run to GCS.
  • Create a dependent delete-failed-test-resources job that would be run in a scheduled pipeline (maybe on every other Sunday 🤔 ) on these environments. This job will download the files from GCS first then run delete.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

Success suite run will produce test-resources-<blah>.json

Screen_Shot_2022-03-08_at_12.21.34_PM

Failed suite run will produce failed-test-resources-<blah>.json

Screen_Shot_2022-03-08_at_12.21.10_PM

These failed files is uploaded to bucket under their environment accordingly

Screen_Shot_2022-03-08_at_12.21.48_PM

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Tiffany Rea

Merge request reports

Loading