Skip to content

Add automation and docs for observability e2e jobs

Daniele Rossetti requested to merge drosse/add-observability-e2e-tests-labels into master

What does this MR do and why?

We have two manual CI jobs that triggers e2e tests on the opstrace CI (https://gitlab.com/gitlab-org/opstrace/opstrace/-/pipelines).

  • e2e:observability-backend-main-branch
  • e2e:observability-backend

image

So far we have been relying on MRs author to trigger the jobs manually, but this is prone to errors as

  • one might forget to do so
  • one might not know that there are external e2e tests running
  • one might not realise they are touching observability-related code
  • one might run it the first time, and forget to run it again on following revisions

We had several instances of opstrace pipeline getting blocked because of the reasons above. And given the team is expanding, I thought some automation and documentation could help.

As an attempt to make the process more smooth, I've create two new labels (https://gitlab.com/gitlab-org/gitlab/-/labels?subscribed=&sort=relevance&search=pipeline%3Arun-observability-e2e):

Depending on which labels get applied to an MR, every pipeline run will run the associated e2e job.

In addition, any change to observability related files, will automatically run the e2e:observability-backend-main-branch

The workflow could look like this:

  • any changes to observability code should trigger the e2e:observability-backend-main-branch job
  • if e2e:observability-backend-main-branch fails, it could mean that the MR made changes that requires opstrace e2e tests to be updated. if that’s the case, one should create a branch on the opstrace repo and fix the tests.
  • Once the branch with fixed tests is pushed, one can add the pipeline:run-observability-e2e-tests-current-branch label to the corresponding gitlab MR. This will trigger the e2e:observability-backend e2e job, and run tests against the opstrace branch with the fixes.
  • If the tests pass we can merge the gitlab MR first, and then the opstrace MR with updated tests and everyone should be happy!

In addition, one could also manually add pipeline:run-observability-e2e-tests-main-branch to make sure the MR will keep running e2e tests against main ( could be useful in case of changes to files that are not tracked by the code pattern rules)

Related to Improve opstrace e2e test integration with gitl... (gitlab-org/opstrace/opstrace#2609 - closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

NA

How to set up and validate locally

  • Add no labels. Confirm the jobs are available to run manually from the test stage menu
  • Add pipeline:run-observability-e2e-tests-main-branch to this MR, trigger a pipeline run and confirm that the e2e:observability-backend-main-branch runs
  • Add pipeline:run-observability-e2e-tests-current-branch to this MR, trigger a pipeline run and confirm that the e2e:observability-backend runs ( it will fail in this case since it requires a branch in the opstrace repo to be named as this one. but this is outside the scope of this testing )
Edited by Daniele Rossetti

Merge request reports