Make `build_package` manual action useful for QA testing
Description
We already have build_package
manual action in CE / EE merge requests. We can use this button to build a complete monolithic docker image of GitLab, that we later use to run QA tests against.
Unfortunately this approach has some drawbacks, we rarely use it to run QA tests in merge requests.
In this issue I would like to summarize what we need to make it production ready, what pieces of the puzzle are missing, and to start discussion about an effort needed to make this QA testing approach useful for developers.
Tasks
In GitLab, we want to make multi-project pipelines feature useful as well. This is a perfect opportunity to think about next iteration of multi-project pipelines because the implementation is still very limited, and build_package
button does not use this feature because of that.
Let's try to describe two approaches for making build_package
button useful. The first approach is to extend multi-project pipelines feature to make it possible to use it, the second approach is shipping a kludge / boring solution and using API polling. We can think about latter if there is no intention to proceed to second iteration of multi-project pipelines from product perspective.
General tasks
-
Rename build_package
topackage:qa
to make it more clear that it runs QA tests -
Make it possible to see feedback in CE / EE merge requests ➡ https://gitlab.com/gitlab-org/gitlab-ce/issues/40431 -
Trigger all QA test scenarios in Omnibus (we do not test everything) ➡ omnibus-gitlab#3010 (closed) -
Trigger QA pipeline in GitLab QA project instead of Omnibus (single source of truth) ➡ https://gitlab.com/gitlab-org/build/team-tasks/issues/70 -
Document how and when to use package:qa
button, announce it on a team call -
Push QA specs image into registry as well ➡ omnibus-gitlab!2028 (merged) (also see gitlab-qa#81 (closed)) -
Make sure our Container Registry storage does not explode (expire S3 buckets?) ➡ gitlab-org/gitlab-ce#20247➡ https://gitlab.com/gitlab-org/gitlab-ce/issues/40747 -
Setup QA notifications -> #development channel for failures on master/nightly, #qa channel for failures in MRs -
Running package:qa
action before merging a merge request by making it a blocking manual action at the end of a pipeline
API polling
-
Extend script that triggers Omnibus pipeline to use pipelines API to provide feedback in CE / EE merge requests
Next iteration of multi-project pipelines (next iteration probably)
-
Add support for passing variables when triggering a pipeline using JOB_TOKEN
(multi-project pipeline feature) -
Implement feedback mechanism when triggering multi-project pipeline -
Developers aren't able to run pipelines on master branch in Omnibus ➡ gitlab-qa#63 (closed)