Add release.gitlab.net deployment validator for patch release pipeline
What does this MR do and why?
After we tag the versions during a patch release, the latest version package gets deployed to release.gitlab.net.
Currently, the release manager has a manual task on the patch release task issue to check the version deployed to release.gitlab.net/help.
This will automate that process, so that this validator will run 2 hours and 30 minutes after the versions have been tagged.
(I was looking at past release timelines for an appropriate wait time, but we've had a few hiccups in the past few patch releases during this phase, so I'm not sure if that's too short/long in actuality.)
Addresses: gitlab-com/gl-infra/delivery#20235 (closed)
Content
- Add CI job to call validator awhile after tagging
- Add release deployment validator for patch release
Testing
Pipeline creation
This was tested in this repo (gitlab-org/release-tools
), just to see that the pipeline gets created as we expect it.
With |
With |
---|---|
This verification gets created with the other tagging jobs, and should run at the end. |
|
Configuration testing
This was to test the configuration of the pipeline (the job order).
Step | Screenshot |
---|---|
For testing - Echo a log line instead of actually doing sth (Testing commit)
|
|
Initialized as manual pipeline (Pipeline)
|
|
After |
|
Slack notification to signal start of tag process, as per usual | |
|
|
|
Execution Testing
This test is simply to test the execution of the verify_release_deploy
job. Mainly we're looking for the slack notifications on success and failure verification cases.
Step | Screenshot |
---|---|
For testing - mimic a passing validation (Testing commit)
|
This is the version that's going to be returned as the latest patch release. On the day of the tagging, this would actually be the first of the to-be-tagged versions. Since today's not the tagging day, this would normally fail, so to simulate the day of the tagging, we hardcoded the version to be the same as the version deployed to the release environment. |
Same as above, set the pipeline vars for testing (Pipeline)
|
The pipeline for testing is much shorter, since we're not re-testing the |
Slack notification | |
The job successfully completed, and posts a successful execution message to slack. |
|
Now let's test the failure case by changing the hardcoding the returned version to "17.3.0" (commit) |
|
Same as above, create a new testing pipeline (Pipeline) |
|
The slack notification to signal the start is the same as above. | |
The job "successfully" fails, and here is the slack failure notification |
Test retrying
-
Test the failure case by changing the hardcoding the returned version to "17.3.0" (commit)
-
Then execute:
[1] pry(main)> ReleaseTools::Security::ReleaseDeployValidator.new.execute 2024-08-27 09:10:41.011937 D [security] ReleaseTools::GitlabReleaseClient -- [HTTParty] [2024-08-27 09:10:41 +0200] 200 "GET https://release.gitlab.net/api/v4/version" - 2024-08-27 09:10:41.012551 W [security] ReleaseTools::Security::ReleaseDeployValidator -- Release.gitlab.net is not running the latest patch version. Retrying. 2024-08-27 09:11:41.327700 D [security] ReleaseTools::GitlabReleaseClient -- [HTTParty] [2024-08-27 09:11:41 +0200] 200 "GET https://release.gitlab.net/api/v4/version" - 2024-08-27 09:11:41.327805 W [security] ReleaseTools::Security::ReleaseDeployValidator -- Release.gitlab.net is not running the latest patch version. Retrying.
As you can see, the two retries are one minute away from each other.