Automate for which supported distro releases we create packages.

What does this MR do?

This MR automates selection of the OS distribution releases for which we publish gitlab-runner packages. With this change, the distro releases for which we publish runner packages (deb ad rpm) will no longer be hard coded in the RPM_PLATFORMS and DEB_PLATFORMS Makefile variables (and require manual updating to keep up with packagecloud). Instead, the distro releases will be determined by scripts/packagecloud-releases/main.go and be based on whatever releases our packagecloud instance currently supports.

What we loose in exchange for this automation is:

• control over exactly when EOL releases stop being supported. The current value of NUM_DISTRO_RELEASES (6) is more eager to drop old releases than our previous manual approach (no surprise). But we were claiming to support already EOLed releases, so this is fine and good.
• Explicit documentation of exactly what releases are supported. This was also out of sync with reality, so this is acceptable if not ideal. We could, if we choose to, update the documentation manually at each release based on what the script outputs.
• The ability to selectively skip distro/release if for whatever reason we won't want to support one.

I'd argue this is a worthwhile tradeoff for keeping up to date with packagecloud.

Why was this MR needed?

Currently we have keep up to date with packagecloud releases manually. This means:

• Manually checking when our packagecloud instance (at https://packages.gitlab.com) has been updated, which brings support for new OS distribution versions.
• Manually update the distro versions to which we publish packages. i.e. update RPM_PLATFORMS and DEB_PLATFORMS here and here

So far we've only been doing this reactively when users file issues requesting we add support for the latest distro versions.

What's the best way to test this MR?

> PACKAGECLOUD_TOKEN="..." PACKAGE_CLOUD_URL="https://packages.gitlab.com" ci/release_packagecloud stable
> PACKAGECLOUD_TOKEN="..." PACKAGE_CLOUD_URL="https://packages.gitlab.com" ci/release_packagecloud bleeding

You'll probably get a bunch of ./ci/push_packagecloud: line 17: package_cloud: command not found errors if ~/.gem/ruby/3.0.0/bin isn't in your path, but that's fine. The relevant part is the rpm distros to package: and deb distros to package: lists.

Currently the output looks like this:

Click to expand

> PACKAGECLOUD_TOKEN="" PACKAGE_CLOUD_URL="https://packages.gitlab.com" ci/release_packagecloud stable
# Installing packagecloud dependencies...
gem install package_cloud --version "~> 0.3.0" --no-document
WARNING:  You don't have ~/.gem/ruby/3.0.0/bin in your PATH,
	  gem executables will not run.
Successfully installed package_cloud-0.3.11
1 gem installed
# Sending Debian compatible packages...
deb distros to package: ubuntu/jammy ubuntu/impish ubuntu/hirsute ubuntu/groovy ubuntu/focal ubuntu/eoan debian/trixie debian/bookworm debian/bullseye debian/buster debian/stretch debian/jessie linuxmint/una linuxmint/uma linuxmint/ulyssa linuxmint/ulyana linuxmint/tricia linuxmint/tina raspbian/bullseye raspbian/buster raspbian/stretch raspbian/jessie raspbian/wheezy
====================
ubuntu/jammy
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ubuntu/impish
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ubuntu/hirsute
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ubuntu/groovy
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ubuntu/focal
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ubuntu/eoan
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
debian/trixie
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
debian/bookworm
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
debian/bullseye
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
debian/buster
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
debian/stretch
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
debian/jessie
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
linuxmint/una
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
linuxmint/uma
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
linuxmint/ulyssa
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
linuxmint/ulyana
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
linuxmint/tricia
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
linuxmint/tina
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
raspbian/bullseye
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
raspbian/buster
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
raspbian/stretch
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
raspbian/jessie
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
raspbian/wheezy
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
make: [Makefile:232: packagecloud-deb] Error 127 (ignored)
# SENDING REDHAT COMPATIBLE PACKAGES...
rpm distros to package: el/9 el/8 el/7 el/6 el/5 fedora/36 fedora/35 fedora/34 fedora/33 fedora/32 fedora/31 ol/8 ol/7 ol/6 ol/5 amazon/2022 amazon/2 amazon/1
====================
el/9
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
el/8
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
el/7
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
el/6
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
el/5
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
fedora/36
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
fedora/35
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
fedora/34
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
fedora/33
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
fedora/32
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
fedora/31
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ol/8
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ol/7
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ol/6
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
ol/5
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
amazon/2022
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
amazon/2
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
====================
amazon/1
====================
./ci/push_packagecloud: line 17: package_cloud: command not found
make: [Makefile:236: packagecloud-rpm] Error 127 (ignored)

What are the relevant issue numbers?

• Closes #29451 (closed)
• https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/16524
• #29336 (closed)
• Closes #29429
• Closes #29423 (closed)
• Closes #29420 (closed)
• Closes #29123 (closed)

Notes

• Best reviewed commit-at-a-time