[CNG] Revisit how we compute CONTAINER_VERSION
Summary
We use CONTAINER_VERSION
to essentially "map" the versions of various components are in an image, including dependencies, and that given component. The calculation of this value has a few flaws. Refactoring this could make the process much clearer.
- These don't take into account sum changes to underlying / base containers.
- Example: an updated
debian:bullseye-slim
does not necessarily result in a rebuild of all containers built upon that container.
- Example: an updated
- Calculation requires re-computation in every job definition.
- Side note: This is absolutely horrendous for manual builds by contributors.
Current
Right now, the general logic of how we compute CONTAINER_VERSION is as follows
-
TARGET_VERSION
is essentiallygit ls-tree HEAD -- <directory of the component>
and using the third column (object ref) from the output. -
For leaf components (things that doesn't depend on any other images,
gitlab-python
orgitlab-graphicsmagic
for example)export CONTAINER_VERSION=${TARGET_VERSION}${<component version from variables.yml>}
-
For components which depend on other images
-
CONTAINER_VERSION
is essentially shasum of a mix of versions of all the images this component depends on, along withTARGET_VERSION
and component version from variables.yml (if any) -
This requires us to re-compute versions of all the dependencies using
get_version
in the script. For example, this is what we do for gitlab-shellruby_version=$(get_version gitlab-ruby) go_dir_version=$(get_version gitlab-go) go_version=($(echo -n "$BASE_VERSION$go_dir_version$GO_VERSION" | sha1sum)) logger_target_version=$(cat artifacts/container_versions/gitlab-logger_tag.txt) export git_container=$(cat artifacts/container_versions/git-base_tag.txt) gomplate_version=$(cat artifacts/container_versions/gitlab-gomplate_tag.txt) export GITLAB_LOGGER_VERSION=($(echo -n "$go_version$logger_target_version$GITLAB_LOGGER_VERSION$(date -u +%D)" | sha1sum)) export CONTAINER_VERSION=($(echo -n "$BASE_VERSION$go_version$git_container$gomplate_version$TARGET_VERSION$GITLAB_SHELL_VERSION$(date -u +%D)" | sha1sum))
-
Proposal
Change computation of CONTAINER_VERSION
to something like the following
export TARGET_VERSION=$(get_target_version)
export DEPS_VERSIONS=$(cat artifacts/container_versions/*.txt) # Because of needs, only the necessary artifacts will be present here.
export CONTAINER_VERSION=$(echo -n ${TARGET_VERSION}${component_version_from_variables_yml}${DEPS_VERSIONS} | sha1sum)
echo -n ${CONTAINER_VERSION} > artifacts/container_versions/${CI_JOB_NAME}.txt
By this method, the earlier gitlab-shell
example becomes
export TARGET_VERSION=$(get_target_version)
export DEPS_VERSIONS=$(cat artifacts/container_versions/*.txt)
export CONTAINER_VERSION=$(echo -n ${TARGET_VERSION}${GITLAB_SHELL_VERSION}${DEPS_VERSIONS} | sha1sum)
PS: I would also like us to rename TARGET_VERSION
to something else. Something that clearly tells us that this is the git object applicable to the component.