Improve logging for job artifact creation and deletion as well as the corresponding changes to the statistics counter
Context
We are seeing project statistics build_artifacts_size
drifting to become incorrect. The value should be equal to the sum of the size
fields in ci_job_artifacts
of a project's job artifacts plus the current amount in the Redis counter. However, there has been instances where this value would no longer be equal.
To improve our ability to debug this problem, we need to be able to correlate each job artifact creation and deletion to its corresponding change on the project statistics counter.
There are currently a few places where we are not logging appropriately:
- in
DestroyBatchService
andDestroyAssociationsService
, the amount logged is the lump sum of all the job artifact sizes being deleted. We need to break this down per job artifact. https://gitlab.com/gitlab-org/gitlab/blob/33689f7203b7d0c0a7e2b32f045ae6aceb1dada7/app/models/concerns/counter_attribute.rb#L100-106 - In
Mutations::Ci::JobArtifact::Destroy
, job artifact is deleted directly, without logging. - in
Projects::ArtifactsController#destroy
, job artifact is deleted directly, without logging.
Edited by Albert