Log the included file count of CI pipelines that exceed the limit
What does this MR do and why?
In #367150 (closed), we discovered a bug that allows the includes limit (currently 100) to be bypassed. In order to gauge how many customers would be affected by enforcing the limit, this MR logs the included file count of pipelines that currently exceed the limit.
In this MR, we leverage the existing pipeline creation logger infrastructure. The logs are outputted to gitlab/log/application_json.log
.
How to set up and validate locally
- Enable the logging feature flag locally:
bundle exec rails c
Feature.enable(:ci_pipeline_creation_logger)
- So that we don't have to trigger the log by exceeding 100 files, update the limit to 1 in
lib/gitlab/ci/config/external/context.rb
.
MAX_INCLUDES = 1
- Tail the log file in your
gitlab
directory:
tail -f log/application_json.log
- Create a
template1.yml
file in your project root with the following content:
job:
script: echo "test"
- Update your
.gitlab-ci.yml
with the following content:
include:
- local: 'template1.yml'
- Run the pipeline and observe that a log containing
"pipeline_includes_count":2
is produced. Note that the count should be 2 because it counts itself (.gitlab-ci.yml
) as well as in the included local file (template1.yml
).
- (Optional). You can also observe that if you update
.gitlab-ci.yml
without an include file, a log entry is not produced as the file count does not exceed the limit (1).
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #367150 (closed)
Edited by Leaminn Ma