Enables parsing nested sections in logs
What does this MR do and why?
This change allows the logs page to display nested log sections.
This is possible when the GitLab Runner produces nested log sections and subsections, such as when multiline commands can be collapsed.
Changelog: changed
How does this work?
Previously, our logs were grouped into sections before rendering them in the frontend. In this design each section had a isClosed
flag that determined if the logs in a given section should be shown. This made having nested sections difficult: We would have to render separate sections in their own data structure and we would have recurse through them to render.
This change separates the "sections" information from the "log lines", so the sections are presented alongside the log lines. A section's isClosed
state still determines if a section is shown, but each line gets rendered independently in the HTML and shown/hidden according to the state of the section it belongs to.
Why this change?
This change implements #409350 (closed), which is needed to render job logs correctly when a runner has FF_SCRIPT_SECTIONS
, which can be nested inside other sections.
Preparation refactor | Adds explicit section footers to logs and harde... (!136170 - merged) | |
Log format change | Enables parsing nested sections in logs (!136186 - merged) | You are here |
Screenshots or screen recordings
Before | After |
---|---|
I also tested on other use cases, to ensure we don't introduce any bugs:
job that gets cancelled | job with a section partially shown | a vanilla command |
---|---|---|
How to set up and validate locally
- Ensure you have installed
GitLab Runner v16.5.0+
. - In a project, create the following pipeline
variables:
FF_SCRIPT_SECTIONS: "true" # this will allow you to collapse arbitrary scripts in the log
stages:
- build
simple-job:
stage: build
script:
- echo 'A simple command'
# A multi-line command that should be collapsible
multi-line-command-job:
stage: build
script:
- |
echo "{
'test': 'data',
'test2': 'data2',
}"
- |
echo "{
'test': 'data',
'test2': 'data2',
}"
# This jobs helps me test long running jobs with many lines that gets cancelled
# You'll have to cancel this job manually to truncate it.
long-running-job:
stage: build
script:
- while sleep 1; do date; done
# A job that triggers "Showing last 500.04 KiB of log" at the top for being too long
showing-last-job:
stage: build
script:
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
- for i in {1..500}; do echo "This is a long line that gets repeated many many many many many many many many many many many many times $i"; done
collapsible_job_single:
variables:
FF_SCRIPT_SECTIONS: "false"
stage: build
script:
- echo -e "\033[0Ksection_start:`date +%s`:my_first_section\r\033[0KHeader of the 1st collapsible section"
- echo 'this line should be hidden when collapsed'
- echo -e "\033[0Ksection_end:`date +%s`:my_first_section\r\033[0K"
collapsible_job_multiple:
variables:
FF_SCRIPT_SECTIONS: "false"
stage: build
script:
- echo -e "\033[0Ksection_start:`date +%s`:my_first_section\r\033[0KHeader of the 1st collapsible section"
- echo 'this line should be hidden when collapsed'
- echo -e "\033[0Ksection_start:`date +%s`:second_section\r\033[0KHeader of the 2nd collapsible section"
- echo 'this line should be hidden when collapsed'
- echo -e "\033[0Ksection_end:`date +%s`:second_section\r\033[0K"
- echo -e "\033[0Ksection_end:`date +%s`:my_first_section\r\033[0K"
- Check the job output of each and confirm they are correct
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #409350 (closed)