Draft: POC- Implement AI slash command to summarise pipeline jobs failures
issue: #468450
What does this MR do and why?
As part of Verify AI hackathon, a POC to summarise pipeline jobs failures
MR acceptance checklist
This MR implements a /summarize_pipeline_failure
slash command to provide a summary of all failed jobs on the current pipeline
Screenshots or screen recordings
AI results
For the initial proof of concept, the AI command tool is asked to perform a basic analysis of an array of failed jobs on the current pipeline.
- the
:status
,:failure_reason
and:job_log
used to analyse failed jobs. - the ai prompt requests an analysts of job logs for the current pipeline to provide a summary
The following is an example of AI output:
Based on the provided information, the summary of the failed jobs in the current pipeline is as follows:
Summary:
There are 4 failed jobs in the current pipeline. Failures:
Job failure due to a non-existent command:
The job log shows that the command "non_existent_command" does not exist, causing the job to fail with a script failure. Job failure due to a non-zero exit code:
The job log indicates that the job was set to exit with a non-zero status code, which caused the job to fail. Job failure due to a syntax error in the script:
The job log shows that there was a syntax error in the script, specifically in the "if" statement, which caused the job to fail. Job failure due to a missing required file:
The job log shows that the job tried to read a file named "non_existent_file.txt", which does not exist, causing the job to fail. In summary, all 4 jobs in the current pipeline failed due to various script-related issues, such as non-existent commands, non-zero exit codes, syntax errors, and missing required files.
How to set up and validate locally
- Setup Duo locally (see the docs from the AI team)
- Ensure you have duo active for a local group that has existing pipelines (
$ GITLAB_SIMULATE_SAAS=1 RAILS_ENV=development bundle exec rake 'gitlab:duo:setup[<test-group-name>]'
) - Navigate to a pipeline with failed jobs
- Get a
clientSubscriptionId
- parsing from any duo chat query - Execute a graphQL query
mutation {
aiAction(
input: {
chat: {
resourceId: "gid://gitlab/Ci::Pipeline/7536",
content: "/summarize_pipeline_failure",
}
clientSubscriptionId: "85fdac4d-8b53-4bfa-a5ab-f843908c7d2d"
}
){
requestId
errors
}
}
- Check duo chat with results on a summary of failed jobs.