Log generate_cube_query prompts
Problem to solve
See discussion for more context: #442058 (comment 1806551057)+
The generate_cube_query
AI action is not currently logged, so we have no way to find and fix problems with our prompt.
We would also like to relate customer feedback back to their specific prompt to enable us to better debug / respond to issues.
Proposed solution
Log the generate_cube_query prompts, similar to how GitLab Duo Chat logs are handled.
Additionally, allow passing a UUID to be logged with the prompt, so that it can be associated back to feedback / errors etc.
If we have errors while generating the prompt, include the UUID with the error data sent to Sentry.
Implementation plan
- Use
Gitlab::Llm::Logger
to intercept calls to Vertex AI. (Example of Chat doing that is here: https://gitlab.com/gitlab-org/gitlab/-/blob/bc668afd4ec23d55c58470b5a25f52eb99f00bd3/ee/lib/gitlab/llm/chain/tools/tool.rb#L36-37)- Probably in ee/lib/gitlab/llm/vertex_ai/client.rb, use logger.info_or_debug to store prompt and response information in kibana. This will respect any disabled feature flags set globally.
Edited by Max Woolf