Add skip_cache option
What does this MR do and why?
Add skip_cache option
It's possible that some AI chain tools use completion service (e.g. SummaryComments tool), in this case we want to avoid storing request/response in cache because it's only intermediate step.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
- Ask a question which uses SummarizeComments tool (e.g. "what is summary of comments on flightjs/Flight#1 issue?")
- Or alternatively can be verified from console with something like:
res = Gitlab::Llm::Chain::Agents::ZeroShot::Executor.new(user_input: 'What is summary of comments on flightjs/Flight#1 issue?', tools: [Gitlab::Llm::Chain::Tools::IssueIdentifier,Gitlab::Llm::Chain::Tools::SummarizeComments], context: ::Gitlab::Llm::Chain::GitlabContext.new(current_user: user,container:nil,resource: <issue>, ai_request: ::Gitlab::Llm::Chain::Requests::Anthropic.new(user))).execute
- Verify in rails console that no "intermediate" messages are stored in cache using
Gitlab::Llm::Cache.new(user).find_all
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Jan Provaznik