Draft: Summarize notes draft
What does this MR do and why?
Related to https://gitlab.com/gitlab-org/gitlab/-/issues/404510
high level plan
This is how I see it, open to debate any point you may think can be changed.
This list may continue to be extended as I keep thinking about it. I will strike out items that are done or changed.
- Required
- ensure this works for issue, work item, MR, epic!
-
make sure this works on large enough resources
- This may require AI Token limit/transformer layer as we may need to split the input into multiple chunks or take other strategies to fit into model's tokens limits
- Can be a follow-up but not GA in any form, not even alpha
- opt-in setting at namespace level
- store api keys at namespace level
-
enable ai experimentation FFs per namespace- Doneas a faster alternative to namespace settings, while we implement those
-
ability to turn off AI BG jobs in case of total AI shut down- Done. The worker is behindopenai_experimentation
FF that is used as a global on/off FF. -
make sure this works on large enough resources
- This may require AI Token limit/transformer layer as we may need to split the input into multiple chunks or take other strategies to fit into model's tokens limits - MR in works: !117176 (closed)
- Can be a follow-up iteration preferably before alpha so that we can effectively iterate
- retry & backoff mechanism, i.e. circuit breaker mechanism - MR in review !117210 (merged)
- a way to switch/test LLM configurations without having to deploy. E.g. store LLM configurations and be able to switch between them though chatops for instance?
- track data on user satisfaction(ai performance) in relation to specific LLM configuration
- Iterative improvements
- opt-in setting at namespace level - MR in works !116984 (closed)
- store api keys at namespace level
- caching layer
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Alexandru Croitor