Define different service names per LLM client
What does this MR do and why?
Gitlab::Llm::Concerns::CircuitBreaker
requires service_name
to be defined.
Before this change, we are only using a single service_name
and that means all client will use a single circuit. If a single provider fails and the circuit opens, all providers will be affected.
To prevent that, since we have different clients (e.g. OpenAI, Vertex, Anthropic), we define a specific service name per client.
This also includes a fix to ExponentialBackoff
concern to raise the correct exception to avoid a NameError
.
This is behind the circuit_breaker
feature flag. See [Feature flag] Enable `circuit_breaker` feature... (#409619 - closed).
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.