gradual rollout code gen

What does this MR do and why?

Right now, the template in the prompt registry for AIGW determines the model version used (see example )

Before moving code generation to the AIGW, we used feature flags to rollout new models gradually to users so that we could easily revert if there was an issue.

We need the ability to do this with the prompt registry version of code generation, and I am sure other tools could benefit from this functionality.

• In this MR we add the ability to gradual rollout new model/provider for code generation through feature flag.
• In this MR we use ff to rollout claude-3.5-sonnet-20241022

References

• #502690
• #503249

How to set up and validate locally

There are two versions of rails: before the MR(old) and after the MR(new).

We need to make sure the AIGW can correctly handle both the old request and new request. The following are the local test:

1. old rails -> new AIGW

use gitlab master branch:

tiangao@tgao--20240202-M7DJJ gitlab % pwd
/Users/tiangao/tgao_gitlab/gitlab-development-kit/gitlab
tiangao@tgao--20240202-M7DJJ gitlab % git checkout master
Already on 'master'

use AIGW feature branch:

(ai-gateway-py3.11) tiangao@tgao--20240202-M7DJJ gitlab-ai-gateway % pwd 
/Users/tiangao/tgao_gitlab/gitlab-development-kit/gitlab-ai-gateway
(ai-gateway-py3.11) tiangao@tgao--20240202-M7DJJ gitlab-ai-gateway % git checkout 502690-gradual_rollout_of_new_code_gen_model
Already on '502690-gradual_rollout_of_new_code_gen_model'

code generation works fine:

2. new rails -> new AIGW

use gitlab feature branch

tiangao@tgao--20240202-M7DJJ gitlab % pwd
/Users/tiangao/tgao_gitlab/gitlab-development-kit/gitlab
tiangao@tgao--20240202-M7DJJ gitlab % git checkout 502690-gradual_rollout_code_gen
Already on '502690-gradual_rollout_code_gen'

use AIGW feature branch:

(ai-gateway-py3.11) tiangao@tgao--20240202-M7DJJ gitlab-ai-gateway % pwd 
/Users/tiangao/tgao_gitlab/gitlab-development-kit/gitlab-ai-gateway
(ai-gateway-py3.11) tiangao@tgao--20240202-M7DJJ gitlab-ai-gateway % git checkout 502690-gradual_rollout_of_new_code_gen_model
Already on '502690-gradual_rollout_of_new_code_gen_model'

2.1 test failover

[4] pry(main)> Feature.enable(:incident_fail_over_generation_provider)
=> true
[5] pry(main)> Feature.enabled?(:incident_fail_over_generation_provider)
=> true

code generation works fine:

2.1 test claude_3_5_sonnet_20240620_for_code_gen

[4] pry(main)> Feature.disable(:incident_fail_over_generation_provider)
=> true
[5] pry(main)> Feature.enabled?(:incident_fail_over_generation_provider)
=> false

[4] pry(main)> Feature.enable(:claude_3_5_sonnet_20241022_for_code_gen)
=> true
[5] pry(main)> Feature.enabled?(:claude_3_5_sonnet_20241022_for_code_gen)
=> true

code generation works fine

2.2 test claude_3_5_sonnet_20241022_for_code_gen

[4] pry(main)> Feature.disable(:incident_fail_over_generation_provider)
=> true
[5] pry(main)> Feature.enabled?(:incident_fail_over_generation_provider)
=> false

[4] pry(main)> Feature.disable(:claude_3_5_sonnet_20241022_for_code_gen)
=> true
[5] pry(main)> Feature.enabled?(:claude_3_5_sonnet_20241022_for_code_gen)
=> false

code generation works fine

Related to #502690