Skip to content

Support mixed vendored and self-hosted models

Igor Drozdov requested to merge id-mixed-vendored-self-hosted into master

What does this MR do and why?

Related issue: Support a mix of self-hosted and GitLab-hosted ... (#463917 - closed)

AI feature settings have the following UI/UX:

Screenshot_2024-09-04_at_10.12.40

The idea is:

  • When Self-hosted model option is chosen and configured, the features send requests to the self-managed AI Gateway with the params related to the self-hosted model
  • When AI vendor option is chosen, the features send requests to the Cloud connector

It will allow non-air-gapped customers to either use self-hosted models for a feature or use the Gitlab vendor models

Implementation details

  • Currently, AI Gateway URL can be configured either via AI_GATEWAY_URL env var or via Gitlab.config.cloud_connector.base_url
  • This feature implies that customers use AI_GATEWAY_URL to configure their self-managed AI Gateway and Gitlab.config.cloud_connector.base_url to point to the cloud connector url
  • When a feature is self-hosted, then AI_GATEWAY_URL value is used, when it's vendored then Gitlab.config.cloud_connector.base_url is used
Edited by Igor Drozdov

Merge request reports

Loading