Skip to content

Use ApplicationSettings for AI Gateway URL

Jessie Young requested to merge jy-config-for-ai-gateway-url into master

What does this MR do and why?

Adds a field so that a custom AI Gateway URL can be set via application settings.

Previously, it was always set via an ENV var. This seems easy enough but is the top reason that developers have difficulty setting up the AI Gateway locally. The result of an incorrectly set AI Gateway URL is an A1001 error and we get reports of these internally weekly.

In addition, this config is more user-friendly for SM instances that are using self-hosted AI Gateways. The custom models team will later add the ability to set this setting in the UI: Allow users to configure AI Gateway endpoint in... (#473143)

Because this is behind a default disabled feature flag, I have not updated the docs to indicate that this is how the value should be set. When the FF is removed we will update the docs to reflect this new setting.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Confirmed locally that with the FF Enabled and setting updated, Chat still work! (instructions below)

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Set up Duo Chat in your GDK: https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/gitlab_ai_gateway.md
  2. In rails console enable the feature flag 
    Feature.enable(:duo_ai_gateway_url_in_application_settings)
  3. In rails console, set the Application Settings value to the URL of the local AI Gateway (below command uses default URL of locally hosted AI Gateway, but yours might be different if you've configured it to live elsewhere) 
    ::Gitlab::CurrentSettings.current_application_settings.update!(duo_ai_gateway_url:"http://0.0.0.0:5052")
  4. Test that Duo Chat still works by visiting a Duo-enabled group and selecting the Duo Chat button
Edited by Jessie Young

Merge request reports