Skip to content

Expand /completions endpoint to support Self-Managed

Roy Zwambag requested to merge 417152-self-managed-completions-endpoint into master

What does this MR do and why?

The MR !125401 (merged) introduced a new endpoint /code_suggestions/completions to the API. This endpoint is proxied directly to the model gateway. The original MR aimed to introduce /completions flow for SaaS. To allow it for Self-Managed, this MR introduces necessary adjustments and safely put the change under the feature flag.

How to set up and validate locally

We have to test this with a JWT token from the original /tokens endpoint, since the cdot <-> gitlab-rails connection does not give us valid tokens yet. In the final flow, the sync_seat_link_worker would do steps 1-4 for us.

  1. Make sure you have access to code suggestions, see https://gitlab.com/gitlab-org/gitlab/-/issues/415393
  2. Create a PAT on gitlab.com
  3. Ask for a JWT token from the old endpoint via curl https://gitlab.com/api/v4/code_suggestions/tokens -H'Authorization: Bearer GITLAB.COM_PAT_TOKEN' -X POST
  4. Save the token in your local database Ai::ServiceAccessToken.create(token: token, category: :code_suggestions, expires_at: Date.tomorrow)
  5. Enable feature flags on your local instance:
::Feature.enable(:self_managed_code_suggestions_completion_api)
::Feature.enable(:code_suggestions_tokens_api)
::Feature.enable(:code_suggestions_for_instance_admin_enabled, current_user)
  1. Create a PAT on your local instance
  2. run the query in your console
curl --request POST --header "Authorization: Bearer LOCAL_PAT_TOKEN" --header "Content-type: application/json" --data '{"prompt_version":"1","current_file":{"file_name":"dog.js","content_above_cursor":"class Car:\n    def __init__(self, make, model, year, color):\n        self.make = make\n        self.model = model\n        self.year = year\n        self.color = color\n        self.is_running = False\n        self.speed = 0\n    \n    def start(self):\n        if not self.is_running:\n            self.is_running = True\n            print(\"The car is now running.\")\n        else:\n            print(\"The car is already running.\")\n    \n    def stop(self):\n        if self.is_running:\n            self.is_running = False\n            self.speed = 0\n            print(\"The car has stopped.\")\n        else:\n            print(\"The car is already stopped.\")\n    \n    def accelerate(self, increment):","content_below_cursor":""},"project_id":"23"}' 'http://gdk.test:3000/api/v4/code_suggestions/completions'

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #417152 (closed)

Edited by Aleksei Lipniagov

Merge request reports