Pin Vertex AI model versions
What does this merge request do and why?
Pin Vertex AI model versions.
code-bison
and text-bison
model no longer supports @latest
versioning
schema and cause 404 errors. This change pins all Vertex AI model
versions to the latest known versions 002
. This would also allow
better predictability and reproducibility of latency and quality moving forward.
To see the latest model versions, please follow the official doc from Google.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
- Check out to this merge request's branch.
- Ensure a local Docker image built successfully.
docker buildx build --platform linux/amd64 \ -t ai-gateway:dev .
- Run a local service on Docker.
docker run --platform linux/amd64 --rm \ -p 5052:5052 \ -e AUTH_BYPASS_EXTERNAL=true \ -v $PWD:/app -it ai-gateway:dev
- Run a cURL command to test code completions
curl --request POST \ --url http://codesuggestions.gdk.test:5052/v2/completions \ --header 'Content-Type: application/json' \ --header 'X-Gitlab-Authentication-Type: oidc' \ --data '{ "prompt_version": 1, "project_path": "awesome_project", "project_id": 23, "current_file": { "file_name": "main.go", "content_above_cursor": "func (h* HotelHandler) HandleGetHotels(c *fiber.Ctx) error {\n\thotels, err := h.Store.Hotels.GetHotels(c.Context(), nil)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn", "content_below_cursor": "}" } }' { "id": "id", "model": { "engine": "vertex-ai", "name": "code-gecko@002", "lang": "go" }, "experiments": [ { "name": "exp_truncate_suffix", "variant": 0 } ], "object": "text_completion", "created": 1702604943, "choices": [ { "text": " c.JSON(hotels)\n", "index": 0, "finish_reason": "length" } ] }
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Closes #383
Edited by Tan Le