Skip to content

Increase the Vertex AI token limit to the new 2048 cap

Gregory Havenga requested to merge ghavenga-boost-vertex-token-limit into master

It has been noted that the Google Vertex API has increased the token limit from 1024 to 2048, giving us an increased degree of flexibility in the size of prompts provided to the API and the responses possible.

This MR boosts this default value, as it's unlikely that GitLab will benefit from a reduced token limit associated with any particular requests.

Merge request reports