Extend VertexAi::Client to add text/code/message_chat methods
What does this MR do and why?
This MR updates Gitlab::Llm::VertexAi::Client
to:
- Add
text
method to utilizetext-bison
model - Add
code
method to utilizecode-bison-001
model - Add
messages_chat
method to utilize multiturnchat-bison-001
model
The newest models endpoints live under different URLs comparing to the default codechat endpoint that makes a request to Gitlab::CurrentSettings.current_application_settings.tofa_url
. Since the endpoint URL is currently hardcoded in the application settings, I am replacing parts of the string during runtime to make each corresponding url valid. For example, for :text
endpoint:
- Removing
-preprod-
from hostname - Replacing
endpoints
withpublishers/google/models
- Model name from
codechat-bison-001
totext-bison
Each model now has slightly different URL endpoint updates and has a corresponding ModelConfigurations::*
class.
Mentions https://gitlab.com/gitlab-org/gitlab/-/issues/407336 & https://gitlab.com/gitlab-org/gitlab/-/issues/410112
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
Assuming you have Vertex AI API setup locally:
- Open rails console
> Gitlab::Llm::VertexAi::Client.new(User.first).chat(content: 'Hi!')
> Gitlab::Llm::VertexAi::Client.new(User.first).messages_chat(content: [{author: 'user', content: 'Who is GitLab CEO?'},{author: 'content', 'content': 'Sytse Sijbrandij'}, {author:'user', content:'what is his middlename'}])
> Gitlab::Llm::VertexAi::Client.new(User.first).code(content: 'function()')
> Gitlab::Llm::VertexAi::Client.new(User.first).text(content: 'Hi!')
- Verify responses are coming from the corresponding models
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.