chore: use anthropic-on-vertex in staging
What does this merge request do and why?
Part of https://gitlab.com/gitlab-org/gitlab/-/issues/455110
How to set up and validate locally
- Modify your
.env
to includeAIGW_DEFAULT_PROMPTS='{"chat/react": "vertex"}'
- Perform a chat request:
curl -X 'POST' \
'http://localhost:5052/v2/chat/agent' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{"prompt": "What is your name?",
"options": {
"agent_scratchpad": {
"agent_type": "react",
"steps": []
},
"chat_history": [""]
}}'
- In the AIGW output, make sure that the endpoint used is
https://us-east5-aiplatform.googleapis.com/v1/projects/ai-enablement-dev-69497ba7/locations/us-east5/publishers/anthropic/models/claude-3-5-sonnet@20240620:streamRawPredict
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.