Skip to content

chore: use anthropic-on-vertex in staging

Alejandro Rodríguez requested to merge anthropic-vertex-staging into main

What does this merge request do and why?

Part of https://gitlab.com/gitlab-org/gitlab/-/issues/455110

How to set up and validate locally

  • Modify your .env to include AIGW_DEFAULT_PROMPTS='{"chat/react": "vertex"}'
  • Perform a chat request:
curl -X 'POST' \
   'http://localhost:5052/v2/chat/agent' \
   -H 'accept: application/json' \
   -H 'Content-Type: application/json' \
   -d '{"prompt": "What is your name?",
        "options": {
            "agent_scratchpad": {
                "agent_type": "react",
                "steps": []
            },
            "chat_history": [""]
        }}'
  • In the AIGW output, make sure that the endpoint used is https://us-east5-aiplatform.googleapis.com/v1/projects/ai-enablement-dev-69497ba7/locations/us-east5/publishers/anthropic/models/claude-3-5-sonnet@20240620:streamRawPredict

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.

Merge request reports

Loading