Use Code Gecko instead of Codestral in APAC
Context
We recently updated the AIGW to support Codestral-on-Vertex code completion requests. See: Add Codestral on Vertex support for code comple... (#564 - closed). Codestral is intended as a replacement for Code Gecko. However, post-deployment latency testing showed that Codestral is significantly slower than Code Gecko in APAC locations. This is because of a discrepancy in which GCP locations the models are supported.
This shows comprehensive list of locations where Code Gecko and Codestral are supported, in summary:
For Code Gecko (Codey):
- USA / North America: Multi-region,
us-central1,us-west4,us-east4,us-west1,northamerica-northeast1 - Europe: EU multi-region,
europe-west1,europe-west2,europe-west3,europe-west4,europe-west9 - APAC:
asia-northeast1,asia-southeast1,asia-northeast3
And for Codestral:
- USA: Multi-region
- Europe: EU multi-region,
europe-west4
From the list above, we can see that the main difference is in the APAC region, where Code Gecko is supported but not Codestral.
Proposal
We are already in conversation with Google about adding Codestral support in APAC.
In the meantime, we should keep using Code Gecko in APAC regions.