Adjust model names to map the exact model names
What does this MR do and why?
According to the findings detailed here: #471084 (comment 2015613809), we need to specify the exact model name to be used because we support multiple models within the same family. For example, we have models like mixtral7b
and mixtral22b
, as well as codellama13
and codellama13b-code
. Currently, we only send the family name of the model, such as codegemma
, which could lead to issues if a customer uses Codegemma-code
for completion and Codegemma
for generation.
Here’s a potential problem:
- model_name: codegemma
litellm_params:
model: ollama/codegemma7b
api_base: http://localhost:11434
- model_name: codegemma
litellm_params:
model: ollama/codegemma2b
api_base: http://localhost:11434
In this case, using the same key codegemma
for both models can cause conflicts, as we are sending the same key for different model configurations.
To address this, the current MR should specify the exact model name, which will resolve this issue and ensure that each model is correctly identified and used. This change will prevent conflicts by clearly differentiating between models from the same family.
UI/UX
Related to: #475212 (closed) and #474402 (comment 2023551898)