Skip to content

Adjust model names to map the exact model names

Mohamed Hamda requested to merge adjust-model-names into master

What does this MR do and why?

According to the findings detailed here: #471084 (comment 2015613809), we need to specify the exact model name to be used because we support multiple models within the same family. For example, we have models like mixtral7b and mixtral22b, as well as codellama13 and codellama13b-code. Currently, we only send the family name of the model, such as codegemma, which could lead to issues if a customer uses Codegemma-code for completion and Codegemma for generation.

Here’s a potential problem:

- model_name: codegemma
  litellm_params:
    model: ollama/codegemma7b
    api_base: http://localhost:11434
- model_name: codegemma
  litellm_params:
    model: ollama/codegemma2b
    api_base: http://localhost:11434

In this case, using the same key codegemma for both models can cause conflicts, as we are sending the same key for different model configurations.

To address this, the current MR should specify the exact model name, which will resolve this issue and ensure that each model is correctly identified and used. This change will prevent conflicts by clearly differentiating between models from the same family.

UI/UX

Screenshot_2024-08-01_at_11.46.47

Related to: #475212 (closed) and #474402 (comment 2023551898)

Edited by Susie Bitters

Merge request reports

Loading