Skip to content

feat: add post-processing for suggestions with meaningless asterisks

Pam Artiaga requested to merge 654/strip-asterisks-suggestions into main

What does this merge request do and why?

Codestral FIM is sometimes giving meaningless ********* suggestions. This is an inherent problem with the Codestral model, and we are currently waiting for Mistral to fix it properly. (See: #654 (comment 2160592437))

In the meantime, this MR is a "band-aid" solution for the problem. Essentially, if the Codestral completion starts with a string of asterisks (ignoring leading spaces), then AIGW just returns an empty suggestion.

How to set up and validate locally

It's not easy to replicate this problem, so we can just test that the change does not result in errors by running this example code completion request:

curl "http://gdk.test:5052/v2/code/completions" \
-X POST \
--header "Content-Type: application/json" \
--data '{
    "current_file": {
      "file_name":"main.py",
      "language_identifier":"python",
      "content_above_cursor":"def get_app_name_and_version():\nreturn{",
      "content_below_cursor":"}"
    },
    "stream":false,
    "prompt_version":1,
    "model_provider":"vertex-ai",
    "model_name":"codestral@2405"
  }' \
| json_pp -json_opt pretty,canonical

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Pam Artiaga

Merge request reports

Loading