feat: add post-processing for suggestions with meaningless asterisks
What does this merge request do and why?
Codestral FIM is sometimes giving meaningless *********
suggestions. This is an inherent problem with the Codestral model, and we are currently waiting for Mistral to fix it properly. (See: #654 (comment 2160592437))
In the meantime, this MR is a "band-aid" solution for the problem. Essentially, if the Codestral completion starts with a string of asterisks (ignoring leading spaces), then AIGW just returns an empty suggestion.
How to set up and validate locally
It's not easy to replicate this problem, so we can just test that the change does not result in errors by running this example code completion request:
curl "http://gdk.test:5052/v2/code/completions" \
-X POST \
--header "Content-Type: application/json" \
--data '{
"current_file": {
"file_name":"main.py",
"language_identifier":"python",
"content_above_cursor":"def get_app_name_and_version():\nreturn{",
"content_below_cursor":"}"
},
"stream":false,
"prompt_version":1,
"model_provider":"vertex-ai",
"model_name":"codestral@2405"
}' \
| json_pp -json_opt pretty,canonical
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Pam Artiaga