Refactor public interface of completion module.
What does this merge request do and why?
- Clean up the public interfaces of the completion module: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/merge_requests/70#note_1566379801
- Change
BatchRequestCompletionsVertex
to be single model only: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/merge_requests/71#note_1564291700 Use aiohttp instead of thread: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/merge_requests/68#note_1568527107- Add dataclass for model metadata for cleaner mapping: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/merge_requests/70#note_1566370176
- Move model input token limit to
ModelMeta
: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/merge_requests/66#note_1571018836 - Avoid importing anything from prompt-lib in prompt-engine because prompt-lib should strictly depend on prompt-engine.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
Merge request checklist
-
I've ran the eval_codebase.py
pipeline to validate that nothing is broken. https://console.cloud.google.com/bigquery?project=unreview-poc-390200e5&ws=!1m5!1m4!4m3!1sunreview-poc-390200e5!2sgl_gitlab_experiments!3stest_huggingface_model -
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Hongtao Yang