feat: introduce base evaluators to build custom ones
What does this merge request do and why?
This MR introduces base evaluators, which provide a foundation for easily building new custom evaluators, including those that use LLMs as judges.
This MR mainly extracts the base classes introduced in !108 (merged) into a subpackage common for all AI feature evaluation pipelines.
Please, check the updated docs for the additional details - https://gitlab.com/gitlab-org/ai-powered/eli5/-/blob/98545ad28bb24eb1b7000c971d2a01bc567d13a5/doc/evaluators/index.md.
This MR is the first step towards consolidating our evaluation pipelines in the ELI5 project.
Ref: #35
How to set up and validate locally
- Check out to this merge request's branch.
- Install dependencies.
poetry run install
- Check the existing commands ELI5 provides:
poetry run eli5 --help
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Alexander Chueshev