Skip to content

feat: fixes self-provided claude_3 prompt

Eduardo Bonet requested to merge claude_3_generations_v2 into main

What does this merge request do and why?

feat: fixes self-provided claude_3 prompt

The generations prompt for claude_3 now matches other prompts for self-hosted models, like mistral

Evaluation results can be found in https://smith.langchain.com/o/477de7ad-583e-47b6-a1c4-c4a0300e7aca/datasets/983c9d6a-9936-4999-bf49-6f423253fd7b/compare?selectedSessions=3539a758-7c8c-4749-a7cf-3979d9e954dc&baseline=undefined

Obtained with:

poetry run eli5 code-suggestions evaluate --dataset="code_generation_mbpp_all_sanitized" --source="gitlab" --experiment-prefix="bonet-anthropic-claude-3" --intent=generation

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Eduardo Bonet

Merge request reports