Skip to content

Pull examples from LangSmith

Alexander Chueshev requested to merge ac/sync-langsmith-pull into main

What does this merge request do and why?

This MR provides a new CLI command to sync local datasets with LangSmith by pulling changes. Part of #40 (comment 2143234528). We will wrap this CLI command into a CI job to automate the process.

How to set up and validate locally

  1. Check out to this merge request's branch.
  2. Update your .env file
  3. Install dependencies. 
    poetry install
  4. Check the existing commands ELI5 provides: 
    poetry run eli5 datasets --help
  5. Download the datasets: 
    poetry run eli5 datasets sync <your folder>
  6. Wait for 10 minutes
  7. Try to download the datasets again: 
    poetry run eli5 datasets sync <your folder>
# no new changes added
  8. Make any changes in LangSmith - create new dataset, add an example, etc.
  9. Get the new changes locally: 
    poetry run eli5 datasets sync <your folder>

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.

Ref: #40 (closed)

Edited by Alexander Chueshev

Merge request reports