Log pipeline progress with tqdm
What does this merge request do and why?
First draft of showing a progress bar for a DoFn
pipeline step.
Example on code-suggestions eval
:
BatchRequestCompletions: 413it [08:02, 1.51s/it]
Example for duo-chat make-dataset-2
:
ApplyQuestionVariations: 18it [00:53, 3.46s/it]
Ref https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/prompt-library/-/issues/101.
Merge request checklist
-
I've ran the affected pipeline(s) to validate that nothing is broken. -
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Bruno Cardoso