Migrate daily runs to GitLab CI
What does this merge request do and why?
This MR:
- Adjust the CI yaml so that we can run prompt library in the piplines
- Adds configs for the daily run for the 4 duo chat datasets
- Adds a brief doc on daily runs (only about the new way introduced by this MR)
- Removes the timestamp from the BQ write step (see comment)
Ref: #203 (closed)
How to set up and validate locally
- Go to Build -> Pipeline schedules
- Create a new pipeline schedule from this branch (see docs for more info)
- Trigger the pipeline manually (or wait for the schedule)
- Confirm that the job is triggered, finished as expected, and that the results are written to BQ
Merge request checklist
-
I've ran the affected pipeline(s) to validate that nothing is broken. -
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Hongtao Yang