Create dataset from local parquet files using hf datasets (!11) · Merge requests · GitLab.org / ModelOps / AI Assisted (formerly Applied ML) / Code Suggestions / Model Development

Hongtao Yang requested to merge gcs-dataset into main Mar 16, 2023

Following !9 (merged), this tiny MR adds the ability to create dataloader from local parquet files.

Edited Mar 16, 2023 by Hongtao Yang

Create dataset from local parquet files using hf datasets