Handle unexpected csv data
What does this MR do and why?
Some of the CSV data (even though it is curated by the External License Database) still has some unexpected data:
- names, versions, licenses that are too long for the text limit on the models
- malformed csv which causes the
csv
to emit an error
This MR ensures that imported package metadata data is truncated to a reasonable default before being saved in the database. Because this is public registry data we don't want to change the allowed size on the columns every time an outlier entry shows up.
In addition, this MR ensures that malformed csv errors are caught and logged so as not to bubble the error up and stop PackageMetadata::SyncService.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Igor Frenkel