Skip to content

Add pre and post import validation to gists import

Kristina Doskich requested to merge fail-gists-import-if-repo-too-big into master

What does this MR do and why?

Adds post-import and pre-import validations to check if files count and repos size was exceeded.

Corresponding issue: Fail gist import if git repository too big (#384860 - closed)

Screenshots or screen recordings

Screenshot_2023-05-08_at_15.44.59

To check repo size post-import validation I substitute git_pull_url with a git repo that exceeds max size limit. (This repo needs to have not more than 10 files, otherwise, the files count validation will fail import even before repo size validation)

Screenshot_2023-05-08_at_17.16.42

How to set up and validate locally

For the testing purpose I changed application settings to have a lower max snippet size using Change application settings for snippet_size_limit

For pre-import validations:

  1. On GitHub.com create gists with more than 10 files and gist with a size bigger than your snippet_size_limit value.
  2. Initiate gists import via {{host}}/api/v4/import/github/gists (Import GitHub gists into GitLab snippets)
  3. After the import is finished check http://gdk.test:3000/rails/letter_opener/ it should have an email with the list of gists that were not imported with an error message or/and importer.log it should have error messages of not imported gists.

For post-import validations:

  1. For testing purposes I commented validate_input! method execution in lib/gitlab/github_gists_import/importer/gist_importer.rb to skip pre-import validations.
  2. On GitHub.com create gists with more than 10 files and gist with a size bigger than your snippet_size_limit value.
  3. Initiate gists import via {{host}}/api/v4/import/github/gists (Import GitHub gists into GitLab snippets)
  4. After the import is finished check http://gdk.test:3000/rails/letter_opener/ it should have an email with the list of gists that were not imported with an error message or/and importer.log is should have error messages of not imported gists.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Kristina Doskich

Merge request reports

Loading