Skip to content

Refactor BulkImports tmpdir variable & update few validations

What does this MR do and why?

This MR:

  • Updates BulkImport file download/decompression dir variable to tmpdir in order to align with ArchiveExtraction service. All 3 services expect temp directory to be used. Before this change 2 services had keyword argument dir while archive service had tmpdir. This change aligns them all to use tmpdir
  • Updates @tmp_dir to @tmpdir for consistency, since we used tmp_dir in some places and tmpdir in others
  • Updates file download/decompression services to add a few more validations, similar to archive extraction service: validate_tmpdir updated to use File.exapnd_path and add new validate_filepath to use Gitlab::Utils.check_path_traversal!
  • Updates tests for changed validations
  • Adds class documentation to file download/decompression services, similar to archive extraction service

Mentions #339405 (closed)

Feature flags

  • Group Migration is behind bulk_import feature flag, default enabled, currently disabled on .com
  • Project Migration is behid bulk_import_projects feature flag, default disabled & disabled on .com

Screenshots or screen recordings

https://monosnap.com/file/tURXbM9RxpDDoCdXkjnrMwW74AjnPy

How to set up and validate locally

  1. Feature.enable(:bulk_import).
  2. Feature.enable(:bulk_import_projects).
  3. Create a top level group and project in it.
  4. Create a few group epics/labels/miletones, project MRs/issues/comments/emojis/uploads
  5. Go to /groups/new#import-group-pane page and enter instance url and access token (needs to be api & read_repository scope).
  6. Select newly created group and click Import.
  7. Wait for Group import to complete and verify imported group & projects. Everything should still migrate as before
  8. Check bulk_import_failures table. It should contain no errors related to the changes.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by George Koltsov

Merge request reports

Loading