Skip to content

Update Import's OverwriteProjectService to remove cross-db transaction

What does this MR do and why?

Mentions #351084 (closed)

This MR updates Projects::OverwriteProjectService (https://docs.gitlab.com/ee/user/project/settings/import_export.html -- allows to perform project import with replacement of existing project under a certain name/path) to no longer wrap DestroyService in Project.transaction, which caused cross-db transaction error, since CI data no longer lives in the same database as the rest of the project data. The approach to overwrite process is slightly changed.

Before:

  1. Move certain relations (e.g. stars) from source to destination project
  2. Destroy source project as part of the overwrite process (sync execute within the same job)
  3. Rename destination project to source's name & path
  4. If an exception occurs - transaction is rolledback along with all the modifications made

After:

  1. Move certain relations (e.g. stars) from source to destination project
  2. Rename source project to temp name <original-path>-old-<SecureRandom.hex>
  3. Rename destination project to source's original name & path
  4. If no errors occurred - schedule source project for deletion, instead of destroying it synchronosly
  5. If exception is raised - move certain relations (e.g. stars) back to source project from destination and re-raise
  6. If source rename failed - raise
  7. If destination rename failed - rename source back to original name & path - raise

The new approach is slightly different, since we scheduled source project destroy instead of doing it within overwrite process. From what I can tell there's no real need in doing it in sync, since if move of relations and renames succeeded the overwrite process can be considered complete, even if source project failed to be deleted. If source project failed to be deleted as part of a separate job - the user can initiate a new project deletion themselves.

Additionally, the rollback process is much simpler now, since we only rename projects and update a few relations, instead of also removing the source project, which should make the overwrite process a bit more performant.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

  1. Create 2 projects, one to act as project to replace and one as project to replace source with
  2. Create a few types of data in both projects (e.g. an issue, a label, etc). Star the project to replace
  3. Export 'project to replace source with' and download its tar.gz archive
  4. Initiare project import with overwrite
curl --request POST --header "PRIVATE-TOKEN: token" --form "namespace=561" --form "path= 
project-to-overwrite" --form "overwrite=true" --form "file=@/path/to/archive.tar.gz" "http://gdk.test:3000/api/v4/projects/import"
  1. Wait for import to finish and verify that project got replaced
  2. Verify source project is removed and replaced project has stars migrated

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Alex Kalderimis

Merge request reports

Loading