Skip to content

Add sidekiq_retries_exhausted to Import::LoadPlaceholderReferencesWorker to fail imports

What does this MR do and why?

This MR adds sidekiq_retries_exhausted that fails whatever import called it if retries have been exhausted. If placeholder user references could not be loaded, then user contributions will fail to be reassigned and implies that the import did not complete successfully.

How to set up and validate locally

It was surprisingly difficult to get worker or service to fail with an exception that wasn't handled gracefully, so the best way I found to test this was to manually raise an error in the worker itself, then call the worker asynchronously.

First, add an error on line 19 of the worker, the first line of perform: raise StandardError, 'Testing failure on retry'. The retry count can be lowered to make this process go faster.

For Direct Transfer:

In a rails console:

bulk_import = BulkImport.create(user_id: User.first.id, source_type: :gitlab, status: 1) # Or find a BulkImport, the User doesn't matter
import_source = 'gitlab'
import_uid = bulk_import.id

# Run the job async
Import::LoadPlaceholderReferencesWorker.perform_async(import_source, import_uid)

# After the job has retried, the BulkImport's status should be failed.
import_state.reload.status

For GitHub, Bitbucket, Bitbucket Server, FogBugz, Gitea and GitLab export upload:

In a rails console:

project = Project.last # Or any project with a ProjectImportState. I used an old, completed imported project from Bitbucket and updated its import_state status to 'started'
import_source = project.import_type # Or any of the types listed above
import_uid = project.import_state.id # Ensure this is in started status so it can transition to failed

# Run the job async
Import::LoadPlaceholderReferencesWorker.perform_async(import_source, import_uid)

# After the job has retried, the ProjectImportState's status should be failed.
import_state.reload.status

This can also be done with a group's GroupImportState as well using import_source: 'import_group_from_file'.

Related to #471466 (closed)

Edited by Sam Word

Merge request reports

Loading