Skip to content

Resolve "Handle Duplication of Resources on Migration"

Allen Cook requested to merge 346398-handle-duplication-of-resources-on-migration into master

What does this MR do and why?

This MR handles pipeline duplication and race condition issues associated with BulkImport, see this note for more information

Specifically multiple EntityWorkers were being scheduled and ran for the same resource, which resulted in duplicate resources or unnecessary failures during migration

Screenshots or screen recordings

After running a migration on a project with issues, you should now have no failures with the IssuesPipeline

[4] pry(main)> BulkImports::Failure.where(pipeline_class: 'BulkImports::Common::Pipelines::IssuesPipeline')
  BulkImports::Failure Load (0.4ms)  SELECT "bulk_import_failures".* FROM "bulk_import_failures" WHERE "bulk_import_failures"."pipeline_class" = 'BulkImports::Common::Pipelines::IssuesPipeline' /*application:console,db_config_name:main,line:bin/rails:4:in `<main>'*/
=> []

How to set up and validate locally

  1. Feature.enable(:bulk_import)
  2. Feature.enable(:bulk_import_projects)
  3. Migrate a project with multiple issues
  4. In the rails console BulkImports::Failure.where(pipeline_class: 'BulkImports::Common::Pipelines::IssuesPipeline') should return zero results

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #346398 (closed)

Edited by Allen Cook

Merge request reports