User mapping - Integration with Direct Transfer
This issue is blocked by User mapping - Create relation contributors in ... (#454522 - closed)
On this issue, we will implement the improved user mapping on Direct Transfer.
The POC available at !148351 (closed) can be used as a reference for implementation purposes.
- Changes should be behind a new feature flag. All flag checks should additionally check the
importer_user_mapping
flag which is the global feature flag toggle for the user mapping feature (see Slack, internal, good for 90 days). - Create a new pipeline that creates a placeholder user for every contributor found in the new relation created in #454522 (closed)
- Update the BulkImports::Common::Pipelines::MembersPipeline to create a placeholder user for each member
- Update Direct Transfer to assign the contributions to the created placeholder users
- Use the service created on #443533 (closed) while deciding whether a placeholder use was already created or a new one should be created.
- Update Direct Transfer to re-run the pipeline in case of a Gitlab::Import::SourceUserMapper::FailedToObtainLockError error. This error may be raised if the lease lock isn't obtained while creating placeholder users.
- Use the service created in !156704 (merged) to record each placeholder user reference, so we can later know which user references to update during re-mapping
- Queue the worker in !156704 (merged) to load placeholder user references into PostgreSQL after certain "stages" of the import, see !156704 (comment 1988728019) for some thoughts about which points to queue.
- Update
BulkImports::ProcessService
to continue to requeue until placeholder user references have been saved to PostgreSQL !158536 (merged).
Notes:
- At this phase, we aren't applying limits to the number of placeholder users, therefore we don't need to create import_detail records like it was done in the POC. This also does not need to include any pre-migration checks either.
- The new pipeline to create
import_source_users
users should run before the other pipelines so that when a contribution needs to be mapped, theimport_source_user
isn't missing a name, username, etc
Edited by Luke Duncalfe