Add BulkImports Project Members Migration & Refactor Group Members Migration
- Group Migration -
bulk_import
feature flag. Default enabled (& enabled on .com) - Project Migration -
bulk_import_projects
feature flag. Default disabled (& disabled on .com)
Mentions #341886 (closed)
What does this MR do and why?
This MR:
- Adds Project Membersip migration, which preserves user memberships from source to destination project (for users with public email info)
- Updates Group Membership migration, to not only import direct memberships, but also handle inherited memberships
- Updates
MembersPipeline
&GetMemberQuery
GraphQL query, which is used to fetch membership information from source, to be used in both Group & Project Migrations - Updates all
bulk_imports/graphql/*
queries to become classes, instead of modules, in order to allow easier implementation ofGetMemberQuery
that caters for both groups and projects (since query differs)
The entire members pipeline flow is split into 3 steps (Extract/Transform/Load) and is the following:
⬇
Data extraction - For each group/project that needs to be imported, fetch direct & inherited memberships using GraphQL (
GetMembersQuery
)
🖌
Data transformation For each user membership:
- Clean up any prohibited attributes
- Map
public_email
information on source to a user on destination, using any confirmed email - If user is found, cache
source_user_id => destination_user_id
in redis for future use
💾
Data load - Create user membership if there is no existing membership
- Create user membership if there is existing membership with lower access level (e.g. a user has Developer inherited membership and has Maintainer direct memberhip).
- Do not create user membership if existing inherited membership has higher access level
- Do not create user membership if user_id is current user id, since current user is already a group/project member
Screenshots or screen recordings
https://monosnap.com/file/QeXr6NeR83o76jTKCqHnyuh7BObujE
How to set up and validate locally
-
Feature.enable(:bulk_import)
. -
Feature.enable(:bulk_import_projects)
. - Create a top level group and project in it.
- Invite group and project members with public email information (as well as without)
- Go to
/groups/new#import-group-pane
page and enter instance url and access token (needs to beapi
&read_repository
scope). - Select newly created group and click Import.
- Wait for Group import to complete and verify imported group & project members. Users with public emails should be present with the same access levels1
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by George Koltsov