Import group boards, board lists & milestones via ndjson when using Bulk Import
What does this MR do?
This MR adds boards and board lists to the list of importable relations when using Bulk Import https://docs.gitlab.com/ee/user/group/import/ The import process itself utilises existing Import/Export codebase to transform the input hash into a Board
object.
This MR also:
- Updates milestones to use ndjson. This is required because milestone created_at value from GraphQL API does not include miliseconds, which sets them to
0000
, and when boards are imported that reference a milestone, existing milestone cannot be found, since milestone reference in the ndjson file includes milliseconds. Because of that, a duplicate milestone is created, which is undesired. - Adds raw ndjson files to
spec/fixtures/bulk_import
in order to provide visibility into what is being tested and for easier editing of ndjson fixtures. EE equivalent for boards_pipeline_spec is also added, as milestone/assignee board lists are EE feature. EE specific fixtures are also added. - DRYs up labels/milestones/boards pipelines and moves
#transform
and#load
methods intoNdjsonPipeline
module, since both of these methods are identical for all of the ndjson pipelines.
Mentions #324731 (closed)
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
-
I have included a changelog entry, or it's not needed. (Does this MR need a changelog?) -
I have added/updated documentation, or it's not needed. (Is documentation required?) -
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) -
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) -
I have self-reviewed this MR per code review guidelines. -
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) -
I have followed the style guides.
Availability and Testing
-
I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) -
I have tested this MR in all supported browsers, or it's not needed. -
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.
Security
Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Edited by George Koltsov