Skip to content

Import group boards, board lists & milestones via ndjson when using Bulk Import

George Koltsov requested to merge georgekoltsov/bulk_import_ndjson_boards into master

What does this MR do?

This MR adds boards and board lists to the list of importable relations when using Bulk Import https://docs.gitlab.com/ee/user/group/import/ The import process itself utilises existing Import/Export codebase to transform the input hash into a Board object.

This MR also:

  1. Updates milestones to use ndjson. This is required because milestone created_at value from GraphQL API does not include miliseconds, which sets them to 0000, and when boards are imported that reference a milestone, existing milestone cannot be found, since milestone reference in the ndjson file includes milliseconds. Because of that, a duplicate milestone is created, which is undesired.
  2. Adds raw ndjson files to spec/fixtures/bulk_import in order to provide visibility into what is being tested and for easier editing of ndjson fixtures. EE equivalent for boards_pipeline_spec is also added, as milestone/assignee board lists are EE feature. EE specific fixtures are also added.
  3. DRYs up labels/milestones/boards pipelines and moves #transform and #load methods into NdjsonPipeline module, since both of these methods are identical for all of the ndjson pipelines.

Mentions #324731 (closed)

Screenshots (strongly suggested)

ndjsonboards

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by George Koltsov

Merge request reports

Loading