Skip to content

Add large project data validation with GPT Data Generator

Nailia Iskhakova (OOO) requested to merge 373-large-project-validation into master

The MR adds large project data validation to the Generator and adds guidance on how to troubleshoot this issue.

  • Large project validation is being used to determine if the project data is correct.
  • Failures warn user that data is incorrect.

If the data that was imported to the Instance is invalid(wrong storage/issue count/MR count), the Generator will warn users that "project X has these failures: (...)" [example output]. User can rerun the Generator to fix the issue and reimport the broken project or they can try to import it manually via UI.

Closes #373 (closed).

Example of the error output:

| Vertical data: importing large projects for GPT...
Group gpt_small2 already exists
Group gpt_small2/large_projects already exists
Checking if project gitlabhq1 already exists in gpt_small2/large_projects/gitlabhq1...
Project gpt_small2/large_projects/gitlabhq1 already exists
- Large project is a different version (1) than configured (2).
- Large Project repository storage 'default' is different than expected 'test' specified in Environment Config file.
To troubleshoot please refer to https://gitlab.com/gitlab-org/quality/performance/-/blob/master/docs/environment_prep.md#large-project-repository-storage-is-different-than-expected.
- Large Project metadata validation failed: issues count '2' should be '6722' or higher as specified in the Project Config file.
To troubleshoot please refer to https://gitlab.com/gitlab-org/quality/performance/-/blob/master/docs/environment_prep.md#large-project-metadata-validation-failed.
- Large Project metadata validation failed: merge requests count '1' should be '3609' or higher as specified in the Project Config file.
To troubleshoot please refer to https://gitlab.com/gitlab-org/quality/performance/-/blob/master/docs/environment_prep.md#large-project-metadata-validation-failed.

Large project gpt_small2/large_projects/gitlabhq1 already exists on environment but with invalid data.
The Generator will replace this project.
Do you want to proceed? [Y/N]

Example - fresh install with validation errors due to MR and issue counts are different from expected: https://gitlab.com/gitlab-org/quality/performance/uploads/1ba97a501ccadf3264d00862e88c58ab/Screenshot_2020-10-29_at_15.48.35.png

Edited by Nailia Iskhakova (OOO)

Merge request reports

Loading