Update fixtures for seeding PG for multi-version upgrade tests
What does this MR do and why?
Update fixtures for seeding PG for multi-version upgrade tests:
- Removing
build_super_group_labels
andbuild_subgroups
and relevant code as seeding on groups, projects, milestones, labels is happening as part ofdb/fixture
seeding with SeedFu (gitlab-org/quality/pg-dump-generator!19 (merged)) - Update
Gitlab::Seeder::Projects
to check if projects were already seeded to resolve issues like https://gitlab.com/gitlab-org/gitlab/-/jobs/6376375244#L818 - Update
03_project_1_user_projects
and03_project_2_group_projects
to includeGitlab::Seeder::Projects
so that fixtures can be run withFILTER
Context: Data Seeder and fixtures from db
are used to seed PG for using in multi-version upgrade testing - gitlab-org/quality/quality-engineering&19 (closed) / https://gitlab.com/gitlab-org/quality/pg-dump-generator#overview
Relevant issue
gitlab-org/quality/quality-engineering/team-tasks#2354 (comment 1801955021) and gitlab-org/quality/quality-engineering/team-tasks#2513 (closed)
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
- Run
02_users
in GDK to seed users:CI=1 MASS_INSERT=1 FILTER=02_users.rb bundle exec rake db:seed_fu
- Run
03_project_1_user_projects
on its own in GDK:FILTER=03_project_1_user_projects.rb bundle exec rake db:seed_fu
- Ensure that there are no errors
NameError: uninitialized constant Gitlab::Seeder::Projects
as it was in https://gitlab.com/gitlab-org/quality/pg-dump-generator/-/jobs/6335016752#L1404 - Run
03_project
multiple times:FILTER=03_project.rb bundle exec rake db:seed_fu
- Ensure that there are no errors
NoMethodError: undefined method '_run_after_commit_queue' for nil:NilClass
- In GDK run the command to seed the data:
GITLAB_LOG_LEVEL=debug bundle exec rake "ee:gitlab:seed:data_seeder[bulk_data.rb]"
✅
Testing - Verified that with these changes DB seeding with Data Seeder is working as expected - https://gitlab.com/gitlab-org/quality/pg-dump-generator/-/jobs/6335016752#L1836
- Verified that
03_project_
fixtures are working as expected when ran in isolation:
output
FILTER=03_project.rb bundle exec rake db:seed_fu
== Filtering seed files against regexp: /03_project.rb/
== Seed from gitlab-development-kit/gitlab/db/fixtures/development/03_project.rb
Project toolbox/gitlab-smoke-tests already exists, skipping
Project gitlab-org/gitlab-test already exists, skipping
Project gitlab-org/gitlab-shell already exists, skipping
Project gnuwget/Wget2 already exists, skipping
Project Commit451/lab-coat already exists, skipping
Project i-user-2-1710450342/Underscore already exists, skipping
Project flightjs/Flight already exists, skipping
Project twitter/Typeahead.Js already exists, skipping
OK
CI=1 MASS_INSERT=1 FILTER=02_users.rb bundle exec rake db:seed_fu
== Filtering seed files against regexp: /02_users.rb/
== Seed from /gitlab-development-kit/gitlab/db/fixtures/development/02_users.rb
Creating 10 Users.
Rough estimated time: less than a minute ⏰
10 Users created!
Creating 98 user namespaces.
Rough estimated time: less than a minute ⏰
98 user namespaces created!
Creating 98 User namespaces routes.
Rough estimated time: less than a minute ⏰
98 User namespaces routes created!
===========================================================
INFO: Password for newly created users is: ffa6ad4bc29f72f4
===========================================================
Creating 1 root namespaces and subgroups 9 levels deep.
Rough estimated time: less than a minute ⏰
2024-03-15 11:52:04 UTC: Creating subgroups at level 1: 2
2024-03-15 11:52:04 UTC: Creating subgroups at level 2: 6
2024-03-15 11:52:04 UTC: Creating subgroups at level 3: 14
2024-03-15 11:52:04 UTC: Creating subgroups at level 4: 30
2024-03-15 11:52:04 UTC: Creating subgroups at level 5: 62
2024-03-15 11:52:04 UTC: Creating subgroups at level 6: 126
2024-03-15 11:52:04 UTC: Creating subgroups at level 7: 254
2024-03-15 11:52:04 UTC: Creating subgroups at level 8: 510
2024-03-15 11:52:04 UTC: Creating subgroups at level 9: 1022
2024-03-15 11:52:04 UTC: creating routes.
2024-03-15 11:52:04 UTC: filling traversal ids.
2024-03-15 11:52:04 UTC: creating namespace settings.
1 root namespaces and subgroups 9 levels deep created!
....................
OK
MASS_INSERT=1 CI=1 FILTER=03_project_1_user_projects.rb bundle exec rake db:seed_fu
== Filtering seed files against regexp: /03_project_1_user_projects.rb/
== Seed from /gitlab-development-kit/gitlab/db/fixtures/development/03_project_1_user_projects.rb
Project toolbox/gitlab-smoke-tests already exists, skipping
Project gitlab-org/gitlab-test already exists, skipping
Project gitlab-org/gitlab-shell already exists, skipping
Project gnuwget/Wget2 already exists, skipping
Project Commit451/lab-coat already exists, skipping
Project i-user-2-1710450342/Underscore already exists, skipping
Project flightjs/Flight already exists, skipping
Project twitter/Typeahead.Js already exists, skipping
OK
Creating 50 User projects and corresponding project namespaces.
Rough estimated time: less than a minute ⏰
2024-03-15 11:58:40 UTC: Creating project namespaces: 50.
2024-03-15 11:58:40 UTC: Creating projects: 50.
50 User projects and corresponding project namespaces created!
OK
Edited by Nailia Iskhakova