Add Suggested Reviewer scenario
What does this MR do and why?
Adds a new orchestrated scenario for testing the Suggested Reviewer feature.
- The scenario runs any tests tagged with
:suggested_reviewer - Includes the ability to create a kubernetes cluster in k3d
- Uses the development k8s manifests created in https://gitlab.com/gitlab-org/modelops/applied-ml/review-recommender/cluster-management/-/merge_requests/11
This scenario will be used to run tests that are yet to be written as part of gitlab-org/quality/testcases#3245 (closed)
The aim of this MR is to create the scenario and get it to the point where it successfully creates the services in a k8s cluster. More changes could be needed later in a separate MR when I write the tests.
Tasks
-
Split the shell command changes off into a separate MR: !1031 (merged)
How to set up and validate locally
export GITLAB_QA_CONTAINER_REGISTRY_ACCESS_TOKEN=token
bundle exec exe/gitlab-qa Test::Integration::SuggestedReviewer EEExpand for example output
[Sep 15 2022 11:09:00 AEST (Gitlab QA)] INFO -- Shell command: `docker pull -q gitlab/gitlab-ee:nightly`
.
[Sep 15 2022 11:09:03 AEST (Gitlab QA)] INFO -- Shell command: `docker container inspect gitlab-suggested-reviewer`
..
[Sep 15 2022 11:09:03 AEST (Gitlab QA)] ERROR -- Shell command output:
[]
Error: No such container: gitlab-suggested-reviewer
[Sep 15 2022 11:09:03 AEST (Gitlab QA)] INFO -- Shell command: `docker network list --format '{{.Name}}' --filter name=^test$`
.
[Sep 15 2022 11:09:03 AEST (Gitlab QA)] INFO -- Shell command: `docker run -d --name gitlab-suggested-reviewer --net test --hostname gitlab-suggested-reviewer.test -p 80 --volume /home/mark/dev/gitlab/gitlab-qa/tmp/certs20220915-222579-10jvbxa/gitlab:/etc/gitlab/ssl:Z --volume /home/mark/dev/gitlab/gitlab-qa/tmp/certs20220915-222579-10jvbxa/authority:/etc/gitlab/trusted-certs:Z --volume /tmp/gitlab-qa/gitlab-qa-run-2022-09-15-11-09-00-1e88cfa6/gitlab-suggested-reviewer/logs:/var/log/gitlab:Z gitlab/gitlab-ee:nightly`
.
[Sep 15 2022 11:09:03 AEST (Gitlab QA)] INFO -- Shell command: `docker exec gitlab-suggested-reviewer bash -c "echo \"# Generated by GitLab QA Omnibus Configurator at 2022-09-15 11:09:00
# Gitlab::QA::Runtime::OmnibusConfigurations::Default
gitlab_rails['gitlab_default_theme'] = 10 # Light Red Theme
gitlab_rails['gitlab_disable_animations'] = true # Disable animations
gitlab_rails['application_settings_cache_seconds'] = 0 # Settings cache expiry
gitlab_rails['initial_root_password'] = '5iveL!fe' # Initial root password\" > /etc/gitlab/gitlab.rb;"`
[Sep 15 2022 11:09:03 AEST (Gitlab QA)] INFO -- Shell command: `docker attach --sig-proxy=false gitlab-suggested-reviewer`
........................................................................................................................................................[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Waiting for GitLab to become healthy ...
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Shell command: `docker inspect --format='{{json .State.Health.Status}}' gitlab-suggested-reviewer`
.
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Shell command: `docker port gitlab-suggested-reviewer 80/tcp`
..
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- -> GitLab is available at `http://localhost:49155`!
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Shell command: `docker cp /home/mark/dev/gitlab/gitlab-qa/support/data gitlab-suggested-reviewer:/tmp/data-seeds`
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Shell command: `docker exec gitlab-suggested-reviewer bash -c "mkdir -p /opt/gitlab/embedded/service/gitlab-shell/hooks/pre-receive.d"`
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Shell command: `docker exec gitlab-suggested-reviewer bash -c "echo '#!/usr/bin/env bash
if [[ \$GL_PROJECT_PATH =~ 'reject-prereceive' ]]; then
echo 'GL-HOOK-ERR: Custom error message rejecting prereceive hook for projects with GL_PROJECT_PATH matching pattern reject-prereceive'
exit 1
fi
' > /opt/gitlab/embedded/service/gitlab-shell/hooks/pre-receive.d/pre-receive.d;"`
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Shell command: `docker exec gitlab-suggested-reviewer bash -c "chmod +x /opt/gitlab/embedded/service/gitlab-shell/hooks/pre-receive.d/*"`
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Running exec_commands...
[Sep 15 2022 11:11:24 AEST (Gitlab QA)] INFO -- Shell command: `docker exec gitlab-suggested-reviewer bash -c "gitlab-rails runner /tmp/data-seeds/admin_access_token_seed.rb"`
.
[Sep 15 2022 11:11:41 AEST (Gitlab QA)] INFO -- Shell command: `docker exec gitlab-suggested-reviewer bash -c ""`
[Sep 15 2022 11:11:41 AEST (Gitlab QA)] INFO -- Shell command: `k3d registry list`
..
checking for kubectl... yes
checking for k3d... yes
[Sep 15 2022 11:11:41 AEST (Gitlab QA)] INFO -- Shell command: `k3d cluster create qa-cluster-20220915-ab83fb1c `
...................
[Sep 15 2022 11:11:56 AEST (Gitlab QA)] INFO -- Shell command: `kubectl apply -f -`
....
[Sep 15 2022 11:11:56 AEST (Gitlab QA)] INFO -- Applying manifest /home/mark/dev/gitlab/gitlab-qa/support/manifests/suggested_reviewer/pubsub.yaml
[Sep 15 2022 11:11:56 AEST (Gitlab QA)] INFO -- Shell command: `kubectl apply -f -`
..
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Applying manifest /home/mark/dev/gitlab/gitlab-qa/support/manifests/suggested_reviewer/recommender-bot.yaml
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Shell command: `kubectl apply -f -`
....
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Applying manifest /home/mark/dev/gitlab/gitlab-qa/support/manifests/suggested_reviewer/recommender.yaml
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Shell command: `kubectl apply -f -`
..
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Applying manifest /home/mark/dev/gitlab/gitlab-qa/support/manifests/suggested_reviewer/postgres.yaml
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Shell command: `kubectl apply -f -`
....
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Applying manifest /home/mark/dev/gitlab/gitlab-qa/support/manifests/suggested_reviewer/authenticator.yaml
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Shell command: `kubectl apply -f -`
..
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Running Suggested Reviewer specs!
[Sep 15 2022 11:11:57 AEST (Gitlab QA)] INFO -- Shell command: `docker pull -q gitlab/gitlab-ee-qa:nightly`
.
[Sep 15 2022 11:14:04 AEST (Gitlab QA)] INFO -- Running test suite `Test::Instance::All` for gitlab-ee
[Sep 15 2022 11:14:04 AEST (Gitlab QA)] INFO -- Shell command: `docker run -t --rm --net=test --env GITLAB_QA_CONTAINER_REGISTRY_ACCESS_TOKEN="$GITLAB_QA_CONTAINER_REGISTRY_ACCESS_TOKEN" --env GITLAB_INITIAL_ROOT_PASSWORD="5iveL!fe" --volume /tmp/gitlab-qa/gitlab-qa-run-2022-09-15-11-09-00-1e88cfa6/gitlab-ee-qa-b99ef584:/home/gitlab/qa/tmp:z --volume /var/run/docker.sock:/var/run/docker.sock:z --name gitlab-ee-qa-b99ef584 gitlab/gitlab-ee-qa:nightly Test::Instance::All http://gitlab-suggested-reviewer.test -- --tag suggested_reviewer`
2022-09-15 01:14:10 / CONF ::
==> Base URL: http://gitlab-suggested-reviewer.test
==> Browser: #<Selenium::WebDriver::Chrome::Driver:0x00007f1b4480b540>
==> Libraries: Chemlab::Vendor
[Sep 15 2022 01:14:10 UTC (QA Tests)] INFO -- Performing sanity check for environment!
Run options:
include {:suggested_reviewer=>true}
exclude {:geo=>true}
Randomized with seed 35058
Top 0 slowest examples (0 seconds, 0.0% of total time):
Finished in 0.00658 seconds (files took 2.72 seconds to load)
0 examples, 0 failures
Randomized with seed 35058
[Sep 15 2022 11:14:11 AEST (Gitlab QA)] INFO -- Shell command: `docker ps -f name=gitlab-suggested-reviewer`
..
[Sep 15 2022 11:14:11 AEST (Gitlab QA)] INFO -- Shell command: `docker rm -f gitlab-suggested-reviewer`
.
[Sep 15 2022 11:14:12 AEST (Gitlab QA)] INFO -- Shell command: `k3d cluster delete qa-cluster-20220915-ab83fb1c`
.......MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Mark Lapierre