Trigger worker to fetch suggested reviewers
What does this MR do and why?
Trigger worker to fetch suggested- reviewers
- Behind a feature flag
- Idea #373173 (comment 1114922558)
Related to #373173 (closed)
Logs of run
- Setup the environment and enabled the project
bin/rails c
[1] pry(main)> mr = MergeRequest.find(162)
[3] pry(main)> Feature.enable(:suggested_reviewers_control, mr.project)
=> true
- I've commit to an MR
- Here are the logs, failed due to machine learning API not being available
log/sidekiq.log:{"severity":"INFO","time":"2022-09-29T14:10:05.884Z","retry":25,"queue":"default","backtrace":true,"version":0,"args":["162"],"class":"MergeRequests::FetchSuggestedReviewersWorker","jid":"e77866c7d6b392e318df75de","created_at":"2022-09-29T14:10:05.860Z","correlation_id":"01GE4SAB7E15AEVM6523TPB9GF","meta.caller_id":"UpdateMergeRequestsWorker","meta.remote_ip":"172.16.123.1","meta.feature_category":"workflow_automation","meta.user":"test1","meta.project":"zama/zingo","meta.root_namespace":"zama","meta.client_id":"user/46","meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:855f6be34663d057be39008b5e9d7709a7a68c6a7193b784c821a7e13cb7b253","size_limiter":"validated","enqueued_at":"2022-09-29T14:10:05.863Z","job_size_bytes":5,"pid":223678,"message":"MergeRequests::FetchSuggestedReviewersWorker JID-e77866c7d6b392e318df75de: start","job_status":"start","scheduling_latency_s":0.020641}
log/sidekiq.log:{"severity":"WARN","time":"2022-09-29T14:10:14.619Z","retry":25,"queue":"default","backtrace":true,"version":0,"args":["162"],"class":"MergeRequests::FetchSuggestedReviewersWorker","jid":"e77866c7d6b392e318df75de","created_at":"2022-09-29T14:10:05.860Z","correlation_id":"01GE4SAB7E15AEVM6523TPB9GF","meta.caller_id":"UpdateMergeRequestsWorker","meta.remote_ip":"172.16.123.1","meta.feature_category":"workflow_automation","meta.user":"test1","meta.project":"zama/zingo","meta.root_namespace":"zama","meta.client_id":"user/46","meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:855f6be34663d057be39008b5e9d7709a7a68c6a7193b784c821a7e13cb7b253","size_limiter":"validated","enqueued_at":"2022-09-29T14:10:05.863Z","job_size_bytes":5,"pid":223678,"message":"MergeRequests::FetchSuggestedReviewersWorker JID-e77866c7d6b392e318df75de: fail: 8.731836 sec","job_status":"fail","scheduling_latency_s":0.020641,"redis_calls":3,"redis_duration_s":0.00038,"redis_read_bytes":9,"redis_write_bytes":249,"redis_queues_calls":3,"redis_queues_duration_s":0.00038,"redis_queues_read_bytes":9,"redis_queues_write_bytes":249,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":5,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.167258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":8.731836,"completed_at":"2022-09-29T14:10:14.616Z","load_balancing_strategy":"primary","exception.class":"Gitlab::AppliedMl::Errors::ConfigurationError","exception.message":"Variable SUGGESTED_REVIEWERS_SECRET is missing","exception.backtrace":["ee/lib/gitlab/applied_ml/suggested_reviewers/client.rb:61:in `read_secret!'","ee/lib/gitlab/applied_ml/suggested_reviewers/client.rb:29:in `initialize'","ee/app/services/merge_requests/fetch_suggested_reviewers_service.rb:22:in `new'","ee/app/services/merge_requests/fetch_suggested_reviewers_service.rb:22:in `suggested_reviewers'","ee/app/services/merge_requests/fetch_suggested_reviewers_service.rb:6:in `execute'","ee/app/workers/merge_requests/fetch_suggested_reviewers_worker.rb:26:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executed.rb:17:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:58:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:113:in `block in use'","lib/gitlab/application_context.rb:113:in `use'","lib/gitlab/application_context.rb:54:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:76:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:103:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:103:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:75:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"db_duration_s":0.012465}
How to set up and validate locally
- Ensure a SaaS (Gitlab.com) environment
- One way of doing this is to add a
env.runit
file to the root GDK folder with the following snippet exportGITLAB_SIMULATE_SAAS=1
- One way of doing this is to add a
- Set ultimate license on a group http://gdk.test:3000/admin/groups
- Create a project in the ultimate group
- Set the feature flag on rails console
bin/rails c
project = Project.find(id-of-your-project) in the rail console
Feature.enable(:suggested_reviewers_control, project)
- Create a merge request on the project
- Push code to the merge request
- You should see some failed workers in the sidekiq logs (due to lack of authentication) and some failed jobs in admin panel http://gdk.test:3000/admin/background_jobs
grep -r FetchSuggestedReviewersWorker log/sidekiq*
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Alper Akgun