Skip to content

Fix GitHub Import workers spread calculation

Rodrigo Tomonari requested to merge rodrigo/fix-github-import-worker-spread into master

What does this MR do and why?

Take into consideration the job runtime when calculating importers workers enqueue delay on GitHub Import. This way, if stage workers run uninterruptedly for a long time, the delay isn't too far in the future.

Additionally, the calculation has been updated to use a floating number, which will help spread the jobs more evenly.

Related to: #432137 (comment 1701524385)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Trigger a migration using GitHub Import via UI (Projects -> New project -> Import project -> GitHub) or using the API
curl --location 'http://gdk.test:3000/api/v4/import/github' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer GDK_ACCESS_TOKEN' \
--data '{
    "personal_access_token": "GITHUB_ACCESS_TOKEN",
    "repo_id": "238972",
    "target_namespace": "root",
    "new_name": "rspec-core"
}'
  1. Check on Sidekiq Dashboard (http://gdk.test:3000/admin/sidekiq/scheduled) or in the logs if workers are executed to run at the correct time
Edited by Rodrigo Tomonari

Merge request reports