Skip to content

Only wait for exports related to users for contribution mapping generation

Keeyan Nejad requested to merge 461282-user-mapping-user-contributions-relation-performance-enhancements into master

What does this MR do and why?

As mentioned in #461282 (closed), we are currently waiting for all exports to finish before starting the user contributions mapping. This MR changes it so it only waits for user related exports to finish processing.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Make a request to generate an export for a project (replace rails%2frails with the URL encoded name of any local repo) and <token> with your PAT.
curl --request POST \
  --url http://127.0.0.1:3000/api/v4/projects/rails%2frails/export_relations \
  --header 'PRIVATE-TOKEN: <token>'
  1. Wait a little (maybe a minute or two for a small repo) for all exports to finish. You can optionally monitor in the logs that the UserContributionsExportWorker runs a few times, since it should be restarted after 20 seconds if other steps still haven't finished.
  2. Request and extract the user_contributions data to ensure it's correct (replace as before):
curl --request GET \
  --url 'http://127.0.0.1:3000/api/v4/projects/rails%2frails/export_relations/download?relation=user_contributions' \
  --header 'PRIVATE-TOKEN: <token>' \
  --output user_contributions.gz
gunzip user_contributions.gz
cat user_contributions

Related to #461282 (closed)

Edited by Keeyan Nejad

Merge request reports