Skip to content

Add threading for deprecation processing

Why is this change being made?

This MR needs !121197 (merged) merged first.

This MR adds threading to the deprecations array processing, which reduces the total time taken to generate the RSS feed by ~97% 🚀 :

From master-patch-2bd1 branch

$ bundle exec bin/generate-breaking-changes-rss  3.43s user 1.01s system 1% cpu 4:36.87 total

From this branch (ashmckenzie/add-threading-for-deprecation-processing)

$ bundle exec bin/generate-breaking-changes-rss  1.35s user 0.45s system 15% cpu 11.809 total

Verification

$ time bundle exec bin/generate-breaking-changes-rss
info: Connecting to GitLab...
info: Retrieving deprecation files from API...
info: Loading breaking changes...
info: Creating /Users/ash/src/gitlab/www-gitlab-com/public/breaking-changes.xml...

$ ls -la
total 608
drwxr-xr-x   4 ash  staff     128 15 Mar 16:27 .
drwxr-xr-x  50 ash  staff    1600 10 Mar 15:56 ..
-rw-r--r--   1 ash  staff  154740 15 Mar 16:26 breaking-changes.xml.orig
-rw-r--r--   1 ash  staff  154740 15 Mar 16:21 breaking-changes.xml.threaded

$ sha256sum breaking-changes.xml.orig breaking-changes.xml.threaded
ae1a7aeda14f5a289fdd51604eae5a24f008dd6050aedb494611af18e8e92917  breaking-changes.xml.orig
ae1a7aeda14f5a289fdd51604eae5a24f008dd6050aedb494611af18e8e92917  breaking-changes.xml.threaded

$ diff breaking-changes.xml.orig breaking-changes.xml.threaded | wc -l
       0

Related: !121197 (merged)

cc @lstahlman

Author Checklist

  • Provided a concise title for this Merge Request (MR)
  • Added a description to this MR explaining the reasons for the proposed change, per say why, not just what
    • Copy/paste the Slack conversation to document it for later, or upload screenshots. Verify that no confidential data is added, and the content is SAFE
  • Assign reviewers for this MR to the correct Directly Responsible Individual/s (DRI)
    • If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the Maintained by section on the page being edited
    • If your manager does not have merge rights, please ask someone to merge it AFTER it has been approved by your manager in #mr-buddies
    • The when to get approval handbook section explains the workflow in more detail
  • If the changes affect team members, or warrant an announcement in another way, please consider posting an update in #whats-happening-at-gitlab linking to this MR
    • If this is a change that directly impacts the majority of global team members, it should be a candidate for #company-fyi. Please work with internal communications and check the handbook for examples.

Edited by Ash McKenzie

Merge request reports

Loading