Skip to content

Share rendered markdown HTML between reference filters

What does this MR do and why?

Problem

HTML::Pipeline executes a chain of ReferenceFilters. Each filter with reference cache tries to fetch parents for references from the document and converts Nokogiri structure into HTML document. HTML generation process consumes extra memory and increases response time. We repeat it for each filter.

Solution

Use shared result variable to store rendered HTML. ReferenceCache will use it when present or populate when missing.

From docs: https://github.com/gjtorikian/html-pipeline/blob/main/lib/html/pipeline/filter.rb#L22

A Result Hash allows filters to make extracted information available to the caller and is mutable.

Screenshots or screen recordings

I tested CHANGELOG.md file from gitlab-foss repository: https://gitlab.com/gitlab-org/gitlab-foss/-/blob/5e9fe672fa0eda6322bd392e8502d1886804bd07/CHANGELOG.md

Before After
Memory Screenshot_2021-09-28_at_17.44.25 Screenshot_2021-09-28_at_17.45.33
Flamegraph Screenshot_2021-09-28_at_17.49.48 7.55s Screenshot_2021-09-28_at_17.46.42 6.44s

How to set up and validate locally

It is not easy to reproduce it, however here is a list of steps to do that.

There are extra steps required to setup GDK:

  1. Rebuild ruby with a patch to enable Memory tab on performance bar (https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/advanced.md#apply-custom-patches-for-ruby)
  2. Add gitlab-foss repository to GDK repositories (https://gitlab.com/gitlab-org/gitlab-foss)

When it is done, you need to disable Banzai cache and Bullet

  1. Add cache_key = false before if cache_key (here)
  2. Set Bullet.enable = false (here)
  3. Restart Rails process (gdk restart rails-web)
  4. Open http://localhost:3000/root/gitlab-foss/-/blob/master/CHANGELOG.md
  5. Wait till page fully loads
  6. Choose CHANGELOG.md?format=json&viewer=rich from select box on performance bar to see Memory consumption
  7. Open http://localhost:3000/root/gitlab-foss/-/blob/master/CHANGELOG.md?format=json&viewer=rich&performance_bar=flamegraph to see flamegraph

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Vasilii Iakliushin

Merge request reports

Loading