Share rendered markdown HTML between reference filters
What does this MR do and why?
- Contributes to #340387 (closed)
- Feature flag: #341849 (closed)
Problem
HTML::Pipeline
executes a chain of ReferenceFilters
. Each filter with
reference cache tries to fetch parents for references from the document and converts
Nokogiri structure into HTML document. HTML generation process consumes extra
memory and increases response time. We repeat it for each filter.
Solution
Use shared result
variable to store rendered HTML. ReferenceCache
will use it when present or populate when missing.
From docs: https://github.com/gjtorikian/html-pipeline/blob/main/lib/html/pipeline/filter.rb#L22
A Result Hash allows filters to make extracted information available to the caller and is mutable.
Screenshots or screen recordings
I tested CHANGELOG.md file from gitlab-foss repository: https://gitlab.com/gitlab-org/gitlab-foss/-/blob/5e9fe672fa0eda6322bd392e8502d1886804bd07/CHANGELOG.md
Before | After | |
---|---|---|
Memory | ||
Flamegraph | 7.55s | 6.44s |
How to set up and validate locally
It is not easy to reproduce it, however here is a list of steps to do that.
There are extra steps required to setup GDK:
- Rebuild ruby with a patch to enable Memory tab on performance bar (https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/advanced.md#apply-custom-patches-for-ruby)
- Add gitlab-foss repository to GDK repositories (https://gitlab.com/gitlab-org/gitlab-foss)
When it is done, you need to disable Banzai cache and Bullet
- Add
cache_key = false
beforeif cache_key
(here) - Set
Bullet.enable = false
(here) - Restart Rails process (
gdk restart rails-web
) - Open
http://localhost:3000/root/gitlab-foss/-/blob/master/CHANGELOG.md
- Wait till page fully loads
- Choose
CHANGELOG.md?format=json&viewer=rich
from select box on performance bar to see Memory consumption - Open
http://localhost:3000/root/gitlab-foss/-/blob/master/CHANGELOG.md?format=json&viewer=rich&performance_bar=flamegraph
to see flamegraph
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.