Extract events from metric definitions in CI to reduce startup time
Problem
Currently we load all metric definitions in the initializer config/initializers/internal_events.rb
.
This is adding a bit more time to the startup than we would like due to the inefficiency of load_metric_definitions
in lib/gitlab/internal_events/event_definitions.rb
.
If the event definitions were accessable in a format that is better suited for the use case we wouldn't have to load and parse all metric definitions.
In addition the inherent problems of caching led to issues like #416627 (closed) where tests where getting flaky.
Desired Outcome
Performance of startup time not significantly affected by loading metric definitions
Proposed Solution
From a Slack conversation with @stanhu in #backend
where the question about how to process the definitions without impacting performance:
could you load some pre-generated file that has all those keys parsed from every YAML file? for example, you could have a Rake task that extracts what you need into a single file, and then load that file upon initialization. There would be a CI task that would ensure that anytime someone added a metric that the Rake task would need to be updated
So, we would like to pre-compute and store a json file in CI that is structured so that we just need to load it to support lib/gitlab/internal_events/event_definitions.rb
How to verify
TBD - We should come up with a way to measure the impact on startup time we have right now, vs what a potential improvement should achieve
Further actions needed
None defined at this time.