Skip to content

Consolidate connection pool config across execution contexts

Need

In #32562 (closed) and #35559 (closed) we introduced changes where we tweak the database and redis connection pool sizes at application start (i.e. at runtime, not build/configuration time) based on both what the user requested through config files such as gitlab.rb as well as what the maximum concurrency level is for the given app server (unicorn, puma). We had previously done a similar change for sidekiq, since it's also multi-threaded. With the pool size now dependent on the different execution environments, it gets increasingly difficult to define this config in a consistent and maintainable way. We should consolidate these concerns into a single place so that it's easy to test and maintain.

Approach

Rather than spreading out this logic across different initializers, we should instead co-locate it into a central class/file.

This relies on having a consistent way to identify the current type of application process. The work for this is defined in #36709 (closed)

The only addition I see is to provide a max_threads helper that we can consult for the concurrency level of the current runtime, which will be considered instead of reaching into config directly.

We should split this issue into two separate MRs, one for DB pool settings, and one for Redis pool settings:

Non-goals:

  • Change/tweak pool settings and sizing. This story is meant to be a refactor only. Any changes to the sizing strategy should be done via a separate issue. This includes considering new components such as RAILS_MAX_THREADS which we're currently not taking into account.

Benefits

Have a single location where we update the pool size based on the execution context.

Competition

Another option might be to pull this out of the initializer altogether and maintain the config at build time. For instance, we could consolidate puma.rb, database.yml etc before the application even starts, so that we don't have to make changes after the fact.

Edited by Matthias Käppler