Re-use Redis instance's connection pool in MultiStore
What does this MR do and why?
This MR reworks MultiStore to use the connection pools in the Redis instances instead of creating new connections.
Issue: gitlab-com/gl-infra/scalability#2646 (closed)
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
- Set up your
config/redis.yml
➜ gitlab git:(sc1-multistore-conn-pool-sharing) ✗ cat config/redis.yml
---
development:
cluster_shared_state:
cluster:
- "redis://localhost:6001"
- "redis://localhost:6002"
- "redis://localhost:6000"
- Run the following snippet in a Rails console:
Feature.enable(:use_primary_and_secondary_stores_for_shared_state)
Feature.enable(:use_primary_and_secondary_stores_for_sidekiq_status)
Gitlab::Redis::SharedState.with {|c| c.pfadd 'abc', 1 }
Gitlab::Redis::ClusterSharedState.with {|c| c.pfadd 'abc', 1 }
- Run the following on your terminal. You should only see 1 connection as all 2 Redis wrapper classes will get the connection to the Redis Cluster using
ClusterSharedState
's connection pool.
redis-cli -p 6001 client list | rg pfadd
- Repeat step 2 on
master
branch, you should see 2 connections as each of the 2 Redis wrapper classes spin their own Redis connections
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Sylvester Chin