Integrate redis cluster service
What does this merge request do and why?
This MR adds redis-cluster
service to gdk
as the Scalability group will be running redis in cluster mode for redis-ratelimiting
presently and redis-cache
in the near future. Having redis-cluster
will empower developers to iterate on the GitLab application better when it comes to testing cluster-specific logic such as cross-slot errors and pipelining.
Note:
- Socket is not used as Redis Cluster requires a TCP port for intra-cluster communication between nodes.
- 2 clusters are created (1 for development, 1 for test) since only db 0 is allowed in cluster mode.
- This could mean as more functional redis (other than
redis-cache
adopts cluster), we may spin up more clusters.
- This could mean as more functional redis (other than
- No new gdk commands are created since the developer may need to connect to each node of the cluster to debug cross-slot issues. A new command may look like
gdk redis-cluster-cli -p xxxx
which is more verbose thanredis-cli -p xxx
.
See gitlab-com/gl-infra/scalability#2032 (closed).
Impacted categories
The following categories relate to this merge request:
-
gdk-reliability - e.g. When a GDK action fails to complete. -
gdk-usability - e.g. Improvements or suggestions around how the GDK functions. -
gdk-performance - e.g. When a GDK action is slow or times out.
Merge request checklist
-
This change is backward compatible. If not, please include steps to communicate to our users. -
Tests added for new functionality. If not, please raise an issue to follow-up. -
Documentation added/updated, if needed. -
gdk doctor
test added, if needed. -
Add the ~highlight
label if this MR should be included in theCHANGELOG.md
.
To reproduce locally
- Update rendered files
gdk reconfigure // updates Procfile and gitlab/config/redis.cache_cluster.yml
- Stop any running instance
gdk kill
rm -rf redis-cluster/ // clear redis-cluster node conf if needed
rm -r services/* // remove all services
gdk start
redis-cluster
should be available in status
➜ gitlab-development-kit git:(sc1-redis-cluster-integration) gdk status
run: /Users/sylvesterchin/work/gitlab-development-kit/services/gitlab-workhorse: (pid 24506) 4863s, normally down; run: log: (pid 24118) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/postgresql: (pid 24125) 4867s, normally down; run: log: (pid 24114) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/praefect: (pid 24397) 4864s, normally down; run: log: (pid 24121) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/praefect-gitaly-0: (pid 24398) 4864s, normally down; run: log: (pid 24117) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/rails-background-jobs: (pid 24507) 4863s, normally down; run: log: (pid 24122) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/rails-web: (pid 24508) 4863s, normally down; run: log: (pid 24120) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/redis: (pid 40277) 4383s, normally down; run: log: (pid 24115) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/redis-cluster: (pid 44097) 4221s, normally down; run: log: (pid 24119) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/sshd: (pid 24510) 4863s, normally down; run: log: (pid 24116) 4867s
run: /Users/sylvesterchin/work/gitlab-development-kit/services/webpack: (pid 24509) 4863s, normally down; run: log: (pid 24123) 4867s
=> GitLab available at http://127.0.0.1:3000.
=> - Ruby: ruby 2.7.6p219 (2022-04-12 revision c9c2245c0a) [arm64-darwin21].
=> - Node.js: v16.15.0.
- Connect to
development
andtest
clusters
➜ gitlab-development-kit git:(sc1-redis-cluster-integration) redis-cli -p 6001 cluster nodes
5a2f764e110aa8ae27db25d3597de6c5cb3528c2 127.0.0.1:6001@16001 myself,master - 0 1669621577000 2 connected 5461-10922
d29320f5f6e3371edd56eaa40e701c584b749fdf 127.0.0.1:6002@16002 master - 0 1669621579635 3 connected 10923-16383
0deaa148509c5310d392dd35ea7bba1ccb6ddea5 127.0.0.1:6000@16000 master - 0 1669621578624 1 connected 0-5460
➜ gitlab-development-kit git:(sc1-redis-cluster-integration) redis-cli -p 6003 cluster nodes
ea01e8e4eebf952e148037a0943e39f0c714cd1e 127.0.0.1:6005@16005 master - 0 1669621590959 3 connected 10923-16383
218b98cd870591caaea8a4f24354ba08b15cc53b 127.0.0.1:6003@16003 myself,master - 0 1669621590000 1 connected 0-5460
062763ee61653bc76dbd22d86f3a0b11cdbc894d 127.0.0.1:6004@16004 master - 0 1669621589947 2 connected 5461-10922
Edited by Sylvester Chin