Use hiredis to increase performance of reading large sets
hiredis-rb uses the hiredis C client library to optimize read/writes to Redis. Note that it merely replaces the underlying read/write/connect calls of the redis-rb library (https://github.com/redis/hiredis-rb/blob/master/ext/hiredis_ext/connection.c#L505-L513), so all high-level functionality is still preserved.
For large payloads, this can increase read throughput by a modest amount. For example, in the benchmark below, we see about a 15% performance increase reading from a large SMEMBERS payload (19,667 i/s to 23,306/s)
require 'redis/connection/hiredis'
require 'redis'
require 'benchmark/ips'
require 'json'
redis = Redis.new(host: 'localhost')
string_key = 'test:string'
set_key = 'test:set'
redis.del(set_key)
data = []
5000.times do |index|
name = "*" * 16
redis.sadd(set_key, name)
data << name
end
redis.set(string_key, data.to_json)
Benchmark.ips do |x|
x.report("smembers") {
members = redis.smembers(set_key)
}
x.report("get") {
members = JSON.parse(redis.get(string_key))
}
x.compare!
end
Redis 3.2.1
Without hiredis
Calculating -------------------------------------
smembers 1.946k i/100ms
get 85.000 i/100ms
-------------------------------------------------
smembers 19.667k (± 8.0%) i/s - 99.246k
get 844.214 (± 3.6%) i/s - 4.250k
Comparison:
smembers: 19667.0 i/s
get: 844.2 i/s - 23.30x slower
With hiredis
Calculating -------------------------------------
smembers 2.301k i/100ms
get 76.000 i/100ms
-------------------------------------------------
smembers 23.306k (± 7.5%) i/s - 117.351k
get 841.539 (± 3.7%) i/s - 4.256k
Comparison:
smembers: 23306.2 i/s
get: 841.5 i/s - 27.69x slower
Redis 5.0.7
Without hiredis
Calculating -------------------------------------
smembers 1.955k i/100ms
get 74.000 i/100ms
-------------------------------------------------
smembers 21.447k (± 4.0%) i/s - 107.525k
get 812.842 (± 4.3%) i/s - 4.070k
Comparison:
smembers: 21447.3 i/s
get: 812.8 i/s - 26.39x slower
With hiredis
Calculating -------------------------------------
smembers 2.404k i/100ms
get 78.000 i/100ms
-------------------------------------------------
smembers 24.274k (± 5.8%) i/s - 122.604k
get 850.175 (± 3.3%) i/s - 4.290k
Comparison:
smembers: 24273.7 i/s
get: 850.2 i/s - 28.55x slower
Edited by 🤖 GitLab Bot 🤖