Skip to content

Verify correctness of ProtectedBranch cache

Vasilii Iakliushin requested to merge 366724_test_protected_branch_cache into master

What does this MR do and why?

Roadmap

  1. Add ProtectedBranches::CacheService for efficient caching - !92922 (merged)
  2. Test ProtectedBranches::CacheService 👈 this MR
  3. Start using ProtectedBranches::CacheService - !92937 (merged)

Problem

Previous MR added a new implementation of ProtectedBranch cache. We want to verify if it works correctly.

Solution

  • Add dry_run option to ProtectedBranch cache
  • Add feature flag to control cache flow

New cache implementation with dry_run: true will create new cache structures but it will still return non-cached values to the user. If we have cache inconsistency then we will log an error message.

How it works now (examples)

Here is an example of current caching keys. I requested master branch protected status 4 times after changing project updated at value. There were no changes to protected branches, however we created 4 keys with the same value.

"cache:gitlab:protected_ref-projects/1-20220114120328112429-4f26aeafdb2367620a393c973eddbe8f8b846ebd",
"cache:gitlab:protected_ref-projects/1-20220719161714620946-4f26aeafdb2367620a393c973eddbe8f8b846ebd",
"cache:gitlab:protected_ref-projects/1-20220719161807555682-4f26aeafdb2367620a393c973eddbe8f8b846ebd",
"cache:gitlab:protected_ref-projects/1-20220719161801445142-4f26aeafdb2367620a393c973eddbe8f8b846ebd"

The key structure: cache:gitlab:protected_ref-projects/<project_id>-<project_updated_at>-<branch_name_hash>

Every time project.updated_at is changed we create a new key. Old keys are left in Redis before they expire.

How it is going to work

After the first request to protected branch status, we are going to create the key

"cache:gitlab:protected_branch:1"

where 1, is a project_id

redis.hgetall  "cache:gitlab:protected_branch:1"
=> {"fc613b4dfd6736a7bd268c8a0e74ed0d1c04a959f59dd74ef2874983fd443fc9"=>"_b:1"}

This Redis key contains a hash structure, where hash key is a branch name hash and hash value is encoded true / false.

If user requests protected statuses of several branches, then they will be collected under the same key:

redis.hgetall  "cache:gitlab:protected_branch:1"
=> {"fc613b4dfd6736a7bd268c8a0e74ed0d1c04a959f59dd74ef2874983fd443fc9"=>"_b:1", "756c33ff34fd6468e238af51d0b6615e0d9cca2345f6dee2dc403b9af1292a80"=>"_b:0", "b2dbadc0889e8ce9fbc6b6b03ecab063f808b53a23542ab5de60be0513939ea2"=>"_b:0"}

The key won't be updated together with every project update, but only when ProtectedBranch object related to this project was changed.

Let's say I create a new branch.

ProtectedBranches::CreateService.new(Project.find(1), User.first, name: 'protected_branch').execute

# Redis key was cleared
redis.hgetall  "cache:gitlab:protected_branch:1"
=> {}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Vasilii Iakliushin

Merge request reports