Read-only metrics for repository specific primaries
Adds support for collecting read-only repository metrics taking per repository primaries in to account. The new functionality is behind a boolean flag in order to keep metric collection working for virtual storage primaries as well. In-memory implementations have been removed as there is no support for per repository primaries in the local elector.
Repository is in read-only mode when the primary is out of date. With that in mind, we need to vary the query slightly to account for the differently scoped primaries.
The old and the new queries compare the repositories
and the storage_repositories
tables to identify primaries which are out of date. Performance wise they should be the same although dropping the in-memory implementation allows us to query the primaries directly in the metric query. That reduces the DB round trips to get the primaries of each virtual storage. Previously there was one DB call for each virtual storage to identify the primary and then the metric query. Now we do it in a single query.
When sqlElector
is used, the primary is virtual storage wide, not repository specific. The primary is stored in the shard_primaries
table's node_name
column, where we get it from when the collector is used with virtual storage scoped primaries.
When PerRepositoryElector
is used, each repository has their own primary. The primary is stored in the repositories
table's primary
column.
The metric is displayed on the dashboards. It's useful for the administrators so they can see if any repositories have been switched in to read-only mode following a failover. Alternative is to run dataloss
after every failover. The dashboard can be found here: https://dashboards.gitlab.net/d/8EAXC-AWz/praefect?orgId=1&refresh=30s
Closes #3198 (closed)