Add `QueryAnalyzers::GitlabSchemasMetrics` as a way to observe used schemas
What does this MR do and why?
This is based on top of !73827 (merged), which needs to be merged first.
This is based on top of !74076 (merged), which needs to be merged first.
This analyzers output is Prometheus metrics observing connections vs gitlab schemas of executed queries.
It will present well in a single metric how many connections do cross-join or are misplaced once additional databases are configured.
This will observe all queries executed and increment prometheus counter
gitlab_database_decomposition_gitlab_schemas_used
.
This effectively allows to see in a single metric depending on a Phase:
- In Phase 1/2: since we have only single
main/main_replica:
a number of queries that are eithergitlab_main
,gitlab_ci
,gitlab_ci/gitlab_main
- In Phase 3: since we have
main/main_replica/ci_replica:
a number of misaligned queries that if go toci_replica:
are not onlygitlab_ci
, butgitlab_main
/gitlab_ci,gitlab_main
- In Phase 4: since we have
main/main_replica/ci/ci_replica
: a number of misaligned queries that that cross boundariesmain:
accessinggitlab_ci
schema, etc.
Related to:
Results
Today it produces results similar to this since not all DB models are yet described, they will be !73839 (merged):
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="ci",gitlab_schemas="undefined_pg_attrdef,undefined_pg_attribute,undefined_pg_collation,undefined_pg_type"} 0
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="ci",gitlab_schemas="undefined_pg_class,undefined_pg_namespace"} 0
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="main",gitlab_schemas="gitlab_main"} 0
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="main",gitlab_schemas="gitlab_shared"} 0
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="main",gitlab_schemas="undefined_pg_attrdef,undefined_pg_attribute,undefined_pg_collation,undefined_pg_type"} 4
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="main",gitlab_schemas="undefined_pg_attribute,undefined_pg_index"} 4
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="main",gitlab_schemas="undefined_pg_class,undefined_pg_namespace"} 2
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="main_replica",gitlab_schemas="gitlab_main"} 22
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="main_replica",gitlab_schemas="gitlab_shared"} 2
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="unknown",gitlab_schemas="undefined_pg_range,undefined_pg_type"} 0
gitlab_database_decomposition_gitlab_schemas_used{db_config_name="unknown",gitlab_schemas="undefined_pg_type"} 40
How to set up and validate locally
- Enable the query analyzer
Feature.enable(:query_analyzer_gitlab_schema_metrics)
- Visit any group or project member pages such as
http://127.0.0.1:3000/groups/flightjs/-/group_members
- Go to
http://127.0.0.1:3000/-/metrics
and look forgitlab_database_decomposition_gitlab_schemas_used
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.