Metrics: Extend dashboard YAML definion validation when viewing the file to exhaustive list of errors
Background
This is a follow up issue for #219395 (closed) which added:
When viewing a file that contains custom dashboard definition (any file in projects
.gitlab/dashboards/
directory), user is presented with information if that dashboard definition is valid or first encountered error is reported.
Proposal
To enable user ability to fix all existing problems in dashboard definition, we should present exhaustive list of encountered violations instead of first encountered one.
@k_cook suggested !33202 (comment 351398662)
Out of scope here, but wanted to make a note of it since it occurred to me: we may want to validate that these keys are only present once (can't have more than one dashboard title).
Validation Examples
- YAML structure validation
- Prometheus version
- Validate that any IDs present in the file are unique within that group of IDs (a panel ID cannot be the same as another panel ID but can be same as a panel group ID). Display error if IDs are not unique.
- Display warnings if the dashboard, panels, panel groups, or metrics don't have IDs.
- Display a warning if any dashboard links are invalid (checked with
UrlBlocker
). - Check that templating variables have their required keys. For example,
metric_label_values
variables require thetype
,series_selector
andlabel
keys.
Implementation
There is alternative approach available based on JSON schema gem: for more details please view discussion
It seems that in order to not duplicate effort already made we should explore framework created to process and validate gitlab-ci.yml
In order to build similar class to https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/yaml_processor.rb we need to define dashboard config model (as gitlab-ci.yml
model was build https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/config/entry/root.rb) with usage of following elements:
- https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/config/entry/configurable.rb
- https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/config/entry/validatable.rb
- https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/config/entry/node.rb
Plausibly that means that existing ActiveModel
based validations has to be rewritten, to the one which are compatible with https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/config/entry/validatable.rb