Calculate xMAUs unions for Redis HLL metrics
What does this MR do?
This MR demonstrates proposed implementation for new feature (https://gitlab.com/gitlab-org/product-analytics/-/issues/421) that will allow for calculation unions over multiple tracked metrics in order to get more accurate data for GMAUs and SMAUs metrics aggregation.
Baseline idea is as follow:
- PM defines events combination in new YAML file
lib/gitlab/usage_data_counters/aggregated_metrics.yml
- As the final step of usage ping generation, when all events are already calculated new method
Gitlab::UsageDataCounters::HLLRedisCounter.combined_events_data
is called -
.aggregated_metrics_data
reads content oflib/gitlab/usage_data_counters/aggregated_metrics.yml
and calculates number of unique elements in union of all listed metrics for given events combination, with usage of pfcount
Screenshots
Does this MR meet the acceptance criteria?
Conformity
Change is hidden behind feature flag
- [ ] Changelog entry
- [ ] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Edited by Mikołaj Wawrzyniak