Migrate `ci_templates_unique` Snowplow event to Internal event
What does this MR do and why?
Context: A self-managed customer would like to track the total count of unique projects using CI templates in their pipelines.
Currently we support Service Ping metrics in the form of e.g. redis_hll_counters.ci_templates.p_ci_templates_matlab_weekly
, however these metrics give the aggregated count of unique projects per template. We formerly had ci_templates_total_unique_counts_weekly
supported but it has since been removed.
We do have a ci_templates_unique
Snowplow event instrumented, which would provide the customer the required metric, however setting up a custom Snowplow collector may be complex for the user.
The solution in this case is to leverage the new Internal Events API in GitLab.
This MR replaces the ci_template_unique
event from Snowplow to an Internal Event. It also names the event ci_template_included
to better describe the action.
We confirmed that the existing ci_template_unique
is safe to fully remove.
Resolves #426424 (closed).
How to set up and validate locally
Verify RedisHll counter
- First observe that the Service Ping payload now includes
count_distinct_project_id_from_ci_template_included_*
monthly and weekly keys.
Run the following commands in the console:
service_ping_payload = Gitlab::Usage::ServicePingReport.for(output: :non_sql_metrics_values)
service_ping_payload['redis_hll_counters']['ci_templates'].select { |key, _| key.include?('count_distinct_project_id_from_ci_template_included_') }
- Run the following command and note the outputted value (it should be 0 if you haven't run Step 3 yet).
::Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'ci_template_included', start_date: Date.today, end_date: Date.today + 7.days)
- Go to your Project's CI/CD Editor and update the contents to include any template:
include:
- template: iOS-Fastlane.gitlab-ci.yml
- Commit the changes and wait for the pipeline to complete running (it's fine if it fails). Now observe the counter value again. It should have incremented by 1.
Verify Snowplow event
- Configure your GDK to run Snowplow Micro.
- Go to
http://gdk.test:9091/micro/good
to observe the events being tracked. - Go to your Project's CI/CD Editor and update the contents to include any template:
include:
- template: iOS-Fastlane.gitlab-ci.yml
- Commit the changes and wait for the pipeline to complete running (it's fine if it fails). Refresh
http://gdk.test:9091/micro/good
and observe that an event appears withse_action:
valueci_template_included
.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #426424 (closed)