Skip to content

Measure how many projects have turned OFF Monitor

Goal

This issue explores the best way to measure success and adoption of GitLab's Incident Management Features.

Context

Users want less incidents. Measuring monthly active users for incident management isn't the best measure for adoption. High MAU isn't a proxy for adoption. Similarly, low MAU doesn't mean low adoption.

Today, Respond GMAU includes the following aggregated metrics:

Click to expand
  • incident_management_incident_created
  • incident_management_incident_reopened
  • incident_management_incident_closed
  • incident_management_incident_assigned
  • incident_management_incident_todo
  • incident_management_incident_comment
  • incident_management_incident_zoom_meeting
  • incident_management_incident_published
  • incident_management_incident_relate
  • incident_management_incident_unrelate
  • incident_management_incident_change_confidential
  • incident_management_alert_status_changed
  • incident_management_alert_assigned
  • incident_management_alert_todo
  • incident_management_timeline_event_created
  • incident_management_timeline_event_edited
  • incident_management_timeline_event_deleted

Historical

Over the last few milestones we've considered how to best measure grouprespond GMAU. Here are some of the things we thought about:

  • Updating GMAU to include all features. Result:

Continually updating GMAU with new features would make it difficult to determine growth. Previous versions may have the feature but weren't reporting usage since metrics weren't instrumented. Or the metric has been instrumented but GitLab hasn't been updated to the current version (self-managed) where the metrics were implemented. (Additional discussion and recordings.)

  • Looking at individual features that are measured in GMAU to see what is driving (any) growth. Result:

There is no way for us to look at events individually in aggregate metrics unless the event is implemented itself in Service Ping. For SaaS events, the same logic applies, we do have a table that collects backend events that we can analyze individually, but the caveat is that it needs to specifically be implemented into our mart_event table. This table in the Respond Group Dashboard, includes the events are being tied to Monitor in our mart_event table. Right now, only incident_labeled_issues and projects_prometheus_active are the only events available for the Monitor group. This handbook page can give you some guidance on how to add new events to mart_event.

Proposal

Create a new metric called "Number of Projects with Monitor Turned On" to determine who is turning on incident management features. This will serve as the top of the funnel to determine how many SaaS and Self-Managed projects have turned on these features.

Note: This toggle recently became available in %15.6 with the release of "Monitor" Visibility Toggle (#361585 - closed). The Monitor toggle can be found by going to Settings > General and expanding Visibility, project features, permissions.

Edited by Alana Bellucci