Skip to content

Enable usage ping on geo secondaries (but mark them)

Alex Ives requested to merge alexives/298780/enable_usage_ping_on_geo_secondary into master

What does this MR do?

This merge request enables usage ping messages from geo secondaries, and identifies pings as coming from secondaries.

High Level Bullet Points:

  • We could potentially send data that is duplicated with some differences from non-database provided sources
  • The duplicate data would be marked as coming from a geo located secondary server with a flag settings: { geo_secondary: true }
  • The duplicate data would generally be ignored unless you're looking for information about secondaries.
  • Data derived from the database or files on disk sould be (nearly) identical
  • Data derived from prometheus, redis, or other sources would differ and would probably pertain the usage of those things on a geo secondary

Risks

This would result in any geo secondary with usage ping enabled to send duplicate data pulled from the database. While that data would be identified as being from a geo secondary, it is possible that data could be contaminated if queries aren't updated

Benefits

Currently we have less than ideal insight into how people are using their geo secondaries and the adoption rate of features developed there. While we can get some of this from the primary (how many projects are replicated, how many replicas are there, how many users have active oauth credentials for secondaries, ect), there's a lot more data we wish we could see that could only fesably be provided by the secondary itself like tracking how many people are pushing or fetching from git, or some other metrics.

Approach

After a discussion about 6 months ago in https://gitlab.com/gitlab-data/analytics/-/issues/4459#note_379239009, it was suggested that we would flag the data rather than updating the uuid in opes that this would be more clear and reliable than trying to track an altered uuid.

we should have a clean separate metric which shows the geo node type the usage ping is coming from. geo_node: "primary" or geo_node: "secondary" or geo_secondary: true rather than reading into the uuid differences, which might end up creating other issues

Relates to #298780 (closed)

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Alex Ives

Merge request reports