Skip to content

Retrieve lost ServicePings

Michał Wielich requested to merge retrieve-lost-servicepings into main

Summary

Related to gitlab-org/gitlab#486036 (closed)

I decided to rebuild the data using a migration, because using POST requests would not scale well [I believe we would have around 150k records to send, as per this Snowflake query:]

select 
parse_json(raw_usage_data_payload):version as version,
parse_json(raw_usage_data_payload):counts:epics_with_parents as parents,
parse_json(raw_usage_data_payload):counts:work_item_parent_links_epic_parent as parents2,
parse_json(raw_usage_data_payload):counts:work_item_parent_links as parents3,
*,
from version_db.version_raw_usage_data_source
where 
(date(created_at) > '2024-07-15') AND
parents3 is not null
order by created_at desc

To test this migration, first follow the steps in !181 (merged) to create a RawUsageData entry without a UsageData entry. Then, run the migration and verify that a new UsageData and RawUsageData entries have been created with correct attributes.

  • I made sure there are no unverified deployments on staging in #version-gitlab-com Slack channel before merging this MR

Merge request reports