Snowflake data warehouse downstream impact due to db_structure changes
Currently @gitlab-data/engineers
is tagged in the MR, if a change is made to the db/structure\.sql
file. This is automatically done via the danger bot, set in this danger file. This is raising visibility for the data team on all and potential GitLab.com db structure changes.
All team members in @gitlab-data/engineers
got pinged, results in quite some noise and lag of ownership. This could result in missing any vital db structure change and subsequently an outage on the Gitlab.com data pipeline.
Proposed new workflow:
- Instead of tagging
@gitlab-data/engineers
a labelData Warehouse::Impact Check
via the danger file as call to action for the data team is added to the MR. - Data Team will check for MRs with label
Data Warehouse::Impact Check
. - If the changes to the SQL file are not causing a break in the operation, the label will be changed to
Data Warehouse::Not Impacted
. - If the changes to the SQL file causing a break in the operation, the Label will be changed to
Data Warehouse::Impacted
- Data Team will make necessary adjustments to the downstream data flow.
Edited by Dennis van Rooijen