Create API to update upcoming reconciliations
What does this MR do?
This is part of #215187
When we implement Quarterly GitLab.com Seat Reconciliation & Billing for SaaS, group owners need to be aware if they have a upcoming charge.
The reconciliations happen in CustomersDot. To show alert to group owners in Gitlab application, we need to get upcoming reconciliations data from CustomersDot. We will save the upcoming reconciliations in the table upcoming_reconciliations
, so that Gitlab application will check local table upcoming_reconciliations
:
- it allows better page loading performance
- it makes Gitlab.com more resilient, in the case of CustomersDot is slow or down
There will be an CustomersDot cron job(MR https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3314) to collect the upcoming reconciliations data. It is estimated to have hundreds of upcoming reconciliations everyday. The CustomersDot cron job will pass all these data together to Gitlab in a single API(which we are creating in this MR) call.
This MR creates an internal Gitlab API to update upcoming reconciliations. As mentioned above, this API is called by the CustomersDot daily cron job. This API accepts an array of upcoming reconciliations. Each array item(one upcoming reconciliation) contains: namespace_id
, next_reconciliation_date
, display_alert_from
.
- If one namespace_id exists in upcoming_reconciliations table, the existing record is updated. Othewise it creates a new record.
- It validate the records, and then used bulk_upsert! to upsert all records together.
- If all records are valid and bulk_upsert! successfully, it returns 200.
- If some records are invalid, it will bulk_upsert! the valid records only, and returns 400 error with the
invalid { namespace_id => errors}
. - bulk_upsert! throw unexpected error, it returns 400 error with both
bulk_upsert exception message
and theinvalid { namespace_id => errors}
.
- Both
invalid records found during validation
andbuld_upsert! error
are unexpected. In other words, the API should always succeed with 200 status. The 400 error code means, something really exceptional happens(such as, some data integrity issue between CustomersDot and Gitlab, which is rare but could happen), in this case we should open new issues to analyse and fix.
Does this MR meet the acceptance criteria?
Conformity
-
I have included a changelog entry, or it's not needed. (Does this MR need a changelog?) -
I have added/updated documentation, or it's not needed. (Is documentation required?) -
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) -
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) -
I have self-reviewed this MR per code review guidelines. -
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) -
I have followed the style guides.
Availability and Testing
-
I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) -
I have tested this MR in all supported browsers, or it's not needed. -
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.
Related to #215187