Store correlation ID with import state & expose via API
What does this MR do?
In #210522 (closed) we are looking to expose more information about import status and possible failures via the REST API to facilitate diagnostics in case of problems.
This MR is an MVC toward that goal by capturing and exposing the correlation_id
for each attempted import.
The correlation ID is useful for two reasons:
- To correlate exceptions with import executions via sidekiq or otherwise (we sometimes have to run imports manually via rake).
- To find all
ImportFailure
records related to a particular import. Theoretically, at least currently, the project ID should suffice to tie these records together, but since we are looking to allow incremental imports in the future, there might actually be multiple failure sets that exist for a single project, and we would likely only want to know those of the latest import attempt. Since correlation IDs are unique per job, and moreover not ephemeral likejid
s, they seem like a good choice.
Implementation wise:
- add a
correlation_id
attribute toProjectImportState
- I did not index it because there is currently a 1-1 relationship between projects & state and even in the future we wouldn't expect to have more than a handful of import attempts per project
- save the current correlation ID whenever an import starts
- GET + POST endpoints for imports now render the
correlation_id
Example:
{
"id": 39,
"description": "Ut soluta dignissimos sapiente atque nam eius perferendis.",
"name": "test-import",
"name_with_namespace": "Administrator / test-import",
"path": "test-import",
"path_with_namespace": "root/test-import",
"created_at": "2020-04-03T12:48:12.314Z",
"import_status": "started",
"correlation_id": "mezklWso3Za"
}
Migration output:
$ bin/rake db:migrate:down VERSION=20200402124802
== 20200402124802 AddCorrelationIdToProjectImportState: reverting =============
-- remove_column(:project_mirror_data, :correlation_id_value)
-> 0.0014s
== 20200402124802 AddCorrelationIdToProjectImportState: reverted (0.0083s) ====
$ bin/rake db:migrate:up VERSION=20200402124802
== 20200402124802 AddCorrelationIdToProjectImportState: migrating =============
-- add_column(:project_mirror_data, :correlation_id_value, :string, {:limit=>128})
-> 0.0009s
== 20200402124802 AddCorrelationIdToProjectImportState: migrated (0.0060s) ====
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines - [-] Merge request performance guidelines
-
Style guides -
Database guides - [-] Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Tested locally via curl
.
There seem to be essentially zero tests for grape entities -- I didn't add one either. Not sure if that's just broken windows or whether we specifically decided not to test these?
Edited by 🤖 GitLab Bot 🤖