BulkImports: Add more information to the logs
What does this MR do?
tl;dr
Add more information to improve debuging/filtering of the BulkImport logs:
- The
bulk_import_id
can be used to filter/check a whole import log stream -
context.extra
contain sub-relation reference. For instance, when importing the Epic Emoji Awards, the current epiciid
is kept as an extra.
long version
When a user starts a new Gitlab Group Migration a BulkImport
and a BulkImports::Entity
is created for each Group to migrate.
If the group has subgroups, each Subgroup will then have a BulkImports::Entity
.
Therefore, having the bulk_import_entity_id
in the logs is very helpful, but to be able to filter in a larger level, per migration, the bulk_import_id
is also very relevant in the logs.
When importing subrelations, like Epic's Emoji Awards, the current epic is passed in the workflow as an extra
field, in the BulkImports::Pipeline::Context
object. This way,
logging the extra
data is required to find subrelations logs.
Related to: #323639 (closed)
Screenshots (strongly suggested)
- before
{
"severity":"INFO",
"time":"2021-03-04T23:06:20.737Z",
"correlation_id":"bfdf54e51dabb567947c7a5908b0ddce",
"bulk_import_entity_id":73,
"bulk_import_entity_type":"group_entity",
"pipeline_class":"EE::BulkImports::Groups::Pipelines::EpicAwardEmojiPipeline"
"message":"Pipeline started"
}
*after
{
"severity":"INFO",
"time":"2021-03-04T23:06:20.737Z",
"correlation_id":"bfdf54e51dabb567947c7a5908b0ddce",
"bulk_import_id":54,
"bulk_import_entity_id":73,
"bulk_import_entity_type":"group_entity",
"pipeline_class":"EE::BulkImports::Groups::Pipelines::EpicAwardEmojiPipeline",
"context_extra":{"epic_iid":1},
"message":"Pipeline started"
}
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. -
I have not included a changelog entry because it's not a user facing change. Changing only logs.
-
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team