Ensure all errors are logged in Group Import

What does this MR do?

Addresses: #38101 (closed)

Ensures that all errors raised in the Group Import/Export are logged.

Also ensures that all errors are logged to the same place (Sentry and exceptions_json.log, which we started doing in this issue: #32906 (closed)).

Currently we have three places for logging import/export errors:

1. importer.log
2. exceptions_json.log + sentry
3. ImportFailure database records

Example Files

Export

1. Success

   importer.log

   {"severity":"INFO","time":"2020-02-24T14:18:23.751Z","correlation_id":"jskSc3r7xf4","group_id":22,"group_name":"Gitlab Org","message":"Group Import/Export: Export succeeded"}

2. Error

   importer.log

   {"severity":"ERROR","time":"2020-02-28T12:52:59.476Z","correlation_id":null,"group_id":22,"group_name":"Gitlab Org","error":"User with ID: 45 does not have permission to Group Gitlab Org with ID: 22.","message":"Group Import/Export: Export failed"}

   exceptions_json.log

   {"severity":"ERROR","time":"2020-02-28T12:52:59.475Z","correlation_id":null,"extra.server":{"os":{"name":"Darwin","version":"Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64","build":"19.3.0","kernel_version":"Darwin fennec.local 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64"},"runtime":{"name":"ruby","version":"ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]"}},"extra.importer":"Import/Export","extra.exportable_id":22,"extra.exportable_path":"gitlab-org","exception.class":"Gitlab::ImportExport::Error","exception.message":"User with ID: 45 does not have permission to Group Gitlab Org with ID: 22."}
   

Import

1. Success

   importer.log

   {"severity":"INFO","time":"2020-02-28T13:01:45.964Z","correlation_id":"as4U046X98a","group_id":62,"group_name":"error-test","message":"Group Import/Export: Import succeeded"}

2. Error on sub-relations

   importer.log

   {"severity":"INFO","time":"2020-02-28T13:03:32.837Z","correlation_id":"YyEOUgdwrZ2","group_id":63,"group_name":"error-test","message":"Group Import/Export: Import succeeded"}

   exceptions_json.log

   {"severity":"ERROR","time":"2020-02-28T13:03:32.583Z","correlation_id":"YyEOUgdwrZ2","extra.server":{"os":{"name":"Darwin","version":"Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64","build":"19.3.0","kernel_version":"Darwin fennec.local 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64"},"runtime":{"name":"ruby","version":"ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]"}},"extra.sidekiq":{"class":"GroupImportWorker","args":[45,63],"retry":3,"queue":"group_import","backtrace":5,"jid":"a13c1482b97e5e79c7e3e172","created_at":1582895011.774604,"meta.user":"external","meta.caller_id":"/api/:version/groups/import","correlation_id":"YyEOUgdwrZ2","uber-trace-id":"3f49c325c2ba0403:6a17a9aac92dffee:40976065be212a7d:1","enqueued_at":1582895011.775948},"extra.source":"process_relation_item!","extra.relation_key":"[FILTERED]","extra.relation_index":0,"extra.retry_count":0,"extra.group_id":63,"exception.class":"ActiveRecord::RecordInvalid","exception.message":"Validation failed: Author can't be blank, Title can't be blank","exception.backtrace":["lib/gitlab/import_export/relation_tree_restorer.rb:77:in `block in process_relation_item!'","lib/gitlab/import_export/import_failure_service.rb:26:in `block in with_retry'","lib/gitlab/import_export/import_failure_service.rb:25:in `with_retry'","lib/gitlab/import_export/relation_tree_restorer.rb:76:in `process_relation_item!'","lib/gitlab/import_export/relation_tree_restorer.rb:64:in `process_relation!'","lib/gitlab/import_export/relation_tree_restorer.rb:50:in `each'","lib/gitlab/import_export/relation_tree_restorer.rb:50:in `create_relations!'","lib/gitlab/import_export/relation_tree_restorer.rb:29:in `block (2 levels) in restore'","lib/gitlab/import_export/relation_tree_restorer.rb:27:in `block in restore'","lib/gitlab/import_export/relation_tree_restorer.rb:26:in `restore'","lib/gitlab/import_export/group/tree_restorer.rb:24:in `restore'","app/services/groups/import_export/import_service.rb:15:in `execute'","app/workers/group_import_worker.rb:13:in `perform'","lib/gitlab/with_request_store.rb:7:in `with_request_store'","lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'"]}

   ImportFailure

   #<ImportFailure id: 1, relation_index: 0, project_id: nil, created_at: "2020-02-28 13:03:32", relation_key: [FILTERED], exception_class: "ActiveRecord::RecordInvalid", correlation_id_value: "YyEOUgdwrZ2", exception_message: "Validation failed: Author can't be blank, Title ca...", retry_count: 0, group_id: 63, source: "process_relation_item!">

3. Error overall

   importer.log

   {"severity":"ERROR","time":"2020-02-28T13:08:30.715Z","correlation_id":"6f06a7dc303b917ce89587156912c03e","group_id":64,"group_name":"error-test","message":"Group Import/Export: Errors occurred, see 'exceptions_json.log' for details"}

   exceptions_json.log

   {"severity":"ERROR","time":"2020-02-28T13:08:11.822Z","correlation_id":"qztglhd6Vz7","extra.server":{"os":{"name":"Darwin","version":"Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64","build":"19.3.0","kernel_version":"Darwin fennec.local 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64"},"runtime":{"name":"ruby","version":"ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]"}},"extra.sidekiq":{"class":"GroupImportWorker","args":[45,64],"retry":3,"queue":"group_import","backtrace":5,"jid":"23aec138b686ad56e72f2011","created_at":1582895291.361981,"meta.user":"external","meta.caller_id":"/api/:version/groups/import","correlation_id":"qztglhd6Vz7","uber-trace-id":"9b59c3d1a1d728e:f7ce64cb245a0164:81d332524d383130:1","enqueued_at":1582895291.363585},"extra.importer":"Import/Export","extra.exportable_id":64,"extra.exportable_path":"1582895291","exception.class":"NoMethodError","exception.message":"undefined method `import_file' for nil:NilClass","exception.backtrace":["lib/gitlab/import_export/file_importer.rb:67:in `copy_archive'","lib/gitlab/import_export/file_importer.rb:28:in `import'","lib/gitlab/import_export/file_importer.rb:14:in `import'","app/services/groups/import_export/import_service.rb:30:in `import_file'","app/services/groups/import_export/import_service.rb:15:in `execute'","app/workers/group_import_worker.rb:13:in `perform'","lib/gitlab/with_request_store.rb:7:in `with_request_store'","lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'"]}

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• ~~Label as security and @ mention @gitlab-com/gl-security/appsec~~
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team