Improve error reporting in project import
What does this MR do?
Helps address this issue: #121732 (closed)
Prior to this change, exceptions raised in project import were only logged to production.log
which is unstructured, so these errors could not be found in Sentry or easily in Kibana. This makes debugging issues like #121732 (closed) really hard.
This MR adds tracking of all exceptions from project import to Sentry and to exceptions_json.log
-> using the method introduced in #32906 (closed). There is another issue open to go through and remove all instances of Rails.logger.(error|info)
: #209057 (closed)
After making these changes I receive the following in my development environment:
exceptions_json.log
{"severity":"ERROR","time":"2020-03-20T11:55:06.144Z","correlation_id":"V1lTYa0CHV2","extra.server":{"os":{"name":"Darwin","version":"Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64","build":"19.3.0","kernel_version":"Darwin fennec.local 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64"},"runtime":{"name":"ruby","version":"ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]"}},"extra.sidekiq":{"retry":false,"args":[30],"class":"RepositoryImportWorker","queue":"repository_import","backtrace":5,"status_expiration":54000,"memory_killer_memory_growth_kb":50,"memory_killer_max_memory_growth_kb":300000,"jid":"c675520af04f4ff21184c136","created_at":1584705303.7070348,"meta.user":"root","meta.caller_id":"Import::GitlabProjectsController#create","correlation_id":"V1lTYa0CHV2","uber-trace-id":"********","enqueued_at":1584705303.716969},"exception.class":"RuntimeError","exception.message":"this is an error in the after import service","exception.backtrace":["app/services/projects/after_import_service.rb:23:in `execute'","app/models/project_import_state.rb:61:in `block (3 levels) in \u003cclass:ProjectImportState\u003e'","lib/after_commit_queue.rb:39:in `instance_eval'","lib/after_commit_queue.rb:39:in `_run_after_commit_queue'","app/models/project.rb:1825:in `after_import'","ee/app/models/ee/project.rb:603:in `after_import'","app/workers/repository_import_worker.rb:36:in `perform'","ee/app/workers/ee/repository_import_worker.rb:9:in `perform'","lib/gitlab/with_request_store.rb:7:in `with_request_store'","lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'"]}
Sentry
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
~~Label as security and @ mention @gitlab-com/gl-security/appsec
~~ -
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Closes #121732 (closed)