Skip to content

Handle GRPC::Core::CallError exception

Vasilii Iakliushin requested to merge 414003_fix_repository_mirror_failures into master

What does this MR do and why?

Contributes to #414003 (closed)

Problem

GRPC gem update to version 1.55 changed a returned exception class due to bug: https://github.com/grpc/grpc/issues/33283.

GRPC::Core::CallError is not inherited from StandardError. As a result, the Sidekiq job is killed before we update the job status in the database.

It leads to jobs being stuck with started status.

Detailed explanation: gitlab-com/gl-infra/production#14878 (comment 1427151384)

Sentry error: https://new-sentry.gitlab.net/organizations/gitlab/issues/167736/

Solution

Before the gem code is fixed, we can manually handle the exception to prevent Sidekiq job from dying.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Vasilii Iakliushin

Merge request reports