Skip to content

Handle some WebHookService errors

Luke Duncalfe requested to merge 337708-handle-web_hook_service-errors into master

What does this MR do?

WebHookWorker is currently experiencing ~12k unhandled errors per week.

The errors can be seen in Kibana of counts over the past 7 days:

image

This MR handles the following errors:

EOFError

EOFError can be due to using port 443 without SSL, or transient network failures.

The SSL configuration problem is unlikely because HTTParty handles setting this, so it's likely this error is due to the remote server terminating the connection.

ActiveRecord::RecordNotFound

Must happen soon after a webhook is deleted.

Errno::ENETUNREACH

Is handled the same as Errno::EHOSTUNREACH is, in Gitlab::HTTP.

#337708 (closed)


By handling EOFError and Errno::ENETUNREACH they will now be logged and contribute to the failing WebHook auto-disabling feature.

We avoid ActiveRecord::RecordNotFound errors by having the worker return when the record is not found.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Related to #337708 (closed)

Edited by Luke Duncalfe

Merge request reports

Loading