Skip to content

Add a separate logger for Sidekiq clients

Bob Van Landuyt requested to merge reprazent/gitlab-ee:bvl-sidekiq-logger-structured into master

What does this MR do?

On omnibus instances running regular sidekiq (not sidekiq cluster) these messages come in the same file. So this is no change to the end user. Who can view this file in the admin interface.

This also adds a Gitlab::SidkiqLogging::ClientLogger, will allow Sidekiq client, in our case Rails, can also emit log messages related to sidekiq jobs.

Instead of having these messages printed to STDOUT (which results in production.log), this changes the Sidekiq.logger to a separate writing to log/sidekiq_client.log. This allows us to ingest these messages in the same index as the messages the Sidekiq process emits. Having them in the same index allows us to gather statistics about jobs that haven't been processed.

The logger will follow the structure configured for sidekiq in gitlab.rb, so when json logging is enabled, the messages would look like this:

{"severity":"INFO","time":"2020-03-05T15:45:01.349Z","message":"hello world"}

Otherwise, we'll be writing that message like this:

I, [2020-03-05T17:16:58.541421 #15874]  INFO -- : hello world

Both messages were written from a rails console (a Sidekiq client). The messages emitted from sidekiq-server remain the same and go to their STDOUT.

This is part of gitlab-com/gl-infra/scalability#200 (closed) which is needed for gitlab-com/gl-infra/scalability#42 (closed) and already used in !26399 (merged)

The sidekiq.log is currently not ingested by ES, as we get the sidekiq logs directly from the sidekiq-cluster STDOUT, so this wouldn't change anything for GitLab.com.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Edited by 🤖 GitLab Bot 🤖

Merge request reports