Skip to content

Improve Gitlab::Elastic::Indexer update logging and document possible use of logs

Dylan Griffith requested to merge elasticsearch-disaster-recovery-logging into master

What does this MR do?

This MR updates the logging format used in Gitlab::Elastic::Indexer. These logs can be useful in the event of disaster recovery by replaying the updates during a period of time from logging information. The previous format was being blocked before ingestion into our logging infrastructure likely due to the wiki field having an incorrect format as field names cannot be re-used in our logs without them being the same type. Either way this new format should not conflict and is more machine readable than the previous one.

This MR also adds some documentation section explaining how the logs could theoretically be used for disaster recovery. This is a first step to increase transparency about an idea we've already considered while the longer term option should be to actually test out this theoretical technique and possibly automate parts of this but that is much trickier so it makes sense to at least write it down for now.

This MR doesn't include a new test since we aren't commonly writing test cases for logging unless they exist in an isolated branch of logic that needs to be tested. A major reason for not always writing a new test case for every log message is that it adds to the cost of logging which should be cheap for developers to encourage adding logs as much as possible. The most important thing is to ensure the logging code is at least executed in a test so we know it doesn't have some typo leading to an exception.

Screenshots (strongly suggested)

{"severity":"DEBUG","time":"2020-12-23T03:00:13.995Z","correlation_id":"01ET6RMBFS8P4YXRHJ29JHA3TD","message":"indexing_commit_range","project_id":48,"from_sha":"810158a57d94eba7356a54e6ab416d1ebae6f9d9","to_sha":"35e885158866584600700b36e48b9131075ec133","index_wiki":false}

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Dylan Griffith

Merge request reports