Skip to content

Resolve "Problems arise if there are multiple parallel Omnibus backup/restore processes"

Javiera Tapia requested to merge 362593-problems-arise-if-there-are-multiple-parallel-omnibus-backup-restore-processes into master

What does this MR do and why?

Since running multiple backups and restores in parallel is unsafe, this MR implements a PID lock mechanism to detect if there is another backup and restore process in progress. If that is the case, we terminate the inflight process until the previous operation finishes.

It also provides Rubocop fixes for backup_rake_spec.rb and removal from .rubocop_todo:

  • .rubocop_todo/layout/first_hash_element_indentation.yml
  • .rubocop_todo/layout/line_length.yml
  • .rubocop_todo/layout/space_inside_parens.yml
  • .rubocop_todo/rspec/context_wording.yml
  • .rubocop_todo/rspec/predicate_matcher.yml
  • .rubocop_todo/style/numeric_literal_prefix.yml
  • .rubocop_todo/style/percent_literal_delimiters.yml

How to set up and validate locally

  1. Run multiple backup and restore rake tasks in parallel
  2. Validate that the first process creates a tmp PID file while running: /tmp/backup_restore.pid
  3. While the PID file exists, then the new backup and restore process should terminate and log a warning message
  4. If the first process finishes, then the PID file should be deleted to allow the following process to run

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #362593 (closed)

Edited by Javiera Tapia

Merge request reports