Skip to content

Improve backup logging

James Fargher requested to merge backup_logging into master

What does this MR do and why?

Fixes #353942 (closed)

During !80578 (merged) it was noticed that some of the logging generated by backup.rake isn't actually all that useful. This MR tries to remedy that, and moves a lot more logging to the new backup log file.

Screenshots or screen recordings

Before:

2022-03-17 13:19:35 +1300 -- done
2022-03-17 13:19:35 +1300 -- Dumping uploads ... 
2022-03-17 13:19:35 +1300 -- done
2022-03-17 13:19:35 +1300 -- Dumping builds ... 
2022-03-17 13:19:35 +1300 -- done
2022-03-17 13:19:35 +1300 -- Dumping artifacts ... 
2022-03-17 13:19:36 +1300 -- done
2022-03-17 13:19:36 +1300 -- Dumping pages ... 
2022-03-17 13:19:36 +1300 -- done
2022-03-17 13:19:36 +1300 -- Dumping lfs objects ... 
2022-03-17 13:19:36 +1300 -- done
2022-03-17 13:19:36 +1300 -- Dumping terraform states ... 
2022-03-17 13:19:36 +1300 -- done
2022-03-17 13:19:36 +1300 -- Dumping container registry images ... 
2022-03-17 13:19:36 +1300 -- [DISABLED]
2022-03-17 13:19:36 +1300 -- Dumping packages ... 
2022-03-17 13:19:36 +1300 -- done
Creating backup archive: 1647476342_2022_03_17_14.9.0-pre_gitlab_backup.tar ... done
Uploading backup archive to remote storage  ... skipped
Deleting tmp directories ... done
done
done
done
done
done
done
done
done
done
Deleting old backups ... skipping
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
Backup task is done.

After:

...
2022-03-21 10:01:06 +1300 -- Dumping repositories ... done
2022-03-21 10:01:06 +1300 -- Dumping uploads ... 
2022-03-21 10:01:06 +1300 -- Dumping uploads ... done
2022-03-21 10:01:06 +1300 -- Dumping builds ... 
2022-03-21 10:01:06 +1300 -- Dumping builds ... done
2022-03-21 10:01:06 +1300 -- Dumping artifacts ... 
2022-03-21 10:01:07 +1300 -- Dumping artifacts ... done
2022-03-21 10:01:07 +1300 -- Dumping pages ... 
2022-03-21 10:01:07 +1300 -- Dumping pages ... done
2022-03-21 10:01:07 +1300 -- Dumping lfs objects ... 
2022-03-21 10:01:07 +1300 -- Dumping lfs objects ... done
2022-03-21 10:01:07 +1300 -- Dumping terraform states ... 
2022-03-21 10:01:07 +1300 -- Dumping terraform states ... done
2022-03-21 10:01:07 +1300 -- Dumping container registry images ... [DISABLED]
2022-03-21 10:01:07 +1300 -- Dumping packages ... 
2022-03-21 10:01:07 +1300 -- Dumping packages ... done
2022-03-21 10:01:07 +1300 -- Creating backup archive: 1647810033_2022_03_21_14.9.0-pre_gitlab_backup.tar ... 
2022-03-21 10:01:08 +1300 -- Creating backup archive: 1647810033_2022_03_21_14.9.0-pre_gitlab_backup.tar ... done
2022-03-21 10:01:08 +1300 -- Uploading backup archive to remote storage  ... [SKIPPED]
2022-03-21 10:01:08 +1300 -- Deleting tar staging files ... 
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/backup_information.yml
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/db
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/repositories
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/uploads.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/builds.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/artifacts.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/pages.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/lfs.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/terraform_state.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/packages.tar.gz
2022-03-21 10:01:08 +1300 -- Deleting tar staging files ... done
2022-03-21 10:01:08 +1300 -- Deleting old backups ... [SKIPPED]
2022-03-21 10:01:08 +1300 -- Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
2022-03-21 10:01:08 +1300 -- Backup task is done.

How to set up and validate locally

  1. Take a backup
    $ bundle exec rake gitlab:backup:create
    ...
    2022-03-17 13:17:11 +1300 -- Creating backup archive: 1647476196_2022_03_17_14.9.0-pre_gitlab_backup.tar ... done
    ...
  2. Restore a backup. Note that BACKUP does not take a path, it is the name of the tar file without the suffix _gitlab_backup.tar.
    $ bundle exec rake gitlab:backup:restore BACKUP=1647476196_2022_03_17_14.9.0-pre

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by James Fargher

Merge request reports

Loading