Improve backup logging
What does this MR do and why?
Fixes #353942 (closed)
During !80578 (merged) it was noticed that some of the logging generated by backup.rake isn't actually all that useful. This MR tries to remedy that, and moves a lot more logging to the new backup log file.
Screenshots or screen recordings
Before:
2022-03-17 13:19:35 +1300 -- done
2022-03-17 13:19:35 +1300 -- Dumping uploads ...
2022-03-17 13:19:35 +1300 -- done
2022-03-17 13:19:35 +1300 -- Dumping builds ...
2022-03-17 13:19:35 +1300 -- done
2022-03-17 13:19:35 +1300 -- Dumping artifacts ...
2022-03-17 13:19:36 +1300 -- done
2022-03-17 13:19:36 +1300 -- Dumping pages ...
2022-03-17 13:19:36 +1300 -- done
2022-03-17 13:19:36 +1300 -- Dumping lfs objects ...
2022-03-17 13:19:36 +1300 -- done
2022-03-17 13:19:36 +1300 -- Dumping terraform states ...
2022-03-17 13:19:36 +1300 -- done
2022-03-17 13:19:36 +1300 -- Dumping container registry images ...
2022-03-17 13:19:36 +1300 -- [DISABLED]
2022-03-17 13:19:36 +1300 -- Dumping packages ...
2022-03-17 13:19:36 +1300 -- done
Creating backup archive: 1647476342_2022_03_17_14.9.0-pre_gitlab_backup.tar ... done
Uploading backup archive to remote storage ... skipped
Deleting tmp directories ... done
done
done
done
done
done
done
done
done
done
Deleting old backups ... skipping
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
Backup task is done.
After:
...
2022-03-21 10:01:06 +1300 -- Dumping repositories ... done
2022-03-21 10:01:06 +1300 -- Dumping uploads ...
2022-03-21 10:01:06 +1300 -- Dumping uploads ... done
2022-03-21 10:01:06 +1300 -- Dumping builds ...
2022-03-21 10:01:06 +1300 -- Dumping builds ... done
2022-03-21 10:01:06 +1300 -- Dumping artifacts ...
2022-03-21 10:01:07 +1300 -- Dumping artifacts ... done
2022-03-21 10:01:07 +1300 -- Dumping pages ...
2022-03-21 10:01:07 +1300 -- Dumping pages ... done
2022-03-21 10:01:07 +1300 -- Dumping lfs objects ...
2022-03-21 10:01:07 +1300 -- Dumping lfs objects ... done
2022-03-21 10:01:07 +1300 -- Dumping terraform states ...
2022-03-21 10:01:07 +1300 -- Dumping terraform states ... done
2022-03-21 10:01:07 +1300 -- Dumping container registry images ... [DISABLED]
2022-03-21 10:01:07 +1300 -- Dumping packages ...
2022-03-21 10:01:07 +1300 -- Dumping packages ... done
2022-03-21 10:01:07 +1300 -- Creating backup archive: 1647810033_2022_03_21_14.9.0-pre_gitlab_backup.tar ...
2022-03-21 10:01:08 +1300 -- Creating backup archive: 1647810033_2022_03_21_14.9.0-pre_gitlab_backup.tar ... done
2022-03-21 10:01:08 +1300 -- Uploading backup archive to remote storage ... [SKIPPED]
2022-03-21 10:01:08 +1300 -- Deleting tar staging files ...
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/backup_information.yml
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/db
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/repositories
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/uploads.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/builds.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/artifacts.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/pages.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/lfs.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/terraform_state.tar.gz
2022-03-21 10:01:08 +1300 -- Cleaning up /home/james/src/gitlab-org/gdk/gitlab/tmp/backups/packages.tar.gz
2022-03-21 10:01:08 +1300 -- Deleting tar staging files ... done
2022-03-21 10:01:08 +1300 -- Deleting old backups ... [SKIPPED]
2022-03-21 10:01:08 +1300 -- Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
2022-03-21 10:01:08 +1300 -- Backup task is done.
How to set up and validate locally
- Take a backup
$ bundle exec rake gitlab:backup:create ... 2022-03-17 13:17:11 +1300 -- Creating backup archive: 1647476196_2022_03_17_14.9.0-pre_gitlab_backup.tar ... done ...
- Restore a backup. Note that
BACKUP
does not take a path, it is the name of the tar file without the suffix_gitlab_backup.tar
.$ bundle exec rake gitlab:backup:restore BACKUP=1647476196_2022_03_17_14.9.0-pre
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by James Fargher