backup: Refactor Pipeline struct
While working through #5357 (closed), I identified a few refactoring opportunities to improve the backup and restore code.
Central to this mechanism is a Pipeline
construct that receives jobs streamed via stdin to the gitaly-backup backup|restore
commands. Each job defines a repository to be backed up or restored. The pipeline is responsible for queuing these requests, possibly executing requests concurrently, and reporting on errors.
Previously there were two pipeline implementations along with an interface to abstract them; a LoggingPipeline
and a ParallelPipeline
that adapts it to enable concurrency. Since a parallel pipeline with a single worker is effectively a sequential pipeline, this MR refactors the pipeline code by consolidating both implementations into a single struct. This makes the overall code a little easier to parse by reducing the amount of cross-object interactions. The MR also makes some changes to how the amount of concurrency is enforced within the pipeline.