Add support for setting the artifact/cache compression level
What does this MR do?
Allows ARTIFACT_COMPRESSION_LEVEL
and CACHE_COMPRESSION_LEVEL
to be set to
- fastest
- fast
- default
- slow
- slowest
to control the compression ratio/performance of the active archiver.
It's intended that not every archiver has to support these different levels, but they made a "best effort" to apply them. Therefore, an unsupported level has no warning.
Only fastzip
currently supports these levels, so they will only be useful in combination with FF_USE_FASTZIP
for now.
Why was this MR needed?
Adjusting compression levels can allow users to cater for their environment and data: Disk, network, memory and CPU performance all play a part in what is overall faster: better compression or faster/no compression.
What's the best way to test this MR?
Compiling the binary and running cache-archiver
at different levels should produce archives at different sizes and speeds.
time FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=fastest ./gitlab-runner cache-archiver --path ".git/**" --file fastest.zip 2> /dev/null && du -h fastest.zip
time FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=fast ./gitlab-runner cache-archiver --path ".git/**" --file fast.zip 2> /dev/null && du -h fast.zip
time FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=default ./gitlab-runner cache-archiver --path ".git/**" --file default.zip 2> /dev/null && du -h default.zip
time FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=slow ./gitlab-runner cache-archiver --path ".git/**" --file slow.zip 2> /dev/null && du -h slow.zip
time FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=slowest ./gitlab-runner cache-archiver --path ".git/**" --file slowest.zip 2> /dev/null && du -h slowest.zip
Example output:
FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=fastest ./gitlab-runner 2 0.16s user 0.23s system 113% cpu 0.342 total
189M fastest.zip
FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=fast ./gitlab-runner cache-archiver 0.53s user 0.86s system 179% cpu 0.775 total
188M fast.zip
FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=default ./gitlab-runner 2 0.62s user 0.91s system 186% cpu 0.821 total
188M default.zip
FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=slow ./gitlab-runner cache-archiver 2.23s user 1.35s system 179% cpu 1.996 total
193M slow.zip
FF_USE_FASTZIP=1 CACHE_COMPRESSION_LEVEL=slowest ./gitlab-runner 2 2.36s user 1.37s system 181% cpu 2.059 total
193M slowest.zip
What are the relevant issue numbers?
closes #4907 (closed)