Parallelize integration script
What does this MR do and why?
This MR reduces the execution time of the integration script (spec/scripts/integration.sh
) execution by 3.3x.
Version | Runtime (m:ss) |
---|---|
Original implementation | 3:17 |
Secuential mode in this MR | 2:13 |
Parallel mode in this MR | 1:00 |
Screenshots or screen recordings
The original script before these changes:
$ git checkout main
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
$ time ./spec/scripts/integration.sh
...
./spec/scripts/integration.sh 6.05s user 2.20s system 4% cpu 3:17.82 total
Sequential mode in this branch:
$ time ./spec/scripts/integration.sh
integration.sh: Using sequential mode [default]
...
Elapsed Time: 134 seconds
./spec/scripts/integration.sh 4.07s user 1.56s system 4% cpu 2:13.61 total
Parallel mode in this branch:
$ time PARALLEL=1 ./spec/scripts/integration.sh
integration.sh: Using parallel mode
...
Elapsed Time: 60 seconds
PARALLEL=1 ./spec/scripts/integration.sh 3.93s user 1.78s system 9% cpu 1:00.60 total
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
Edited by Dmitry Gruzd