Enable parallel testing on test-on-gdk pipeline

What does this MR do and why?

This MR enables parallel running, relying on the parallel_tests gem. ⚠ This is still an experimental way of running our tests and we need to carefully observe potential side-effects. We have a main pull-the-plug variable to stop running in parallel if we observe issues.

This enables the runs on test-on-gdk and package-and-test pipelines.

Guide for the reviewer

These are a lot of changes to review but I left a few guiding points on the code directly.

Credits

Kudos to @acunskis that improved a lot the original implementation MR and fixed several blindpots 🙇

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.