Skip to content

E2E: Wait for duration defined in timeout for successful? check

John McDonnell requested to merge jmd/e2e-wait-for-build-traces into master

What does this MR do and why?

E2E: Wait for duration defined in timeout for successful? check

We've noted that in live environments that the test jobs can be in a pending status as there are no active runners available for to pick up the test job. Given that the tests are configured to wait 800sec, we should consider using the full allowance to ensure that they won't become available before actually failing the test.

The class of failure appears relatively infrequently, (I found 5 failures over the past month) but that is also frequently enough that the underlying cause needs to be addressed or better understood to help avoid these failures.
I also noted that the failures appear to exclusively appear in our live environments which is why I'm wondering if we might be able to work around this by just waiting a little while longer for the runner to become available?

Sample failures:

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

I've found it somewhat possible to reproduce the behaviour of the aforementioned failures, by applying the following diff to skip starting the runners. Prior to this change this would default to 60secs before timing out and failing. With this change it will wait 800sec (or whatever is defined in the timeout param)

NB - this change doesn't investigate the reason for the underlying failure, but rather tries to give more allowance for a runner to begin before reporting failure.

git diff 
diff --git a/qa/qa/resource/user_runners.rb b/qa/qa/resource/user_runners.rb
index da1c808bb681..06264d37943b 100644
--- a/qa/qa/resource/user_runners.rb
+++ b/qa/qa/resource/user_runners.rb
@@ -36,7 +36,7 @@ def fabricate!
 
       def fabricate_via_api!
         api_post
-        start_container_and_register
+        # start_container_and_register
       end
 
       def remove_via_api!
$ bundle exec rspec './qa/specs/features/browser_ui/5_package/package_registry/maven/maven_group_level_spec.rb[1:1:1:2:1]'
Edited by John McDonnell

Merge request reports