Fix metrics regex in get_test_results method
Fixing regex for http_req_waiting and other metrics in results parsing to adjust recent update in k6 error output
After 0.43.1 if one of the thresholds failed the error message became expErr := "thresholds on metrics 'iterations{scenario:sc1}, iterations{scenario:sc2}' have been breached" which breaks the existing GPT regex.
Example error in 10k pipeline https://gitlab.com/gitlab-org/quality/performance/-/jobs/3985806541#L5522
Error
http_req_tls_handshaking.........................................................: avg=4.81ms min=0.00ms med=0.00ms max=43.13ms p(90)=15.43ms p(95)=39.48ms
http_req_waiting.................................................................: avg=1096.42ms min=284.60ms med=990.78ms max=2523.43ms p(90)=2268.25ms p(95)=2515.80ms
✓ { controller:Projects::MergeRequests::ContentController#cached_widget.json }...: avg=419.87ms min=410.34ms med=419.87ms max=429.40ms p(90)=427.49ms p(95)=428.44ms
✓ { controller:Projects::MergeRequests::ContentController#widget.json }..........: avg=301.43ms min=284.60ms med=301.43ms max=318.26ms p(90)=314.90ms p(95)=316.58ms
✗ { controller:Projects::MergeRequestsController#discussions.json }..............: avg=1802.75ms min=990.78ms med=1797.70ms max=2523.43ms p(90)=2516.75ms p(95)=2520.09ms
✓ { controller:Projects::MergeRequestsController#show.json }.....................: avg=294.31ms min=287.72ms med=294.31ms max=300.89ms p(90)=299.58ms p(95)=300.23ms
✓ { controller:Projects::MergeRequestsController#show }..........................: avg=560.71ms min=519.72ms med=560.71ms max=601.70ms p(90)=593.50ms p(95)=597.60ms
✓ http_reqs........................................................................: 17 0.693207/s
✓ { controller:Projects::MergeRequests::ContentController#cached_widget.json }...: 2 0.081554/s
✓ { controller:Projects::MergeRequests::ContentController#widget.json }..........: 2 0.081554/s
✓ { controller:Projects::MergeRequestsController#discussions.json }..............: 8 0.326215/s
✓ { controller:Projects::MergeRequestsController#show.json }.....................: 2 0.081554/s
✓ { controller:Projects::MergeRequestsController#show }..........................: 2 0.081554/s
iteration_duration...............................................................: avg=8172.99ms min=1247.63ms med=11445.03ms max=11826.33ms p(90)=11750.07ms p(95)=11788.20ms
iterations.......................................................................: 2 0.081554/s
✓ successful_requests..............................................................: 100.00% ✓ 16 ✗ 0
vus..............................................................................: 1 min=0 max=1
vus_max..........................................................................: 1 min=1 max=1
running (24.5s), 0/1 VUs, 2 complete and 0 interrupted iterations
default ✓ [ 100% ] 0/1 VUs 20s
time="2023-03-23T16:18:46+01:00" level=error msg="thresholds on metrics 'http_req_waiting{controller:Projects::MergeRequestsController#discussions.json}' have been breached"
Test failed and output couldn't be parsed:
undefined method `[]' for nil:NilClass
Validated that with changes in this MR test is reported correctly:
validation
█ setup
█ Web - Project Merge Request Page
data_received....................................................................: 1.8 MB 67 kB/s
data_sent........................................................................: 6.7 kB 252 B/s
group_duration...................................................................: avg=12990.97ms min=12046.64ms med=12990.97ms max=13935.30ms p(90)=13746.43ms p(95)=13840.86ms
http_req_blocked.................................................................: avg=9.92ms min=0.00ms med=0.00ms max=120.19ms p(90)=19.39ms p(95)=62.80ms
http_req_connecting..............................................................: avg=2.29ms min=0.00ms med=0.00ms max=26.13ms p(90)=5.13ms p(95)=15.48ms
http_req_duration................................................................: avg=1283.35ms min=312.37ms med=566.01ms max=3956.22ms p(90)=3149.91ms p(95)=3878.72ms
{ expected_response:true }.....................................................: avg=1283.35ms min=312.37ms med=566.01ms max=3956.22ms p(90)=3149.91ms p(95)=3878.72ms
http_req_failed..................................................................: 0.00% ✓ 0 ✗ 17
http_req_receiving...............................................................: avg=63.04ms min=0.12ms med=32.37ms max=263.15ms p(90)=165.61ms p(95)=229.06ms
http_req_sending.................................................................: avg=0.49ms min=0.05ms med=0.30ms max=1.60ms p(90)=1.24ms p(95)=1.37ms
http_req_tls_handshaking.........................................................: avg=4.63ms min=0.00ms med=0.00ms max=43.58ms p(90)=14.08ms p(95)=36.88ms
http_req_waiting.................................................................: avg=1219.82ms min=310.64ms med=533.38ms max=3827.04ms p(90)=2943.59ms p(95)=3756.53ms
✓ { controller:Projects::MergeRequests::ContentController#cached_widget.json }...: avg=447.96ms min=422.51ms med=447.96ms max=473.41ms p(90)=468.32ms p(95)=470.86ms
✓ { controller:Projects::MergeRequests::ContentController#widget.json }..........: avg=431.49ms min=383.39ms med=431.49ms max=479.60ms p(90)=469.98ms p(95)=474.79ms
✗ { controller:Projects::MergeRequestsController#discussions.json }..............: avg=2082.86ms min=1032.43ms med=1666.30ms max=3827.04ms p(90)=3765.35ms p(95)=3796.19ms
✓ { controller:Projects::MergeRequestsController#show.json }.....................: avg=381.61ms min=310.64ms med=381.61ms max=452.58ms p(90)=438.39ms p(95)=445.48ms
✓ { controller:Projects::MergeRequestsController#show }..........................: avg=525.05ms min=516.73ms med=525.05ms max=533.38ms p(90)=531.71ms p(95)=532.54ms
✓ http_reqs........................................................................: 17 0.637697/s
✗ { controller:Projects::MergeRequests::ContentController#cached_widget.json }...: 2 0.075023/s
✗ { controller:Projects::MergeRequests::ContentController#widget.json }..........: 2 0.075023/s
✓ { controller:Projects::MergeRequestsController#discussions.json }..............: 8 0.300093/s
✗ { controller:Projects::MergeRequestsController#show.json }.....................: 2 0.075023/s
✗ { controller:Projects::MergeRequestsController#show }..........................: 2 0.075023/s
iteration_duration...............................................................: avg=8879.77ms min=653.61ms med=12046.74ms max=13938.97ms p(90)=13560.53ms p(95)=13749.75ms
iterations.......................................................................: 2 0.075023/s
✓ successful_requests..............................................................: 100.00% ✓ 16 ✗ 0
vus..............................................................................: 1 min=1 max=1
vus_max..........................................................................: 1 min=1 max=1
running (26.7s), 0/1 VUs, 2 complete and 0 interrupted iterations
default ✓ [ 100% ] 0/1 VUs 20s
time="2023-03-23T16:47:40+01:00" level=error msg="thresholds on metrics 'http_req_waiting{controller:Projects::MergeRequestsController#discussions.json}, http_reqs{controller:Projects::MergeRequests::ContentController#cached_widget.json}, http_reqs{controller:Projects::MergeRequests::ContentController#widget.json}, http_reqs{controller:Projects::MergeRequestsController#show.json}, http_reqs{controller:Projects::MergeRequestsController#show}' have been breached"
All k6 tests have finished after 29.31s!
█ Results summary
* Environment: Staging
* Environment Version: 15.11.0-pre `3ff3d49df96`
* Option: 20s_2rps
* Date: 2023-03-23
* Run Time: 29.31s (Start: 15:47:11 UTC, End: 15:47:40 UTC)
* GPT Version: v2.12.2
NAME | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
--------------------------|-----|------------------|-----------|---------------------|----------------|------------------
web_project_merge_request | 1/s | 0.64/s (>0.16/s) | 1219.82ms | 2943.59ms (<1800ms) | 100.00% (>99%) | FAILED¹²