Skip to content

New result status for close "failures" with the TTFB warning threshold

Nailia Iskhakova requested to merge close-failures-result-status into master

Closes #261 (closed)

This MR adds a new result status for close "failures" using new TTFB Warning Threshold. If test failed within its threshold + warning threshold, we mark it as PASSED with the caveat that test result is above the threshold by a small amount but within the margin of error.

The warning threshold is 20% of the normal threshold up to a maximum of 200ms.

Example (TTFB 500ms threshold was changed for testing purposes):

█ Results summary

* Environment:                Localhost
* Environment Version:        13.0.0-ee `ef3e35341f6`
* Option:                     30s_2rps
* Date:                       2020-06-05
* Run Time:                   31.06s (Start: 17:33:59 UTC, End: 17:34:30 UTC)
* GPT Version:                v2.0.0

❯ Overall Results Score: 100.0%

NAME        | RPS | RPS RESULT      | TTFB AVG | TTFB P90          | REQ STATUS     | RESULT          
------------|-----|-----------------|----------|-------------------|----------------|-----------------
web_project | 1/s | 1.0/s (>0.80/s) | 169.42ms | 236.55ms (<220ms) | 100.00% (>95%) | Passed³

³ Test has passed but with a warning as it posted a TTFB result just over the target threshold. Rerun the test to confirm if actual issue.
Edited by Nailia Iskhakova

Merge request reports