Skip to content

Add Results Score

Grant Young requested to merge gy-results-score into master

Closes #55 (closed)

Adds in an overall percentage score to signify at a high level how well an environment has performed against expected targets. This is calculated on RPS, namely what RPS was achieved versus the target and then distilled into a percentage.

Known issue endpoint tests are excluded from the calculation in this version for now as the aim here is to show how well a specific environment setup is performing via good endpoints (i.e. if a good endpoint is found to be notable slower than reference then that environment will have a reduced score).

MR also updates docs and several other small improvements. Will lead to a v1.1.1 release of GPT.

Good

* Environment:                Docker
* Environment Version:        12.7.0 `680a5284470`
* Option:                     20s_20rps
* Date:                       2020-02-05
* Run Time:                   9m 11.71s (Start: 17:29:09 UTC, End: 17:38:21 UTC)
* GPT Version:                v1.1.1

█ Overall Results Score: 94.76%

NAME                                                     | RPS  | RPS RESULT         | TTFB AVG  | TTFB P90             | REQ STATUS     | RESULT
---------------------------------------------------------|------|--------------------|-----------|----------------------|----------------|------------------
api_v4_groups_group                                      | 20/s | 19.0/s (>16.00/s)  | 51.45ms   | 57.76ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_groups_projects                                   | 20/s | 19.05/s (>16.00/s) | 51.27ms   | 55.59ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_deploy_keys                              | 20/s | 19.1/s (>16.00/s)  | 21.29ms   | 23.70ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_languages                                | 20/s | 19.1/s (>16.00/s)  | 21.21ms   | 23.17ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_merge_requests                           | 20/s | 18.7/s (>12.80/s)  | 141.49ms  | 185.70ms (<1500ms)   | 100.00% (>95%) | Passed¹
api_v4_projects_merge_requests_merge_request             | 20/s | 19.1/s (>16.00/s)  | 51.40ms   | 55.32ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_merge_requests_merge_request_changes     | 20/s | 19.2/s (>16.00/s)  | 41.86ms   | 45.17ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_merge_requests_merge_request_commits     | 20/s | 19.1/s (>16.00/s)  | 54.40ms   | 66.40ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_merge_requests_merge_request_discussions | 20/s | 5.95/s (>8.00/s)   | 2341.61ms | 2880.16ms (<4000ms)  | 100.00% (>95%) | FAILED¹²
api_v4_projects_pagination_keyset                        | 20/s | 19.0/s (>16.00/s)  | 62.87ms   | 70.27ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_pagination_offset                        | 20/s | 19.0/s (>16.00/s)  | 63.54ms   | 71.43ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_project                                  | 20/s | 19.05/s (>16.00/s) | 57.96ms   | 65.78ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_project_pipelines                        | 20/s | 19.15/s (>16.00/s) | 33.01ms   | 35.19ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_project_search_blobs                     | 20/s | 18.85/s (>16.00/s) | 102.80ms  | 115.86ms (<500ms)    | 100.00% (>95%) | Passed
api_v4_projects_project_services                         | 20/s | 19.1/s (>16.00/s)  | 20.04ms   | 21.99ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_repository_branches                      | 20/s | 1.6/s (>0.80/s)    | 6445.03ms | 9275.59ms (<15000ms) | 100.00% (>95%) | Passed¹
api_v4_projects_repository_branches_branch               | 20/s | 16.05/s (>16.00/s) | 172.55ms  | 270.00ms (<500ms)    | 100.00% (>95%) | Passed
api_v4_projects_repository_commits                       | 20/s | 19.1/s (>16.00/s)  | 51.84ms   | 56.14ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_repository_commits_sha                   | 20/s | 19.1/s (>16.00/s)  | 45.07ms   | 48.81ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_repository_commits_sha_diff              | 20/s | 19.1/s (>16.00/s)  | 42.56ms   | 46.29ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_repository_commits_sha_signature         | 20/s | 19.15/s (>16.00/s) | 37.91ms   | 40.56ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_repository_files_file                    | 20/s | 19.15/s (>16.00/s) | 42.33ms   | 45.33ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_repository_files_file_raw                | 20/s | 19.1/s (>16.00/s)  | 46.73ms   | 49.23ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_projects_repository_tree                          | 20/s | 19.15/s (>16.00/s) | 39.21ms   | 60.19ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_user                                              | 20/s | 19.1/s (>16.00/s)  | 19.10ms   | 22.46ms (<500ms)     | 100.00% (>95%) | Passed
api_v4_users                                             | 20/s | 19.1/s (>16.00/s)  | 22.45ms   | 24.86ms (<500ms)     | 100.00% (>95%) | Passed

Bad

* Environment:                Docker
* Environment Version:        12.7.0 `680a5284470`
* Option:                     60s_100rps
* Date:                       2020-02-05
* Run Time:                   2m 4.84s (Start: 17:39:31 UTC, End: 17:41:36 UTC)
* GPT Version:                v1.1.0

█ Overall Results Score: 58.58%

NAME                   | RPS   | RPS RESULT         | TTFB AVG  | TTFB P90           | REQ STATUS     | RESULT
-----------------------|-------|--------------------|-----------|--------------------|----------------|-----------------
api_v4_groups_group    | 100/s | 59.62/s (>80.00/s) | 1515.92ms | 1668.36ms (<500ms) | 100.00% (>95%) | FAILED²
api_v4_groups_projects | 100/s | 57.55/s (>80.00/s) | 1571.39ms | 1748.19ms (<500ms) | 100.00% (>95%) | FAILED²
Edited by Grant Young

Merge request reports

Loading