Skip to content

Support handling test thresholds for older GitLab versions

Nailia Iskhakova (OOO) requested to merge 251-thresholds-for-older-versions into master

The MR adds support to handle test thresholds for older GitLab versions. Using the environment version in env variable and calculating threshold in gpt_k6_modules.

Walked through the closed and opened performance issues to populate the historical data. Additionally, readjusted the current thresholds to tighten them up.

Thresholds new structure:

{
  'rps': { 'gitlabVer1': 0.2, 'gitlabVer2': 0.7, 'latest': 0.8 },
  'ttfb': { 'gitlabVer1': 5000, 'gitlabVer2': 2000, 'latest': 1200 },
}

Where 'gitlabVer' is the version where the test performance has changed due to application code changes. 'gitlabVer' should always be ordered by the version increase (ASC). Went with a boring solution for now not to check that env versions that are passed have a incorrect order as we're populating it manually. Could be improved in a follow up. 'latest' threshoold is being used to store the current threshold adjustment. When it is improved, it can be easily updated to the version when the fix MR was fixed and then GPT will use default thresholds.

New format supports backward compatability - getRpsThresholds() and getRpsThresholds(0.8) will still work as before.

Closes #251 (closed)

Edited by Nailia Iskhakova (OOO)

Merge request reports