Improved Web Tests

Closes #142 (closed) #175 (closed)

Related #73 (closed)

MR updates Web Tests to be more representative by calling more of the relevant endpoints a user's browser would for a specific page. Goal of this is to increase most popular Web Controllers coverage while mimicking a user (i.e. load a web page's main URLs together rather targeting controllers and their actions directly). Dynamic or authenticated URLs aren't covered by this MR. In addition, rendering is specifically not covered under this tool (as k6 isn't a browser) and falls under SiteSpeed.

Task required solving several notable problems, specifically how best to handle multiple URL calls in one test to be checked correctly and have enough information to debug each.

MR update notes:

• All Web Tests have been updated to new format based on goals above.
• Test output has been improved to have more useful information utilising more percentiles and k6 tags to show more details about each URL called in tests where there's more than one.
• k6 is updated to 0.26.0 as this is compatible and improves batch() functionality for the new tests
• Other small improvements such as updating the Bundler version and syntax to follow new deprecations.

Most popular Web Controller endpoints for the last 24 hours are available to see on our Kibana server. MR aims to investigate and cover all of following based on the criteria above and for areas we have data in (more tests will be added along with API if data is still to be generated, e.g. CI Jobs).

Current progress list:

• Projects::GitHttpController (1,671,118)
• Projects::MergeRequests::ContentController (901,124)
• Projects::PipelinesController (331,956)
• Projects::CommitsController (300,211)
• Projects::MergeRequestsController (280,959)
• Projects::CommitController (172,599)
• ProjectsController (144,375)
• Projects::BlobController (96,827)
• Projects::MergeRequests::DiffsController (78,064)
• Projects::BranchesController (28,570)
• Projects::TreeController (26,889)
• GroupsController (26,235)
• UsersController (24,740)

Example output of the new web test format with more details on multiple urls:

    █ Web - User

    data_received..................: 11 MB   186 kB/s
    data_sent......................: 120 kB  2.0 kB/s
    group_duration.................: avg=1884.46ms min=200.88ms med=1998.48ms max=2107.73ms p(50)=1998.48ms p(75)=2003.40ms p(90)=2009.62ms p(95)=2025.28ms count=577
    http_req_blocked...............: avg=3.93ms    min=0.00ms   med=0.01ms    max=114.81ms  p(50)=0.01ms    p(75)=0.01ms    p(90)=0.01ms    p(95)=0.02ms    count=1154
    http_req_connecting............: avg=3.93ms    min=0.00ms   med=0.00ms    max=114.71ms  p(50)=0.00ms    p(75)=0.00ms    p(90)=0.00ms    p(95)=0.00ms    count=1154
    http_req_duration..............: avg=220.49ms  min=143.62ms med=216.41ms  max=391.58ms  p(50)=216.41ms  p(75)=284.58ms  p(90)=290.70ms  p(95)=307.68ms  count=1154
    http_req_receiving.............: avg=56.05ms   min=0.03ms   med=1.30ms    max=153.65ms  p(50)=1.30ms    p(75)=112.59ms  p(90)=113.37ms  p(95)=113.73ms  count=1154
    http_req_sending...............: avg=0.02ms    min=0.01ms   med=0.02ms    max=0.15ms    p(50)=0.02ms    p(75)=0.02ms    p(90)=0.03ms    p(95)=0.04ms    count=1154
    http_req_tls_handshaking.......: avg=0.00ms    min=0.00ms   med=0.00ms    max=0.00ms    p(50)=0.00ms    p(75)=0.00ms    p(90)=0.00ms    p(95)=0.00ms    count=1154
    http_req_waiting...............: avg=164.42ms  min=143.55ms med=166.14ms  max=279.16ms  p(50)=166.14ms  p(75)=172.46ms  p(90)=181.29ms  p(95)=200.37ms  count=1154
    ✓ { endpoint:calender.json }...: avg=151.93ms  min=143.55ms med=149.15ms  max=255.43ms  p(50)=149.15ms  p(75)=150.85ms  p(90)=159.21ms  p(95)=166.94ms  count=577
    ✓ { endpoint:root }............: avg=176.91ms  min=163.88ms med=171.91ms  max=279.16ms  p(50)=171.91ms  p(75)=175.46ms  p(90)=194.36ms  p(95)=210.56ms  count=577
  ✓ http_reqs......................: 1154    19.233265/s
    ✓ { endpoint:calender.json }...: 577     9.616633/s
    ✓ { endpoint:root }............: 577     9.616633/s
    iteration_duration.............: avg=1881.22ms min=0.14ms   med=1998.47ms max=2107.74ms p(50)=1998.47ms p(75)=2003.39ms p(90)=2009.63ms p(95)=2025.28ms count=578
    iterations.....................: 577     9.616633/s
  ✓ successful_requests............: 100.00% ✓ 1154 ✗ 0
    vus............................: 1       min=1  max=20
    vus_max........................: 20      min=20 max=20

All k6 tests have finished after 61.3s!

Results summary:

* Environment:    10k
* Version:        12.7.0-pre `60ddd8fb177`
* Option:         60s_200rps
* Date:           2020-01-29
* Run Time:       1m 1.3s (Start: 15:04:53 UTC, End: 15:05:54 UTC)

NAME     | RPS  | RPS RESULT         | REQ AVG  | REQ P95  | REQ STATUS     | RESULT
---------|------|--------------------|----------|----------|----------------|----------------
web_user | 20/s | 19.23/s (>16.00/s) | 164.42ms | 200.37ms | 100.00% (>95%) | Passed

Results files:
k6/results/10k_v12-7-0-pre_2020-01-29_150453/10k_v12-7-0-pre_2020-01-29_150453_results_output.log
k6/results/10k_v12-7-0-pre_2020-01-29_150453/10k_v12-7-0-pre_2020-01-29_150453_results.json