Something Went Wrong While Fetching Pipeline Statistics Error Displays on Group Usage Quotas Page After Purchase
Summary
Several automated E2E tests in the GitLab project have been failing intermittently on Staging and Staging-Canary due to a Something went wrong while fetching pipeline statistics
error on the group usage quotas page.
The error is suspected to come from either the getGroupCiMinutesUsage
or getGroupProjectsCiMinutesUsage
GraphQL queries.
#426743 (closed) was created to help us log this error to Sentry, but we have not been able to find the associated error there (see this related thread)
Kibana logs for the various test failures have been added to the "Relevant logs and/or screenshots" section below. It appears this generic error can occur for both a 502 error as well as when the queries return a 200 status. For the 200 status, we suspect there may be a GraphQL error within the response body itself that we are unable to find logs for.
Related QA failure issues:
Steps to reproduce
Unfortunately, this error cannot be reliably reproduced and is only being surfaced intermittently through E2E tests running on Staging and Staging-Canary.
Here is an example from #426178 (closed) of the steps the test takes to reach this error:
- A newly created user creates a new top-level group with a project
- User logs in and visits the pipelines tab on the group usage quotas page and clicks on the "buy compute minutes" button
- User selects 5 compute minutes packs to purchase on the checkout page, and proceeds to fill out their billing and payment information
- User confirms purchase and the purchase goes through successfully
- User re-visits the pipelines tab on the group usage quotas page and encounters the error
- Even after refreshing the page for 60 seconds to wait for the compute minutes to sync back to GitLab, the error still continues to occur
What is the current bug behavior?
A Something went wrong while fetching pipeline statistics
error displays on the pipelines tab of the group usage quotas page after a user purchases compute minutes.
What is the expected correct behavior?
No errors should be present and the user should be able to see their compute minutes synced to GitLab.
Relevant logs and/or screenshots
GraphQL 502 Status
Additional Info:
- Username:
qa-user-f6773864bf9569b7
- Group:
test-group-fulfillmentba8b2340
- Time failure occurred:
2023-10-27 07:32:17 UTC
- Failed job: https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/11669455
- Kibana logs:
-
pubsub-rails-inf-gstg*
(shows 200 status for GraphQL queries, but we can see there is no log forgetGroupProjectsCiMinutesUsage
) -
pubsub-workhorse-inf-gstg*
(correlation ID:5df36e7600a339ecdef2c232f6cbbea8
)-
First log (shows 502 status, possibly from
getGroupProjectsCiMinutesUsage
) -
Second log (shows
badgateway: failed to receive response: dial tcp 127.0.0.1:8080: connect: connection refused
error)
-
First log (shows 502 status, possibly from
-
GraphQL 200 Status
Additional Info:
- Username:
qa-user-ca31857cbbab6b10
- User Id:
3505791
- Email:
test-user-1ea474f8@gitlab.com
- Group:
test-group-fulfillment1ea474f8
- Time failure occurred:
2023-10-23 10:56:11 UTC
- Failed job: https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/11605954
- Kibana logs:
-
pubsub-rails-inf-gstg*
(shows 200 status for GraphQL queries) -
pubsub-workhorse-inf-gstg*
(shows 200 status for various GraphQL requests)
-