Skip to content

E2E: Add Kibana Dashboard URL to Job Failure Logs

Valerie Burton requested to merge e2e-add-kibana-dashboard-url into master

What does this MR do and why?

In our E2E tests today, when an error occurs in a request that is associated with a correlation ID, we print out the corresponding links to the Kibana and Sentry logs in our E2E test output logs.

For Kibana, this link only redirects to a direct search in the Discover app which only searches the Rails component and includes all fields by default. This can make it difficult to sift through all the given information, and we could miss errors that could also be occurring in other components.

We now have 3 new Kibana dashboards that combine search results across several GitLab components by correlation ID

Note: Just replace the <CORRELATION_ID> text in the json.correlation_id filter if you are visiting the dashboards above. The generated links in the job failures will automatically contain the correlation ID & time.

These dashboards currently contain panels for Rails, Gitaly, Sidekiq, Workhorse and Postgres (where available), and also only display applicable fields we care about so that information can be gathered quickly.

This MR adds a link to the appropriate dashboard based on the current environment, along with the basic Kibana Discover search link, to help aid in troubleshooting failures. It also refactors the Loglinking module to utilize new Kibana and Sentry classes for better organization and maintainability.

Relates to gitlab-org/quality/quality-engineering/team-tasks#1543 (closed)

Screenshots or screen recordings

Quick dashboard demo:

Screen_Recording_2022-12-13_at_10.56.10_AM

How to set up and validate locally

  1. Go to qa/qa/resource/api_fabricator.rb
  2. Replace api_get_from method body to just raise an error, such as:
def api_get_from(get_path)
  raise InternalServerError, "Loglinking Test: \n#{QA::Support::Loglinking.failure_metadata('abc123')}"
end
  1. Next, go to qa/qa/support/loglinking.rb
  2. Replace env = logging_environment on line 18 with an existing environment, such as env = :staging
  3. Run an E2E spec such as qa/qa/specs/features/browser_ui/1_manage/project/create_project_spec.rb. The first GET request should error out and print out an example of what the new links would look like.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports