Skip to content

Stop returning alerts from prometheus alerts endpoint

What does this MR do and why?

Implements #372115 (closed)

Do not return any JSON response from the Prometheus alerts endpoint.

When Prometheus sends an alert it does not expect any returning values. We can safely drop the JSON response.

How to verify locally

Prometheus

  1. Setup [Prometheus Integration]https://docs.gitlab.com/ee/operations/incident_management/integrations.html#http-endpoints) via Settings > Monitor > Alerts
  2. Clone https://gitlab.com/splattael/gitlab-manual-prometheus
  3. Edit gitlab-manual-prometheus/alertmanager.yml:
global: {}
receivers:
- name: gitlab
  webhook_configs:
  - send_resolved: true
    # e.g. url: http://192.168.10.38:3001/manual-prometheus/autodevops-deploy/prometheus/alerts/notify.json
    url: http://localhost:3000/gitlab-org/gitlab-test/prometheus/alerts/notify.json
    http_config:
      # e.g. bearer_token: 2195dc9c136400428d0124805e78a7bb
      bearer_token: c34fc9a8840666ba815429a2af6dab94
route:
  group_interval: 5s
  group_wait: 5s
  receiver: gitlab
  repeat_interval: 5m
  1. gdk stop prometheus (if running) to allow our prometheus to be bind on port 9090
  2. docker-compose up and wait ~10 seconds
  3. See http://localhost:3000/gitlab-org/gitlab-test/-/alert_management/2/details
  4. Via curl
$ curl -v --request POST --data '{"version":"4","receiver":"gitlab","status":"firing","alerts":[{"status":"firing","labels":{"alertname":"title","gitlab":"hook","gitlab_alert_id":""},"annotations":{},"startsAt":"2018-09-24T08:57:31.095725221Z","endsAt":"0001-01-01T00:00:00Z","generatorURL":"http://prometheus-prometheus-server-URL"}],"groupLabels":{"alertname":"title"},"commonLabels":{"alertname":"title","gitlab":"hook","gitlab_alert_id":null},"commonAnnotations":{},"externalURL":"","groupKey":"{}:{alertname='title'}"}
' --header "Authorization: Bearer TOKEN" --header "Content-Type: application/json"  http
://localhost:3000/gitlab-org/gitlab-test/prometheus/alerts/notify.json

# => 201 Created

HTTP Endpoint

This endpoints still returns information for created alerts 👍

  1. Setup HTTP Endpoint via Settings > Monitor > Alerts
  2. Fire an alert via curl
$ curl -v --request POST --data '{ "title": "Testing" }' --header "Authorization: Bearer TOKEN" --header "Content-Type: application/json"  http://localhost:3000/gitlab-org/gitlab-test/alerts/notify/test/ID.json
  1. Request succeeds and returns [{"iid":1,"title":"Testing"}]
  2. http://localhost:3000/gitlab-org/gitlab-test/-/alert_management/1/details is available

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Peter Leitzen

Merge request reports

Loading