E2E recovery alert resolves correct alert && incident

What does this MR do and why?

Describe in detail what your merge request does and why.

Recovery alert resolves correct alert

1. Login to gitlab
2. Create project via API -> visit project

With Http enpoint integration

3. Go to Settings -> Monitor -> setup HTTP endpoint integration
4. Send 2 different test alerts by sending in different JSON payloads (payload_to_be_resolved, unresolved_payload)
5. Append end_time field to payload_to_be_resolved
6. Send payload_to_be_resolved again
7. Verify
   • payload_to_be_resolved does not appear in Open tab, it is in Resolved tab instead
   • unresolved_payload is in Open tab, not in Resolved tab

With Prometheus integration

3. Go to Settings -> Monitor -> setup Prometheus integration
4. Send 2 different test alerts by sending in different JSON payloads (payload_to_be_resolved, unresolved_payload)
5. Append end_time field to payload_to_be_resolved as well as update status to resolved
6. Send payload_to_be_resolved again
7. Verify
   • Alert with resolve_title does not appear in Open tab, it is in Resolved tab instead
   • Alert with unresolve_title is in Open tab, not in Resolved tab

Recovery alert closes correct incident

1. Login to gitlab
2. Create project via API -> visit project

With Http enpoint integration

3. Go to Settings -> Monitor -> setup HTTP endpoint integration and enable to auto create incident
4. Send 2 different test alerts by sending in different JSON payloads (payload_to_be_resolved, unresolved_payload)
5. Append end_time field to payload_to_be_resolved
6. Send payload_to_be_resolved again
7. Go to Monitor -> Incidents
8. Verify
   • Incident with resolve_title does not appear in Open tab, it is in Closed tab instead
   • Incident with unresolve_title is in Open tab, not in Closed tab

With Prometheus integration

3. Go to Settings -> Monitor -> setup Prometheus integration and enable to auto create incident
4. Send 2 different test alerts by sending in different JSON payloads (payload_to_be_resolved, unresolved_payload)
5. Append end_time field to payload_to_be_resolved as well as update status to resolved
6. Send payload_to_be_resolved again
7. Go to Monitor -> Incidents
8. Verify
   • Incident with resolve_title does not appear in Open tab, it is in Closed tab instead
   • Incident with unresolve_title is in Open tab, not in Closed tab

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.