Improve duplication detection logic

What does this MR do and why?

Improve how we handle duplication detection. This MR also improves how we handle empty failed jobs during duplication detection.

Closes gitlab-org/quality/engineering-productivity/team#563 (closed)

Expected impact & dry-runs

These are strongly recommended to assist reviewers and reduce the time to merge your change.

See https://gitlab.com/gitlab-org/quality/triage-ops/-/tree/master/doc/scheduled#testing-policies-with-a-dry-run on how to perform dry-runs for new policies.

See https://gitlab.com/gitlab-org/quality/triage-ops/-/blob/master/doc/reactive/best_practices.md#use-the-sandbox-to-test-new-processors on how to make sure a new processor can be tested.

Action items

• If adding environment variables for reactive processors, update config/triage-web.yaml and .gitlab/ci/triage-web.yml
• (If applicable) Add documentation to the handbook pages for Triage Operations =>
• (If applicable) Identify the affected groups and how to communicate to them:
  • /cc @person_or_group =>
  • Relevant Slack channels =>
  • Engineering week-in-review