Draft: Use `InOperatorOptimization::QueryBuilder` for group audit events

What does this MR do and why?

This feature flag has never been turned on since it was added. This is because the query, as writen, times out almost 100% of the time.

We would like to iterate on this feature, attempting to use the Gitlab::Pagination::Keyset::InOperatorOptimization::QueryBuilder to improve the performance to a usable state

The or portion of this query seems like it is not handled by the query builder.

There is a described way to handle multiple in queries described in the documentation here. However, that appears to be targeted at adding additional filters (an AND type scenario)

I couldn't get it to work in this OR scenario. I think we might be able to get it to work with a value_list of tuples of entity_type, entity_id, but I have not figured that out yet. I will coontinue investiggating that route in future iterations.

For this iteration, I used the query builder to optimize the project audit event portion of the query, manually constructed the group level, and then used a raw SQL UNION query to get the events.

I am hoping the UNION performs better than an OR. I then use a bit of trickery to get back an ActiveRecord::Relation rather than a PGResponse object.

Since this is behind a feature flag and is non-functional in it's current state, I'd like to see how this change performs in staging to see if this is a fruitful path to continue down

How to set up and validate locally

1. have a EE instance setup
2. create an access token to be used in the API calls
3. create some group and project audit events for a group

# you can use factorybot to do this
user = User.find(1) # or whatever user you use locally
project = Project.find(19)
2.times { FactoryBot.create(:project_audit_event, user: user, target_project: project) }
2.times { FactoryBot.create(:group_audit_event, user: user, target_group: project.group) }
2.times { FactoryBot.create(:group_audit_event, user: user, target_group: Group.find(80)) } # should not show up in results

4. query the endpoint

  curl --header "PRIVATE-TOKEN: glpat-ACCESS-TOKEN" \
       -X GET \
       -H 'Content-Type: application/json' \
       -H 'Accept: application/json, text/plain, */*' \
       "localhost:3000/api/v4/groups/70/audit_events?per_page=100&page=1" | json_pp

5. you should see the group audit events only. I see:

[
   {
      "author_id" : 1,
      "created_at" : "2023-01-10T00:20:17.555Z",
      "details" : {
         "author_name" : "Jane Doe",
         "change" : "project_creation_level",
         "entity_path" : "gitlab-instance-03dfacec",
         "from" : "",
         "ip_address" : "127.0.0.1",
         "target_details" : "GitLab Instance",
         "target_id" : 70,
         "target_type" : "Group",
         "to" : "Developers + Maintainers"
      },
      "entity_id" : 70,
      "entity_type" : "Group",
      "id" : 37
   },
   {
      "author_id" : 1,
      "created_at" : "2023-01-10T00:20:17.552Z",
      "details" : {
         "author_name" : "Jane Doe",
         "change" : "project_creation_level",
         "entity_path" : "gitlab-instance-03dfacec",
         "from" : "",
         "ip_address" : "127.0.0.1",
         "target_details" : "GitLab Instance",
         "target_id" : 70,
         "target_type" : "Group",
         "to" : "Developers + Maintainers"
      },
      "entity_id" : 70,
      "entity_type" : "Group",
      "id" : 36
   }
]

6. enable the feature flag

Feature.enable(:audit_log_group_level)

7. repeat the query. you should now see group and project audit events. i see:

[
   {
      "author_id" : 1,
      "created_at" : "2023-01-10T00:20:17.555Z",
      "details" : {
         "author_name" : "Jane Doe",
         "change" : "project_creation_level",
         "entity_path" : "gitlab-instance-03dfacec",
         "from" : "",
         "ip_address" : "127.0.0.1",
         "target_details" : "GitLab Instance",
         "target_id" : 70,
         "target_type" : "Group",
         "to" : "Developers + Maintainers"
      },
      "entity_id" : 70,
      "entity_type" : "Group",
      "id" : 37
   },
   {
      "author_id" : 1,
      "created_at" : "2023-01-10T00:20:17.552Z",
      "details" : {
         "author_name" : "Jane Doe",
         "change" : "project_creation_level",
         "entity_path" : "gitlab-instance-03dfacec",
         "from" : "",
         "ip_address" : "127.0.0.1",
         "target_details" : "GitLab Instance",
         "target_id" : 70,
         "target_type" : "Group",
         "to" : "Developers + Maintainers"
      },
      "entity_id" : 70,
      "entity_type" : "Group",
      "id" : 36
   },
   {
      "author_id" : 1,
      "created_at" : "2023-01-10T00:20:17.473Z",
      "details" : {
         "author_name" : "Jane Doe",
         "change" : "packages_enabled",
         "entity_path" : "gitlab-instance-03dfacec/Monitoring",
         "from" : "true",
         "ip_address" : "127.0.0.1",
         "target_details" : "Monitoringg",
         "target_id" : 19,
         "target_type" : "Project",
         "to" : "false"
      },
      "entity_id" : 19,
      "entity_type" : "Project",
      "id" : 35
   },
   {
      "author_id" : 1,
      "created_at" : "2023-01-10T00:20:17.444Z",
      "details" : {
         "author_name" : "Jane Doe",
         "change" : "packages_enabled",
         "entity_path" : "gitlab-instance-03dfacec/Monitoring",
         "from" : "true",
         "ip_address" : "127.0.0.1",
         "target_details" : "Monitoringg",
         "target_id" : 19,
         "target_type" : "Project",
         "to" : "false"
      },
      "entity_id" : 19,
      "entity_type" : "Project",
      "id" : 34
   }
]

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.