Add agent config type, resolve, finder with permission check (!158587) · Merge requests · GitLab.org / GitLab

Zhaochen Li requested to merge zl/471197 into master Jul 08, 2024

Issue - Backend: GraphQL type and resolver for agent co... (#471197 - closed)

What does this MR do and why?

This issue is created as the follow up on the thread here, #427519 (comment 1985555517)

we are going to have a new type for the remote_development_agent_configs under ee/app/graphql/types/remote_development (like ee/app/graphql/types/remote_development/workspace_type.rb), and then set up the correct connections/resolvers to make it work and have its fields be retrievable in a single query like you have above.

And we would like to do the similar thing for remote_development_namespace_cluster_agent_mappings as well

MR acceptance checklist

new type for agent_config table
new resolver for agent_config by agent
new finder for fetching agent_config
proper policy defined at each level
sufficient unit testing for each new class
Update ee/spec/requests/remote_development/integration_spec.rb to exercise and assert on the newly-added fields.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before this MR, the type remoteDevelopmentAgentConfig and its fields are not available to the queries.

Before	After

Also is a prove that we do not run into N+1 issue with lookahead and batch-loader. The log when I run the above query locally.

  ↳ app/models/concerns/use_sql_function_for_primary_key_lookups.rb:8:in `_query_by_sql'
  Route Load (0.1ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 21 AND "routes"."source_type" = 'Project' LIMIT 1 /*application:web,correlation_id:01J347XA8XY86Y0QXRVZJGKH88,endpoint_id:GraphqlController#execute,db_config_name:main,line:/app/models/concerns/routable.rb:155:in `block in full_attribute'*/
  ↳ app/models/concerns/routable.rb:155:in `block in full_attribute'
  RemoteDevelopment::RemoteDevelopmentAgentConfig Load (0.1ms)  SELECT "remote_development_agent_configs".* FROM "remote_development_agent_configs" WHERE "remote_development_agent_configs"."cluster_agent_id" IN (5, 4) ORDER BY "remote_development_agent_configs"."id" DESC /*application:web,correlation_id:01J347XA8XY86Y0QXRVZJGKH88,endpoint_id:GraphqlController#execute,db_config_name:main,line:/ee/app/graphql/resolvers/remote_development/agent_config_for_agent_resolver.rb:22:in `block in resolve_with_lookahead'*/
  ↳ ee/app/graphql/resolvers/remote_development/agent_config_for_agent_resolver.rb:22:in `block in resolve_with_lookahead'
  Clusters::Agent Load (0.1ms)  SELECT "cluster_agents".* FROM "cluster_agents" WHERE "cluster_agents"."id" = 5 LIMIT 1 /*application:web,correlation_id:01J347XA8XY86Y0QXRVZJGKH88,endpoint_id:GraphqlController#execute,db_config_name:main,line:/ee/app/policies/remote_development/remote_development_agent_config_policy.rb:6:in `block in <class:RemoteDevelopmentAgentConfigPolicy>'*/
  ↳ ee/app/policies/remote_development/remote_development_agent_config_policy.rb:6:in `block in <class:RemoteDevelopmentAgentConfigPolicy>'
  Route Load (0.1ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 19 AND "routes"."source_type" = 'Project' LIMIT 1 /*application:web,correlation_id:01J347XA8XY86Y0QXRVZJGKH88,endpoint_id:GraphqlController#execute,db_config_name:main,line:/app/models/concerns/routable.rb:155:in `block in full_attribute'*/
  ↳ app/models/concerns/routable.rb:155:in `block in full_attribute'
  Clusters::Agent Load (0.1ms)  SELECT "cluster_agents".* FROM "cluster_agents" WHERE "cluster_agents"."id" = 4 LIMIT 1 /*application:web,correlation_id:01J347XA8XY86Y0QXRVZJGKH88,endpoint_id:GraphqlController#execute,db_config_name:main,line:/ee/app/policies/remote_development/remote_development_agent_config_policy.rb:6:in `block in <class:RemoteDevelopmentAgentConfigPolicy>'*/
  ↳ ee/app/policies/remote_development/remote_development_agent_config_policy.rb:6:in `block in <class:RemoteDevelopmentAgentConfigPolicy>'

How to set up and validate locally

checkout this MR & start gdk
go to GraphQL Explorer, http://gdk.test:3000/-/graphql-explorer, and copy below query

query getRemoteDevelopmentClusterAgents($namespace: ID!) {
  namespace(fullPath: $namespace) {
    id
    remoteDevelopmentClusterAgents(filter: AVAILABLE) {
      nodes {
        id
        name
        project {
          id
          nameWithNamespace
        }
        remoteDevelopmentAgentConfig{
          id
          defaultMaxHoursBeforeTermination
        }
      }
    }
  }
}

also add in variables as needed

{
  "namespace": "gitlab-org"
}

Then if we have configured agent with remote development, then we should be able to see agent returned with proper config. Follow this doc for configuring local remote development agent.
We would also need to map agent with project with the new authorization strategy. This could be done, under the group page, settings/remote_development tab. Or go to DB and make a new DB record between the group and cluster agent directly.
If you would like to test for N+1, then pls setup at least 2 agents under the same group. Then check on development log after running the query.

Database Info

There is no code change in workspaces_finder and base_finder. The main reason for touching these 2 classes is, we would like to re-use some argument validation methods, so I lifted them into the parent class. There should be no SQL involved.

Rails Console Commands to obtain finder query with all arguments

[1] pry(main)> u = User.find 1
  User Load (6.3ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1 /*application:console,db_config_name:main,console_hostname:Zhaochens-MacBook-Pro.local,console_username:zhaochen.li,line:/app/models/concerns/use_sql_function_for_primary_key_lookups.rb:8:in `_query_by_sql'*/
=> #<User id:1 @root>
[2] pry(main)> RemoteDevelopment::AgentConfigsFinder.execute(current_user: u, ids: [4, -1], cluster_agent_ids: [4, -1])
  License Load (0.5ms)  SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 100 /*application:console,db_config_name:main,console_hostname:Zhaochens-MacBook-Pro.local,console_username:zhaochen.li,line:/ee/app/models/license.rb:94:in `filter_map'*/
  RemoteDevelopment::RemoteDevelopmentAgentConfig Load (0.9ms)  SELECT "remote_development_agent_configs".* FROM "remote_development_agent_configs" WHERE "remote_development_agent_configs"."id" IN (4, -1) AND "remote_development_agent_configs"."cluster_agent_id" IN (4, -1) ORDER BY "remote_development_agent_configs"."id" DESC /*application:console,db_config_name:main,console_hostname:Zhaochens-MacBook-Pro.local,console_username:zhaochen.li,line:bin/rails:4:in `<main>'*/

Raw SQL query 1

SELECT "remote_development_agent_configs".* FROM "remote_development_agent_configs" WHERE "remote_development_agent_configs"."id" IN (1000261, 1000262) AND "remote_development_agent_configs"."cluster_agent_id" IN (1105642, 1105688) ORDER BY "remote_development_agent_configs"."id" DESC

Query Plan 1

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/30626/commands/95054

Raw SQL query 2

SELECT *
FROM cluster_agents
INNER JOIN remote_development_agent_configs
ON remote_development_agent_configs.cluster_agent_id = cluster_agents.id
WHERE cluster_agents.id IN (1105688, 1105642)
AND remote_development_agent_configs.enabled = TRUE
ORDER BY cluster_agents.name ASC, cluster_agents.id DESC
LIMIT 101;

Query Plan 2

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/30626/commands/95055

Edited Aug 07, 2024 by Zhaochen Li

Add agent config type, resolve, finder with permission check