Resolve "Improve performance of Search API (Advanced): projects scope" (!32688) · Merge requests · GitLab.org / GitLab

Terri Chu requested to merge 215707-improve-performance-of-search-api-advanced-projects-scope into master May 20, 2020

What does this MR do?

Add with_api_entity_associations scope for project model. Use the new scope as a preload method in the search API when calling search_objects. This scope is setup to preload items needed by the Search API requests which uses the BasicProjectDetails Entity for the projects scope.

Performance Improvements

Before

k6 performance

█ API - Global Search

✓ { endpoint:projects }.........: avg=6776.93ms min=1174.30ms med=7720.07ms max=9190.52ms p(90)=8377.97ms p(95)=8497.59ms

█ API - Group Search

✓ { endpoint:projects }.........: avg=1094.51ms min=37.79ms   med=253.80ms  max=4042.56ms p(90)=3302.85ms p(95)=3515.54ms

█ Results summary

* Environment:                Localhost
* Environment Version:        13.0.0-pre `e437426bc12`
* Option:                     60s_20rps
* Date:                       2020-05-20
* Run Time:                   3m 4.45s (Start: 23:12:32 UTC, End: 23:15:36 UTC)
* GPT Version:                v1.3.0

NAME                   | RPS  | RPS RESULT        | TTFB AVG  | TTFB P90             | REQ STATUS      | RESULT
-----------------------|------|-------------------|-----------|----------------------|-----------------|----------------
api_v4_search_global   | 20/s | 13.48/s (>4.80/s) | 6556.83ms | 8278.60ms (<25000ms) | 100.00% (>9.5%) | Passed
api_v4_search_groups   | 20/s | 19.13/s (>4.80/s) | 1205.41ms | 3419.57ms (<25000ms) | 100.00% (>9.5%) | Passed
api_v4_search_projects | 20/s | 19.27/s (>4.80/s) | 285.65ms  | 700.86ms (<25000ms)  | 100.00% (>9.5%) | Passed

█ Results files

k6/results/localhost_v13-0-0-pre_2020-05-20_191232/localhost_v13-0-0-pre_2020-05-20_191232_results_output.log
k6/results/localhost_v13-0-0-pre_2020-05-20_191232/localhost_v13-0-0-pre_2020-05-20_191232_results.json
k6/results/localhost_v13-0-0-pre_2020-05-20_191232/localhost_v13-0-0-pre_2020-05-20_191232_results.txt

GET /api/v4/search?scope=projects&search=*

121 Queries

Started GET "/api/v4/search?scope=projects&search=*" for 127.0.0.1 at 2020-05-20 13:42:01 -0400
Creating scope :of_projects. Overwriting existing method MergeRequest.of_projects.
Creating scope :join_project. Overwriting existing method MergeRequest.join_project.
Creating scope :references_project. Overwriting existing method MergeRequest.references_project.
Creating scope :system. Overwriting existing method Note.system.
Creating scope :group_view_details. Overwriting existing method User.group_view_details.
Creating scope :merged. Overwriting existing method Event.merged.
Creating scope :without_statuses. Overwriting existing method CommitStatus.without_statuses.
Creating scope :opened. Overwriting existing method Epic.opened.
Creating scope :closed. Overwriting existing method Epic.closed.
An enum element in Ci::Runner uses the prefix 'not_'. This will cause a conflict with auto generated negative scopes.
Creating scope :with_files_stored_remotely. Overwriting existing method Ci::JobArtifact.with_files_stored_remotely.
Creating scope :order_created_desc. Overwriting existing method Packages::Package.order_created_desc.
Creating scope :order_name_desc. Overwriting existing method Packages::Package.order_name_desc.
   (0.3ms)  SELECT EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))::float as lag
  ↳ lib/gitlab/middleware/basic_health_check.rb:25:in `call'
   (2.3ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
  ↳ lib/gitlab/middleware/basic_health_check.rb:25:in `call'
  ApplicationSetting Load (2.4ms)  SELECT "application_settings".* FROM "application_settings" ORDER BY "application_settings"."id" DESC LIMIT 1
  ↳ app/models/concerns/cacheable_attributes.rb:19:in `current_without_cache'
  PersonalAccessToken Load (3.4ms)  SELECT "personal_access_tokens".* FROM "personal_access_tokens" WHERE "personal_access_tokens"."token_digest" = 'R27ZoLo1xAaezTWG/7z4uMnF9M8yUPN1lDKLEh0cQq4=' LIMIT 1
  ↳ app/models/concerns/token_authenticatable_strategies/digest.rb:8:in `find_token_authenticatable'
  User Load (7.3ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
  ↳ lib/gitlab/auth/auth_finders.rb:102:in `find_user_from_access_token'
  Feature::FlipperGate Load (0.4ms)  SELECT "feature_gates".* FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'user_mode_in_session'
  ↳ lib/feature.rb:67:in `enabled?'
   (0.4ms)  SELECT "features"."key" FROM "features"
  ↳ lib/feature.rb:15:in `feature_names'
  License Load (0.4ms)  SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 100
  ↳ ee/app/models/license.rb:265:in `load_license'
  Project Search (117.4ms) {index: "gitlab-development", type: "doc", body: {query: {bool: {must: [{simple_query_string: {fields: ["name^10", "name_with_namespace^2", "path_with_namespace", "path^9", "description"], query: "*", default_operator: :and}}], filter: [{terms: {type: ["project"]}}, {bool: {should: [{term: {visibility_level: 0}}, {term: {visibility_level: 10}}, {term: {visibility_level: 20}}]}}]}}, sort: [:_score], highlight: {fields: {name: {}, name_with_namespace: {}, path_with_namespace: {}, path: {}, description: {}}}}, size: 20, from: 0}
   (3.4ms)  SELECT COUNT(*) FROM "projects" WHERE "projects"."id" IN (16, 3, 9, 17, 60, 70, 75, 8, 6, 4, 76, 83, 84, 10, 2, 14, 15, 61, 62, 68)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Project Load (1.3ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" IN (16, 3, 9, 17, 60, 70, 75, 8, 6, 4, 76, 83, 84, 10, 2, 14, 15, 61, 62, 68)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Route Load (2.4ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."source_id" IN (2, 60, 4, 62, 84, 3, 6, 61, 8, 9, 83, 10, 16, 14, 15, 68, 75, 17, 70, 76)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Namespace Load (3.8ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" IN (22, 53, 24, 57, 23, 26, 28, 36, 1, 45, 16, 6, 35, 29)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  ComplianceManagement::ComplianceFramework::ProjectSettings Load (3.0ms)  SELECT "project_compliance_framework_settings".* FROM "project_compliance_framework_settings" WHERE "project_compliance_framework_settings"."project_id" IN (2, 60, 4, 62, 84, 3, 6, 61, 8, 9, 83, 10, 16, 14, 15, 68, 75, 17, 70, 76)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 1 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 53 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (2.8ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 53
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 53 AND "namespaces"."parent_id" IS NULL LIMIT 1
  ↳ app/models/namespace.rb:285:in `block in root_ancestor'
  SamlProvider Load (1.7ms)  SELECT "saml_providers".* FROM "saml_providers" WHERE "saml_providers"."group_id" = 53 LIMIT 1
  ↳ ee/lib/gitlab/auth/group_saml/sso_enforcer.rb:35:in `group_access_restricted?'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 24 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 24
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 53 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.4ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 53
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.6ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 53 AND "namespaces"."parent_id" IS NULL LIMIT 1
  ↳ app/models/namespace.rb:285:in `block in root_ancestor'
  SamlProvider Load (0.3ms)  SELECT "saml_providers".* FROM "saml_providers" WHERE "saml_providers"."group_id" = 53 LIMIT 1
  ↳ ee/lib/gitlab/auth/group_saml/sso_enforcer.rb:35:in `group_access_restricted?'
  Group Load (0.4ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 57 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 57
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.4ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 23 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 23
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 23 AND "namespaces"."parent_id" IS NULL LIMIT 1
  ↳ app/models/namespace.rb:285:in `block in root_ancestor'
  SamlProvider Load (0.2ms)  SELECT "saml_providers".* FROM "saml_providers" WHERE "saml_providers"."group_id" = 23 LIMIT 1
  ↳ ee/lib/gitlab/auth/group_saml/sso_enforcer.rb:35:in `group_access_restricted?'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 26 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 26
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.4ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 53 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 53
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 53 AND "namespaces"."parent_id" IS NULL LIMIT 1
  ↳ app/models/namespace.rb:285:in `block in root_ancestor'
  SamlProvider Load (0.3ms)  SELECT "saml_providers".* FROM "saml_providers" WHERE "saml_providers"."group_id" = 53 LIMIT 1
  ↳ ee/lib/gitlab/auth/group_saml/sso_enforcer.rb:35:in `group_access_restricted?'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 28 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 28
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.4ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 36 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 1 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 45 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 16 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 6 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 35 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 1 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 1 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 29 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.6ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 1 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 22 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 22
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  GeoNode Exists? (0.6ms)  SELECT 1 AS one FROM "geo_nodes" LIMIT 1
  ↳ ee/lib/gitlab/geo.rb:36:in `block in enabled?'
  ActsAsTaggableOn::Tag Load (4.7ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 76 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 1 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
  Feature::FlipperGate Load (0.3ms)  SELECT "feature_gates".* FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'gitaly_enforce_requests_limits'
  ↳ lib/feature.rb:67:in `enabled?'
   (3.9ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 76
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  User Load (0.6ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
  ↳ app/models/namespace.rb:60:in `avatar_url'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 60 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 53 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 60
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ActsAsTaggableOn::Tag Load (0.5ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 4 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.4ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 24 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 4
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 62 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 62
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ActsAsTaggableOn::Tag Load (0.5ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 84 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 57 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 84
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 3 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 23 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 3
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ActsAsTaggableOn::Tag Load (0.5ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 6 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 26 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 6
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 61 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 61
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 8 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 28 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 8
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ProjectFeature Load (0.6ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 9 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  ActsAsTaggableOn::Tag Load (0.5ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 9 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 36 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.4ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 9
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  User Load (0.6ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 29 LIMIT 1
  ↳ app/models/namespace.rb:60:in `avatar_url'
  ProjectFeature Load (0.3ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 83 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 83 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 83
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ProjectFeature Load (0.4ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 10 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 10 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 45 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 10
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  User Load (0.4ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 38 LIMIT 1
  ↳ app/models/namespace.rb:60:in `avatar_url'
  ProjectFeature Load (0.2ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 16 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 16 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 16 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 16
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  User Load (0.5ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 16 LIMIT 1
  ↳ app/models/namespace.rb:60:in `avatar_url'
  ProjectFeature Load (0.2ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 14 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 14 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 6 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 14
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  User Load (0.4ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 6 LIMIT 1
  ↳ app/models/namespace.rb:60:in `avatar_url'
  ProjectFeature Load (0.2ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 15 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 15 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 35 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 15
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  User Load (0.6ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 28 LIMIT 1
  ↳ app/models/namespace.rb:60:in `avatar_url'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 68 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
   (0.2ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 68
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ActsAsTaggableOn::Tag Load (0.3ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 75 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 75
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ProjectFeature Load (0.3ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 17 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 17 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 29 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 17
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  User Load (0.6ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 22 LIMIT 1
  ↳ app/models/namespace.rb:60:in `avatar_url'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 70 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 70
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  ProjectFeature Load (0.3ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 2 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 2 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 22 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 2
  ↳ app/services/base_count_service.rb:26:in `uncached_count'

GET /api/v4/search?scope=projects&search=*

26 Queries

Started GET "/api/v4/groups/54/search?scope=projects&search=*" for 127.0.0.1 at 2020-05-20 14:45:17 -0400
   (0.9ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
  ↳ lib/gitlab/middleware/basic_health_check.rb:25:in `call'
  PersonalAccessToken Load (5.1ms)  SELECT "personal_access_tokens".* FROM "personal_access_tokens" WHERE "personal_access_tokens"."token_digest" = 'R27ZoLo1xAaezTWG/7z4uMnF9M8yUPN1lDKLEh0cQq4=' LIMIT 1
  ↳ app/models/concerns/token_authenticatable_strategies/digest.rb:8:in `find_token_authenticatable'
  Feature::FlipperGate Load (0.8ms)  SELECT "feature_gates".* FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'prometheus_metrics_method_instrumentation'
  ↳ lib/gitlab/metrics/methods.rb:55:in `disabled_by_feature'
  User Load (13.2ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
  ↳ lib/gitlab/auth/auth_finders.rb:102:in `find_user_from_access_token'
  Feature::FlipperGate Load (0.3ms)  SELECT "feature_gates".* FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'user_mode_in_session'
  ↳ lib/feature.rb:67:in `enabled?'
  License Load (0.6ms)  SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 100
  ↳ ee/app/models/license.rb:265:in `load_license'
  Group Load (9.9ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 54 LIMIT 1
  ↳ lib/api/helpers.rb:129:in `find_group'
  IpRestriction Load (2.1ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 54
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 54 LIMIT 1
  ↳ app/services/search_service.rb:36:in `group'
  Route Load (4.4ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 54 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (23.9ms)  SELECT "projects"."id" FROM "projects" INNER JOIN routes rs ON rs.source_id = projects.id AND rs.source_type = 'Project' WHERE (EXISTS (SELECT 1 FROM "project_authorizations" WHERE "project_authorizations"."user_id" = 1 AND (project_authorizations.project_id = projects.id)) OR projects.visibility_level IN (0,10,20)) AND "projects"."archived" = FALSE AND (rs.path LIKE 'qa-perf-test-land/%') ORDER BY "projects"."id" DESC
  ↳ ee/app/services/ee/search/group_service.rb:15:in `elastic_projects'
  Project Search (255.6ms) {index: "gitlab-development", type: "doc", body: {query: {bool: {must: [{simple_query_string: {fields: ["name^10", "name_with_namespace^2", "path_with_namespace", "path^9", "description"], query: "*", default_operator: :and}}], filter: [{terms: {type: ["project"]}}, {bool: {should: [{terms: {id: [63]}}]}}]}}, sort: [:_score], highlight: {fields: {name: {}, name_with_namespace: {}, path_with_namespace: {}, path: {}, description: {}}}}, routing: "project_63", size: 20, from: 0}
   (1.9ms)  SELECT COUNT(*) FROM "projects" WHERE "projects"."id" = 63
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Project Load (2.3ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 63 ORDER BY "projects"."id" ASC LIMIT 1
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Route Load (0.4ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."source_id" = 63
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Namespace Load (0.6ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" = 54
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  ComplianceManagement::ComplianceFramework::ProjectSettings Load (1.5ms)  SELECT "project_compliance_framework_settings".* FROM "project_compliance_framework_settings" WHERE "project_compliance_framework_settings"."project_id" = 63
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Group Load (0.6ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 54 AND "namespaces"."type" = 'Group' LIMIT 1
  ↳ ee/app/policies/ee/project_policy.rb:345:in `block (2 levels) in <module:ProjectPolicy>'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 54
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  ProjectFeature Load (2.7ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 63 LIMIT 1
  ↳ app/policies/project_policy.rb:633:in `feature_available?'
  Feature::FlipperGate Load (0.6ms)  SELECT "feature_gates".* FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'gitaly_enforce_requests_limits'
  ↳ lib/feature.rb:67:in `enabled?'
  ActsAsTaggableOn::Tag Load (4.0ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 63 AND "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' ORDER BY taggings.id
  ↳ lib/api/entities/basic_project_details.rb:15:in `map'
  Route Load (1.1ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 54 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (2.5ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 63
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 54 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
  License Load (0.7ms)  SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT $1  [["LIMIT", 100]]
  ↳ ee/app/models/license.rb:265:in `load_license'

After

k6 performance

█ API - Global Search

✓ { endpoint:projects }.........: avg=5810.39ms min=892.22ms  med=6793.91ms max=8057.54ms p(90)=7411.19ms p(95)=7616.76ms

█ API - Group Search

✓ { endpoint:projects }.........: avg=781.43ms  min=37.93ms   med=320.28ms  max=2731.70ms p(90)=2064.73ms p(95)=2288.17ms

█ Results summary

* Environment:                Localhost
* Environment Version:        13.0.0-pre `1efe97476c1`
* Option:                     60s_20rps
* Date:                       2020-05-20
* Run Time:                   3m 4.4s (Start: 23:18:04 UTC, End: 23:21:08 UTC)
* GPT Version:                v1.3.0

NAME                   | RPS  | RPS RESULT        | TTFB AVG  | TTFB P90             | REQ STATUS      | RESULT
-----------------------|------|-------------------|-----------|----------------------|-----------------|----------------
api_v4_search_global   | 20/s | 14.25/s (>4.80/s) | 5899.89ms | 7625.60ms (<25000ms) | 100.00% (>9.5%) | Passed
api_v4_search_groups   | 20/s | 19.2/s (>4.80/s)  | 913.01ms  | 2239.37ms (<25000ms) | 100.00% (>9.5%) | Passed
api_v4_search_projects | 20/s | 19.22/s (>4.80/s) | 321.12ms  | 788.37ms (<25000ms)  | 100.00% (>9.5%) | Passed

█ Results files

k6/results/localhost_v13-0-0-pre_2020-05-20_191804/localhost_v13-0-0-pre_2020-05-20_191804_results_output.log
k6/results/localhost_v13-0-0-pre_2020-05-20_191804/localhost_v13-0-0-pre_2020-05-20_191804_results.json
k6/results/localhost_v13-0-0-pre_2020-05-20_191804/localhost_v13-0-0-pre_2020-05-20_191804_results.txt

GET /api/v4/search?scope=projects&search=*

42 Queries

Started GET "/api/v4/search?scope=projects&search=*" for 127.0.0.1 at 2020-05-20 14:42:18 -0400
   (1.7ms)  SELECT EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))::float as lag
  ↳ lib/gitlab/middleware/basic_health_check.rb:25:in `call'
   (0.8ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
  ↳ lib/gitlab/middleware/basic_health_check.rb:25:in `call'
  ApplicationSetting Load (2.3ms)  SELECT "application_settings".* FROM "application_settings" ORDER BY "application_settings"."id" DESC LIMIT 1
  ↳ app/models/concerns/cacheable_attributes.rb:19:in `current_without_cache'
  PersonalAccessToken Load (0.8ms)  SELECT "personal_access_tokens".* FROM "personal_access_tokens" WHERE "personal_access_tokens"."token_digest" = 'R27ZoLo1xAaezTWG/7z4uMnF9M8yUPN1lDKLEh0cQq4=' LIMIT 1
  ↳ app/models/concerns/token_authenticatable_strategies/digest.rb:8:in `find_token_authenticatable'
  User Load (2.2ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
  ↳ lib/gitlab/auth/auth_finders.rb:102:in `find_user_from_access_token'
   (0.5ms)  SELECT "features"."key" FROM "features"
  ↳ lib/feature.rb:15:in `feature_names'
  License Load (0.5ms)  SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 100
  ↳ ee/app/models/license.rb:265:in `load_license'
  Project Search (180.2ms) {index: "gitlab-development", type: "doc", body: {query: {bool: {must: [{simple_query_string: {fields: ["name^10", "name_with_namespace^2", "path_with_namespace", "path^9", "description"], query: "*", default_operator: :and}}], filter: [{terms: {type: ["project"]}}, {bool: {should: [{term: {visibility_level: 0}}, {term: {visibility_level: 10}}, {term: {visibility_level: 20}}]}}]}}, sort: [:_score], highlight: {fields: {name: {}, name_with_namespace: {}, path_with_namespace: {}, path: {}, description: {}}}}, size: 20, from: 0}
   (3.9ms)  SELECT COUNT(*) FROM "projects" WHERE "projects"."id" IN (16, 3, 9, 17, 60, 70, 75, 8, 6, 4, 76, 83, 84, 10, 2, 14, 15, 61, 62, 68)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Project Load (1.8ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" IN (16, 3, 9, 17, 60, 70, 75, 8, 6, 4, 76, 83, 84, 10, 2, 14, 15, 61, 62, 68)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  ProjectFeature Load (1.0ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" IN (2, 60, 4, 62, 84, 3, 6, 61, 8, 9, 83, 10, 16, 14, 15, 68, 75, 17, 70, 76)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Route Load (0.7ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."source_id" IN (2, 60, 4, 62, 84, 3, 6, 61, 8, 9, 83, 10, 16, 14, 15, 68, 75, 17, 70, 76)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  ComplianceManagement::ComplianceFramework::ProjectSettings Load (1.4ms)  SELECT "project_compliance_framework_settings".* FROM "project_compliance_framework_settings" WHERE "project_compliance_framework_settings"."project_id" IN (2, 60, 4, 62, 84, 3, 6, 61, 8, 9, 83, 10, 16, 14, 15, 68, 75, 17, 70, 76)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  ActsAsTaggableOn::Tagging Load (2.6ms)  SELECT "taggings".* FROM "taggings" WHERE "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' AND "taggings"."taggable_id" IN (2, 60, 4, 62, 84, 3, 6, 61, 8, 9, 83, 10, 16, 14, 15, 68, 75, 17, 70, 76) ORDER BY taggings.id
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Group Load (1.7ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."type" = 'Group' AND "namespaces"."id" IN (22, 53, 24, 57, 23, 26, 28, 36, 1, 45, 16, 6, 35, 29)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  IpRestriction Load (0.4ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" IN (22, 23, 24, 26, 28, 53, 57)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Namespace Load (0.7ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" IN (22, 53, 24, 57, 23, 26, 28, 36, 1, 45, 16, 6, 35, 29)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Route Load (0.4ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."source_id" IN (1, 6, 16, 22, 23, 24, 26, 28, 29, 35, 36, 45, 53, 57)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  User Load (0.7ms)  SELECT "users".* FROM "users" WHERE "users"."id" IN (1, 6, 16, 22, 28, 29, 38)
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Group Load (0.7ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 53 AND "namespaces"."parent_id" IS NULL LIMIT 1
  ↳ app/models/namespace.rb:285:in `block in root_ancestor'
  SamlProvider Load (1.3ms)  SELECT "saml_providers".* FROM "saml_providers" WHERE "saml_providers"."group_id" = 53 LIMIT 1
  ↳ ee/lib/gitlab/auth/group_saml/sso_enforcer.rb:35:in `group_access_restricted?'
  Group Load (0.6ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 23 AND "namespaces"."parent_id" IS NULL LIMIT 1
  ↳ app/models/namespace.rb:285:in `block in root_ancestor'
  SamlProvider Load (0.2ms)  SELECT "saml_providers".* FROM "saml_providers" WHERE "saml_providers"."group_id" = 23 LIMIT 1
  ↳ ee/lib/gitlab/auth/group_saml/sso_enforcer.rb:35:in `group_access_restricted?'
   (2.6ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 76
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 60
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 4
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 62
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.4ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 84
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 3
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 6
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 61
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 8
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.6ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 9
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 83
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.5ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 10
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 16
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.4ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 14
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 15
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.4ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 68
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 75
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.4ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 17
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 70
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 2
  ↳ app/services/base_count_service.rb:26:in `uncached_count'

GET /api/v4/groups/54/search?scope=projects&search=*

22 Queries

Started GET "/api/v4/groups/54/search?scope=projects&search=*" for 127.0.0.1 at 2020-05-20 14:43:10 -0400
  Project Load (0.9ms)  SELECT "projects".* FROM "projects" INNER JOIN project_mirror_data import_state ON import_state.project_id = projects.id WHERE "projects"."archived" = $1 AND "projects"."mirror" = $2 AND "import_state"."status" NOT IN ($3, $4) AND (import_state.next_execution_timestamp <= '2020-05-20 18:43:10.931845') AND (import_state.retry_count <= 14) ORDER BY import_state.next_execution_timestamp LIMIT $5  [["archived", false], ["mirror", true], ["status", "scheduled"], ["status", "started"], ["LIMIT", 60]]
  ↳ ee/app/workers/update_all_mirrors_worker.rb:58:in `schedule_mirrors!'
  PersonalAccessToken Load (0.4ms)  SELECT "personal_access_tokens".* FROM "personal_access_tokens" WHERE "personal_access_tokens"."token_digest" = 'R27ZoLo1xAaezTWG/7z4uMnF9M8yUPN1lDKLEh0cQq4=' LIMIT 1
  ↳ app/models/concerns/token_authenticatable_strategies/digest.rb:8:in `find_token_authenticatable'
  User Load (0.5ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
  ↳ lib/gitlab/auth/auth_finders.rb:102:in `find_user_from_access_token'
  License Load (0.2ms)  SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 100
  ↳ ee/app/models/license.rb:265:in `load_license'
  Group Load (0.4ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 54 LIMIT 1
  ↳ lib/api/helpers.rb:129:in `find_group'
  IpRestriction Load (0.3ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 54
  ↳ ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  Group Load (0.5ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 54 LIMIT 1
  ↳ app/services/search_service.rb:36:in `group'
  Route Load (0.4ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 54 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'
   (11.2ms)  SELECT "projects"."id" FROM "projects" INNER JOIN routes rs ON rs.source_id = projects.id AND rs.source_type = 'Project' WHERE (EXISTS (SELECT 1 FROM "project_authorizations" WHERE "project_authorizations"."user_id" = 1 AND (project_authorizations.project_id = projects.id)) OR projects.visibility_level IN (0,10,20)) AND "projects"."archived" = FALSE AND (rs.path LIKE 'qa-perf-test-land/%') ORDER BY "projects"."id" DESC
  ↳ ee/app/services/ee/search/group_service.rb:15:in `elastic_projects'
  Project Search (27.2ms) {index: "gitlab-development", type: "doc", body: {query: {bool: {must: [{simple_query_string: {fields: ["name^10", "name_with_namespace^2", "path_with_namespace", "path^9", "description"], query: "*", default_operator: :and}}], filter: [{terms: {type: ["project"]}}, {bool: {should: [{terms: {id: [63]}}]}}]}}, sort: [:_score], highlight: {fields: {name: {}, name_with_namespace: {}, path_with_namespace: {}, path: {}, description: {}}}}, routing: "project_63", size: 20, from: 0}
   (0.6ms)  SELECT COUNT(*) FROM "projects" WHERE "projects"."id" = 63
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Project Load (0.6ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 63 ORDER BY "projects"."id" ASC LIMIT 1
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  ProjectFeature Load (0.7ms)  SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = 63
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."source_id" = 63
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  ComplianceManagement::ComplianceFramework::ProjectSettings Load (0.2ms)  SELECT "project_compliance_framework_settings".* FROM "project_compliance_framework_settings" WHERE "project_compliance_framework_settings"."project_id" = 63
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  ActsAsTaggableOn::Tagging Load (0.3ms)  SELECT "taggings".* FROM "taggings" WHERE "taggings"."taggable_type" = 'Project' AND "taggings"."context" = 'tags' AND "taggings"."taggable_id" = 63 ORDER BY taggings.id
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Group Load (0.4ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."type" = 'Group' AND "namespaces"."id" = 54
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  IpRestriction Load (0.2ms)  SELECT "ip_restrictions".* FROM "ip_restrictions" WHERE "ip_restrictions"."group_id" = 54
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Namespace Load (0.3ms)  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" = 54
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."source_id" = 54
  ↳ ee/lib/gitlab/elastic/search_results.rb:176:in `eager_load'
   (0.3ms)  SELECT COUNT(*) FROM "fork_network_members" WHERE "fork_network_members"."forked_from_project_id" = 63
  ↳ app/services/base_count_service.rb:26:in `uncached_count'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 54 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:77:in `full_path'

## Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
[-] Tested in all supported browsers
[-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

[-] Label as security and @ mention @gitlab-com/gl-security/appsec
[-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
[-] Security reports checked/validated by a reviewer from the AppSec team

Edited May 31, 2022 by 🤖 GitLab Bot 🤖

Resolve "Improve performance of Search API (Advanced): projects scope"

What does this MR do?

Performance Improvements

Before

k6 performance

GET /api/v4/search?scope=projects&search=*

GET /api/v4/search?scope=projects&search=*

After

k6 performance

GET /api/v4/search?scope=projects&search=*

GET /api/v4/groups/54/search?scope=projects&search=*

Conformity

Availability and Testing

Security

Merge request reports