Skip to content

find 500 Internal Server Error when get group API Endpoint: GET /groups/ or GET /groups/{id}/descendant_groups or /groups/{id}/subgroups

Summary

500 Internal Server Error when get groups API Endpotin: GET /groups/

GET /groups/{id}/descendant_groups/ and GET /groups/{id}/subgroups similary with GET /groups/

Steps to reproduce

  1. curl --location --request POST 'http://192.168.1.123:30000/api/v4/groups?name=dcR&path=VJurK' --header 'Authorization: Bearer xxx'

  2. curl --location --request GET 'http://192.168.1.123:30000/api/v4/groups?all_available=false&statistics=true' --header 'Authorization: Bearer xxx'

get the group descendant created in step 1

  1. curl --location --request GET 'http://192.168.1.123:30000/api/v4/groups/11882/descendant_groups?all_available=false&statistics=true' --header 'Authorization: Bearer xxx'

get the group subgroups created in step 1

  1. curl --location --request GET 'http://192.168.1.123:30000/api/v4/groups/11882/subgroups?all_available=false&statistics=true' --header 'Authorization: Bearer xxx'

Example Project

The bug is reproducible on our local deployment of enterprise edition (14.4.2-ee).

What is the current bug behavior?

HTTP Response: "500 Internal Server Error."

What is the expected correct behavior?

The request should return HTTP status code 200 and groups information with statistics.

Relevant logs and/or screenshots

==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/api/v4/groups/11882/descendant_groups?all_available=false&statistics=true" for 172.17.0.1 at 2022-10-23 06:39:00 +0000

==> /var/log/gitlab/postgresql/current <==
2022-10-23_06:39:00.44276 ERROR:  column "namespaces.name" must appear in the GROUP BY clause or be used in an aggregate function at character 16010
2022-10-23_06:39:00.44281 STATEMENT:  /*application:web,correlation_id:01GG1S1V9KJHSTJ3HDJF8EMT5C,endpoint_id:GET /api/:version/groups/:id/descendant_groups,db_config_name:main*/ WITH RECURSIVE "base_and_ancestors" AS ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM ((WITH "direct_groups" AS MATERIALIZED (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM ((SELECT "namespaces".* FROM "namespaces" INNER JOIN "members" ON "namespaces"."id" = "members"."source_id" WHERE "members"."type" = 'GroupMember' AND "members"."source_type" = 'Namespace' AND "namespaces"."type" = 'Group' AND "members"."user_id" = 1 AND "members"."requested_at" IS NULL AND (access_level >= 10))
2022-10-23_06:39:00.44286       UNION
2022-10-23_06:39:00.44286       (SELECT "namespaces".* FROM "projects" INNER JOIN "project_authorizations" ON "projects"."id" = "project_authorizations"."project_id" INNER JOIN "namespaces" ON "namespaces"."id" = "projects"."namespace_id" WHERE "project_authorizations"."user_id" = 1)) namespaces WHERE "namespaces"."type" = 'Group') SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "direct_groups" "namespaces" WHERE "namespaces"."type" = 'Group')
2022-10-23_06:39:00.44290       UNION
2022-10-23_06:39:00.44290       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" INNER JOIN "group_group_links" ON "group_group_links"."shared_group_id" = "namespaces"."id" WHERE "namespaces"."type" = 'Group' AND "group_group_links"."shared_with_group_id" IN (SELECT "namespaces"."id" FROM "direct_groups" "namespaces" WHERE "namespaces"."type" = 'Group'))) namespaces WHERE "namespaces"."type" = 'Group')) namespaces WHERE "namespaces"."type" = 'Group')
2022-10-23_06:39:00.44293       UNION
2022-10-23_06:39:00.44294       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces", "base_and_ancestors" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = "base_and_ancestors"."parent_id")), "base_and_descendants" AS ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" INNER JOIN "members" ON "namespaces"."id" = "members"."source_id" WHERE "members"."type" = 'GroupMember' AND "members"."source_type" = 'Namespace' AND "namespaces"."type" = 'Group' AND "members"."user_id" = 1 AND "members"."requested_at" IS NULL AND (access_level >= 10))
2022-10-23_06:39:00.44301       UNION
2022-10-23_06:39:00.44301       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces", "base_and_descendants" WHERE "namespaces"."type" = 'Group' AND "namespaces"."parent_id" = "base_and_descendants"."id")) SELECT namespaces.*, COALESCE(SUM(ps.storage_size), 0) AS storage_size, COALESCE(SUM(ps.repository_size), 0) AS repository_size, COALESCE(SUM(ps.wiki_size), 0) AS wiki_size, COALESCE(SUM(ps.snippets_size), 0) AS snippets_size, COALESCE(SUM(ps.lfs_objects_size), 0) AS lfs_objects_size, COALESCE(SUM(ps.build_artifacts_size), 0) AS build_artifacts_size, COALESCE(SUM(ps.packages_size), 0) AS packages_size, COALESCE(SUM(ps.uploads_size), 0) AS uploads_size FROM ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "base_and_ancestors" AS "namespaces" WHERE "namespaces"."type" = 'Group')
2022-10-23_06:39:00.44312       UNION
2022-10-23_06:39:00.44312       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "base_and_descendants" AS "namespaces" WHERE "namespaces"."type" = 'Group')) namespaces LEFT JOIN project_statistics ps ON ps.namespace_id = namespaces.id WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" IN (WITH RECURSIVE "base_and_descendants" AS ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."parent_id" = 11882)
2022-10-23_06:39:00.44319       UNION
2022-10-23_06:39:00.44319       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces", "base_and_descendants" WHERE "namespaces"."type" = 'Group' AND "namespaces"."parent_id" = "base_and_descendants"."id")) SELECT "namespaces"."id" FROM "base_and_descendants" AS "namespaces") GROUP BY "namespaces"."id" ORDER BY "namespaces"."name" ASC, "namespaces"."id" ASC LIMIT 20 OFFSET 0

==> /var/log/gitlab/gitlab-rails/exceptions_json.log <==
{"severity":"ERROR","time":"2022-10-23T06:39:00.449Z","correlation_id":"01GG1S1V9KJHSTJ3HDJF8EMT5C","exception.class":"ActiveRecord::StatementInvalid","exception.message":"PG::GroupingError: ERROR:  column \"namespaces.name\" must appear in the GROUP BY clause or be used in an aggregate function\nLINE 9: ...parent_id\" = \"base_and_descendants\".\"id\")) SELECT namespaces...\n                                                             ^\n","exception.backtrace":["lib/gitlab/database/load_balancing/connection_proxy.rb:103:in `block in read_using_load_balancer'","lib/gitlab/database/load_balancing/load_balancer.rb:53:in `read'","lib/gitlab/database/load_balancing/connection_proxy.rb:102:in `read_using_load_balancer'","lib/gitlab/database/load_balancing/connection_proxy.rb:47:in `select_all'","lib/api/groups.rb:108:in `present_groups'","lib/api/groups.rb:362:in `block (2 levels) in \u003cclass:Groups\u003e'","ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call'","ee/lib/gitlab/ip_address_state.rb:10:in `with'","ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call'","lib/api/api_guard.rb:213:in `call'","lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'","lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'","lib/gitlab/middleware/speedscope.rb:13:in `call'","lib/gitlab/request_profiler/middleware.rb:17:in `call'","lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'","lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'","lib/gitlab/metrics/web_transaction.rb:21:in `run'","lib/gitlab/metrics/rack_middleware.rb:16:in `call'","lib/gitlab/jira/middleware.rb:19:in `call'","lib/gitlab/middleware/go.rb:20:in `call'","lib/gitlab/etag_caching/middleware.rb:21:in `call'","lib/gitlab/middleware/multipart.rb:173:in `call'","lib/gitlab/middleware/read_only/controller.rb:50:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:21:in `call'","config/initializers/fix_local_cache_middleware.rb:11:in `call'","lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'","lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:75:in `call'","lib/gitlab/middleware/release_env.rb:12:in `call'"],"user.username":"root","tags.program":"web","tags.locale":"en","tags.feature_category":"subgroups","tags.correlation_id":"01GG1S1V9KJHSTJ3HDJF8EMT5C"}

==> /var/log/gitlab/gitlab-rails/production.log <==

ActiveRecord::StatementInvalid (PG::GroupingError: ERROR:  column "namespaces.name" must appear in the GROUP BY clause or be used in an aggregate function
LINE 9: ...parent_id" = "base_and_descendants"."id")) SELECT namespaces...
                                                             ^
):
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `exec_params'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:671:in `block in exec_no_cache'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:670:in `exec_no_cache'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:649:in `execute_and_clear'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/marginalia-1.10.0/lib/marginalia.rb:95:in `execute_and_clear_with_marginalia'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:53:in `exec_query'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/database_statements.rb:532:in `select'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/database_statements.rb:69:in `select_all'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/query_cache.rb:101:in `block in select_all'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/query_cache.rb:118:in `block in cache_sql'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/query_cache.rb:109:in `cache_sql'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/query_cache.rb:101:in `select_all'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:103:in `block in read_using_load_balancer'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:53:in `read'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:102:in `read_using_load_balancer'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:47:in `select_all'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/querying.rb:47:in `find_by_sql'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:843:in `block in exec_queries'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:861:in `skip_query_cache_if_necessary'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:828:in `exec_queries'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:631:in `load'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:249:in `records'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:244:in `to_ary'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-entity-0.10.0/lib/grape_entity/entity.rb:443:in `represent'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/dsl/inside_route.rb:424:in `entity_representation_for'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/dsl/inside_route.rb:361:in `present'
  /opt/gitlab/embedded/service/gitlab-rails/lib/api/groups.rb:108:in `present_groups'
  /opt/gitlab/embedded/service/gitlab-rails/lib/api/groups.rb:362:in `block (2 levels) in <class:Groups>'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:59:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:59:in `block (2 levels) in generate_api_method'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/notifications.rb:205:in `instrument'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:58:in `block in generate_api_method'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:341:in `execute'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:267:in `block in run'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/notifications.rb:205:in `instrument'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:247:in `run'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:322:in `block in build_stack'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:36:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:36:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call'
  /opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/ip_address_state.rb:10:in `with'
  /opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:36:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/api/api_guard.rb:213:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:36:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-oauth2-1.16.0/lib/rack/oauth2/server/resource.rb:20:in `_call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-oauth2-1.16.0/lib/rack/oauth2/server/resource/bearer.rb:8:in `_call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-oauth2-1.16.0/lib/rack/oauth2/server/abstract/handler.rb:17:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/error.rb:39:in `block in call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/error.rb:38:in `catch'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/error.rb:38:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape_logging-1.8.3/lib/grape_logging/middleware/request_logger.rb:60:in `block in call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape_logging-1.8.3/lib/grape_logging/middleware/request_logger.rb:58:in `catch'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape_logging-1.8.3/lib/grape_logging/middleware/request_logger.rb:58:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:231:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:225:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router/route.rb:58:in `exec'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:116:in `process_route'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:72:in `block in identity'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:91:in `transaction'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:70:in `identity'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:55:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:132:in `with_optimization'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:54:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/api/instance.rb:167:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/api/instance.rb:71:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/api/instance.rb:66:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/mapper.rb:20:in `block in <class:Constraints>'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/mapper.rb:49:in `serve'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:50:in `block in serve'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:32:in `each'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:32:in `serve'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/route_set.rb:842:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/gitlab-experiment-0.6.4/lib/gitlab/experiment/middleware.rb:19:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/flipper-0.21.0/lib/flipper/middleware/memoizer.rb:77:in `memoized_call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/flipper-0.21.0/lib/flipper/middleware/memoizer.rb:42:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/speedscope.rb:13:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/request_profiler/middleware.rb:17:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/web_transaction.rb:21:in `run'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/rack_middleware.rb:16:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/jira/middleware.rb:19:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/go.rb:20:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/etag_caching/middleware.rb:21:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/batch-loader-2.0.1/lib/batch_loader/middleware.rb:11:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-attack-6.3.0/lib/rack/attack.rb:97:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/apollo_upload_server-2.1.0/lib/apollo_upload_server/middleware.rb:19:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/multipart.rb:173:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-attack-6.3.0/lib/rack/attack.rb:111:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/warden-1.2.8/lib/warden/manager.rb:36:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/warden-1.2.8/lib/warden/manager.rb:34:in `catch'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/warden-1.2.8/lib/warden/manager.rb:34:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-cors-1.0.6/lib/rack/cors.rb:98:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/conditional_get.rb:27:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/http/permissions_policy.rb:22:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/read_only/controller.rb:50:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/read_only.rb:18:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/cookies.rb:689:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/basic_health_check.rb:25:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:37:in `call_app'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:26:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:99:in `block in tagged'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:37:in `tagged'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:99:in `tagged'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:26:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/request_context.rb:21:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/request_store-1.5.0/lib/request_store/middleware.rb:19:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-timeout-0.5.2/lib/rack/timeout/core.rb:123:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-timeout-0.5.2/lib/rack/timeout/support/timeout.rb:19:in `timeout'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-timeout-0.5.2/lib/rack/timeout/core.rb:122:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/config/initializers/fix_local_cache_middleware.rb:11:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/host_authorization.rb:92:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/requests_rack_middleware.rb:75:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/gitlab-labkit-0.21.1/lib/labkit/middleware/rack.rb:19:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/gitlab-labkit-0.21.1/lib/labkit/context.rb:39:in `with_context'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/gitlab-labkit-0.21.1/lib/labkit/middleware/rack.rb:18:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/request_id.rb:26:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/sentry-raven-3.1.2/lib/raven/integrations/rack.rb:51:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/engine.rb:539:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/railtie.rb:207:in `public_send'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/railtie.rb:207:in `method_missing'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/release_env.rb:12:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/configuration.rb:249:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/request.rb:77:in `block in handle_request'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:338:in `with_force_shutdown'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/request.rb:76:in `handle_request'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/server.rb:438:in `process_client'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:145:in `block in spawn_thread'

==> /var/log/gitlab/gitlab-rails/api_json.log <==
{"time":"2022-10-23T06:39:00.450Z","severity":"INFO","duration_s":0.03684,"db_duration_s":0.01292,"view_duration_s":0.02392,"status":500,"method":"GET","path":"/api/v4/groups/11882/descendant_groups","params":[{"key":"all_available","value":"false"},{"key":"statistics","value":"true"}],"host":"192.168.1.123","remote_ip":"172.17.0.1, 127.0.0.1","ua":"PostmanRuntime/7.29.2","route":"/api/:version/groups/:id/descendant_groups","user_id":1,"username":"root","exception.class":"ActiveRecord::StatementInvalid","exception.message":"PG::GroupingError: ERROR:  column \"namespaces.name\" must appear in the GROUP BY clause or be used in an aggregate function\nLINE 9: ...parent_id\" = \"base_and_descendants\".\"id\")) SELECT namespaces...\n                                                             ^\n","exception.backtrace":["lib/gitlab/database/load_balancing/connection_proxy.rb:103:in `block in read_using_load_balancer'","lib/gitlab/database/load_balancing/load_balancer.rb:53:in `read'","lib/gitlab/database/load_balancing/connection_proxy.rb:102:in `read_using_load_balancer'","lib/gitlab/database/load_balancing/connection_proxy.rb:47:in `select_all'","lib/api/groups.rb:108:in `present_groups'","lib/api/groups.rb:362:in `block (2 levels) in <class:Groups>'","ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call'","ee/lib/gitlab/ip_address_state.rb:10:in `with'","ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call'","lib/api/api_guard.rb:213:in `call'","lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'","lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'","lib/gitlab/middleware/speedscope.rb:13:in `call'","lib/gitlab/request_profiler/middleware.rb:17:in `call'","lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'","lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'","lib/gitlab/metrics/web_transaction.rb:21:in `run'","lib/gitlab/metrics/rack_middleware.rb:16:in `call'","lib/gitlab/jira/middleware.rb:19:in `call'","lib/gitlab/middleware/go.rb:20:in `call'","lib/gitlab/etag_caching/middleware.rb:21:in `call'","lib/gitlab/middleware/multipart.rb:173:in `call'","lib/gitlab/middleware/read_only/controller.rb:50:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:21:in `call'","config/initializers/fix_local_cache_middleware.rb:11:in `call'","lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'","lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:75:in `call'","lib/gitlab/middleware/release_env.rb:12:in `call'"],"queue_duration_s":0.009205,"redis_calls":5,"redis_duration_s":0.001166,"redis_read_bytes":203,"redis_write_bytes":254,"redis_cache_calls":1,"redis_cache_duration_s":0.000224,"redis_cache_read_bytes":203,"redis_cache_write_bytes":52,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000942,"redis_shared_state_write_bytes":202,"db_count":11,"db_write_count":0,"db_cached_count":3,"db_replica_count":0,"db_replica_cached_count":0,"db_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_count":11,"db_primary_cached_count":3,"db_primary_wal_count":0,"db_primary_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"cpu_s":0.034913,"mem_objects":20276,"mem_bytes":1627913,"mem_mallocs":4916,"mem_total_bytes":2438953,"pid":10203,"correlation_id":"01GG1S1V9KJHSTJ3HDJF8EMT5C","meta.user":"root","meta.caller_id":"GET /api/:version/groups/:id/descendant_groups","meta.remote_ip":"172.17.0.1","meta.feature_category":"subgroups","meta.client_id":"user/1"}

==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"application/json","correlation_id":"01GG1S1V9KJHSTJ3HDJF8EMT5C","duration_ms":48,"host":"192.168.1.123:30000","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/","status":500,"system":"http","time":"2022-10-23T06:39:00Z","ttfb_ms":48,"uri":"/api/v4/groups/11882/descendant_groups?all_available=false\u0026statistics=true","user_agent":"PostmanRuntime/7.29.2","written_bytes":39}

==> /var/log/gitlab/nginx/gitlab_access.log <==
172.17.0.1 - - [23/Oct/2022:06:39:00 +0000] "GET /api/v4/groups/11882/descendant_groups?all_available=false&statistics=true HTTP/1.1" 500 39 "" "PostmanRuntime/7.29.2" -

==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/api/v4/groups/11882/subgroups?all_available=false&statistics=true" for 172.17.0.1 at 2022-10-23 06:39:02 +0000

==> /var/log/gitlab/sidekiq/current <==
{"severity":"INFO","time":"2022-10-23T06:39:02.512Z","message":"Enqueuing scheduled jobs","status":"start","sorted_set":"retry","retry":0}
{"severity":"INFO","time":"2022-10-23T06:39:02.513Z","message":"Enqueuing scheduled jobs","status":"done","sorted_set":"retry","jobs_count":0,"redundant_jobs_count":0,"duration_s":0.0004865999799221754,"retry":0}
{"severity":"INFO","time":"2022-10-23T06:39:02.513Z","message":"Enqueuing scheduled jobs","status":"start","sorted_set":"schedule","retry":0}
{"severity":"INFO","time":"2022-10-23T06:39:02.513Z","message":"Enqueuing scheduled jobs","status":"done","sorted_set":"schedule","jobs_count":0,"redundant_jobs_count":0,"duration_s":0.0003461000742390752,"retry":0}

==> /var/log/gitlab/postgresql/current <==
2022-10-23_06:39:02.68694 ERROR:  column "namespaces.name" must appear in the GROUP BY clause or be used in an aggregate function at character 16002
2022-10-23_06:39:02.68700 STATEMENT:  /*application:web,correlation_id:01GG1S1X9ZENDRH96VAMHTPD2M,endpoint_id:GET /api/:version/groups/:id/subgroups,db_config_name:main*/ WITH RECURSIVE "base_and_ancestors" AS ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM ((WITH "direct_groups" AS MATERIALIZED (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM ((SELECT "namespaces".* FROM "namespaces" INNER JOIN "members" ON "namespaces"."id" = "members"."source_id" WHERE "members"."type" = 'GroupMember' AND "members"."source_type" = 'Namespace' AND "namespaces"."type" = 'Group' AND "members"."user_id" = 1 AND "members"."requested_at" IS NULL AND (access_level >= 10))
2022-10-23_06:39:02.68704       UNION
2022-10-23_06:39:02.68705       (SELECT "namespaces".* FROM "projects" INNER JOIN "project_authorizations" ON "projects"."id" = "project_authorizations"."project_id" INNER JOIN "namespaces" ON "namespaces"."id" = "projects"."namespace_id" WHERE "project_authorizations"."user_id" = 1)) namespaces WHERE "namespaces"."type" = 'Group') SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "direct_groups" "namespaces" WHERE "namespaces"."type" = 'Group')
2022-10-23_06:39:02.68708       UNION
2022-10-23_06:39:02.68708       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" INNER JOIN "group_group_links" ON "group_group_links"."shared_group_id" = "namespaces"."id" WHERE "namespaces"."type" = 'Group' AND "group_group_links"."shared_with_group_id" IN (SELECT "namespaces"."id" FROM "direct_groups" "namespaces" WHERE "namespaces"."type" = 'Group'))) namespaces WHERE "namespaces"."type" = 'Group')) namespaces WHERE "namespaces"."type" = 'Group')
2022-10-23_06:39:02.68717       UNION
2022-10-23_06:39:02.68717       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces", "base_and_ancestors" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = "base_and_ancestors"."parent_id")), "base_and_descendants" AS ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" INNER JOIN "members" ON "namespaces"."id" = "members"."source_id" WHERE "members"."type" = 'GroupMember' AND "members"."source_type" = 'Namespace' AND "namespaces"."type" = 'Group' AND "members"."user_id" = 1 AND "members"."requested_at" IS NULL AND (access_level >= 10))
2022-10-23_06:39:02.68722       UNION
2022-10-23_06:39:02.68722       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces", "base_and_descendants" WHERE "namespaces"."type" = 'Group' AND "namespaces"."parent_id" = "base_and_descendants"."id")) SELECT namespaces.*, COALESCE(SUM(ps.storage_size), 0) AS storage_size, COALESCE(SUM(ps.repository_size), 0) AS repository_size, COALESCE(SUM(ps.wiki_size), 0) AS wiki_size, COALESCE(SUM(ps.snippets_size), 0) AS snippets_size, COALESCE(SUM(ps.lfs_objects_size), 0) AS lfs_objects_size, COALESCE(SUM(ps.build_artifacts_size), 0) AS build_artifacts_size, COALESCE(SUM(ps.packages_size), 0) AS packages_size, COALESCE(SUM(ps.uploads_size), 0) AS uploads_size FROM ((SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "base_and_ancestors" AS "namespaces" WHERE "namespaces"."type" = 'Group')
2022-10-23_06:39:02.68733       UNION
2022-10-23_06:39:02.68733       (SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "base_and_descendants" AS "namespaces" WHERE "namespaces"."type" = 'Group')) namespaces LEFT JOIN project_statistics ps ON ps.namespace_id = namespaces.id WHERE "namespaces"."type" = 'Group' AND "namespaces"."parent_id" = 11882 GROUP BY "namespaces"."id" ORDER BY "namespaces"."name" ASC, "namespaces"."id" ASC LIMIT 20 OFFSET 0

==> /var/log/gitlab/gitlab-rails/exceptions_json.log <==
{"severity":"ERROR","time":"2022-10-23T06:39:02.693Z","correlation_id":"01GG1S1X9ZENDRH96VAMHTPD2M","exception.class":"ActiveRecord::StatementInvalid","exception.message":"PG::GroupingError: ERROR:  column \"namespaces.name\" must appear in the GROUP BY clause or be used in an aggregate function\nLINE 9: ...parent_id\" = \"base_and_descendants\".\"id\")) SELECT namespaces...\n                                                             ^\n","exception.backtrace":["lib/gitlab/database/load_balancing/connection_proxy.rb:103:in `block in read_using_load_balancer'","lib/gitlab/database/load_balancing/load_balancer.rb:53:in `read'","lib/gitlab/database/load_balancing/connection_proxy.rb:102:in `read_using_load_balancer'","lib/gitlab/database/load_balancing/connection_proxy.rb:47:in `select_all'","lib/api/groups.rb:108:in `present_groups'","lib/api/groups.rb:349:in `block (2 levels) in \u003cclass:Groups\u003e'","ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call'","ee/lib/gitlab/ip_address_state.rb:10:in `with'","ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call'","lib/api/api_guard.rb:213:in `call'","lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'","lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'","lib/gitlab/middleware/speedscope.rb:13:in `call'","lib/gitlab/request_profiler/middleware.rb:17:in `call'","lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'","lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'","lib/gitlab/metrics/web_transaction.rb:21:in `run'","lib/gitlab/metrics/rack_middleware.rb:16:in `call'","lib/gitlab/jira/middleware.rb:19:in `call'","lib/gitlab/middleware/go.rb:20:in `call'","lib/gitlab/etag_caching/middleware.rb:21:in `call'","lib/gitlab/middleware/multipart.rb:173:in `call'","lib/gitlab/middleware/read_only/controller.rb:50:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:21:in `call'","config/initializers/fix_local_cache_middleware.rb:11:in `call'","lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'","lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:75:in `call'","lib/gitlab/middleware/release_env.rb:12:in `call'"],"user.username":"root","tags.program":"web","tags.locale":"en","tags.feature_category":"subgroups","tags.correlation_id":"01GG1S1X9ZENDRH96VAMHTPD2M"}

==> /var/log/gitlab/gitlab-rails/production.log <==

ActiveRecord::StatementInvalid (PG::GroupingError: ERROR:  column "namespaces.name" must appear in the GROUP BY clause or be used in an aggregate function
LINE 9: ...parent_id" = "base_and_descendants"."id")) SELECT namespaces...
                                                             ^
):
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `exec_params'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:671:in `block in exec_no_cache'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:670:in `exec_no_cache'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql_adapter.rb:649:in `execute_and_clear'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/marginalia-1.10.0/lib/marginalia.rb:95:in `execute_and_clear_with_marginalia'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:53:in `exec_query'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/database_statements.rb:532:in `select'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/database_statements.rb:69:in `select_all'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/query_cache.rb:101:in `block in select_all'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/query_cache.rb:118:in `block in cache_sql'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/query_cache.rb:109:in `cache_sql'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/query_cache.rb:101:in `select_all'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:103:in `block in read_using_load_balancer'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:53:in `read'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:102:in `read_using_load_balancer'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:47:in `select_all'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/querying.rb:47:in `find_by_sql'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:843:in `block in exec_queries'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:861:in `skip_query_cache_if_necessary'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:828:in `exec_queries'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:631:in `load'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:249:in `records'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation.rb:244:in `to_ary'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-entity-0.10.0/lib/grape_entity/entity.rb:443:in `represent'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/dsl/inside_route.rb:424:in `entity_representation_for'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/dsl/inside_route.rb:361:in `present'
  /opt/gitlab/embedded/service/gitlab-rails/lib/api/groups.rb:108:in `present_groups'
  /opt/gitlab/embedded/service/gitlab-rails/lib/api/groups.rb:349:in `block (2 levels) in <class:Groups>'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:59:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:59:in `block (2 levels) in generate_api_method'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/notifications.rb:205:in `instrument'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:58:in `block in generate_api_method'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:341:in `execute'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:267:in `block in run'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/notifications.rb:205:in `instrument'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:247:in `run'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:322:in `block in build_stack'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:36:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:36:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call'
  /opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/ip_address_state.rb:10:in `with'
  /opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:36:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/api/api_guard.rb:213:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:36:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-oauth2-1.16.0/lib/rack/oauth2/server/resource.rb:20:in `_call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-oauth2-1.16.0/lib/rack/oauth2/server/resource/bearer.rb:8:in `_call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-oauth2-1.16.0/lib/rack/oauth2/server/abstract/handler.rb:17:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/error.rb:39:in `block in call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/error.rb:38:in `catch'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/error.rb:38:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape_logging-1.8.3/lib/grape_logging/middleware/request_logger.rb:60:in `block in call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape_logging-1.8.3/lib/grape_logging/middleware/request_logger.rb:58:in `catch'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape_logging-1.8.3/lib/grape_logging/middleware/request_logger.rb:58:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/middleware/base.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:231:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/endpoint.rb:225:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router/route.rb:58:in `exec'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:116:in `process_route'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:72:in `block in identity'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:91:in `transaction'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:70:in `identity'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:55:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:132:in `with_optimization'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/router.rb:54:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/api/instance.rb:167:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/api/instance.rb:71:in `call!'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/grape-1.5.2/lib/grape/api/instance.rb:66:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/mapper.rb:20:in `block in <class:Constraints>'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/mapper.rb:49:in `serve'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:50:in `block in serve'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:32:in `each'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:32:in `serve'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/route_set.rb:842:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/gitlab-experiment-0.6.4/lib/gitlab/experiment/middleware.rb:19:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/flipper-0.21.0/lib/flipper/middleware/memoizer.rb:77:in `memoized_call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/flipper-0.21.0/lib/flipper/middleware/memoizer.rb:42:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/speedscope.rb:13:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/request_profiler/middleware.rb:17:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/web_transaction.rb:21:in `run'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/rack_middleware.rb:16:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/jira/middleware.rb:19:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/go.rb:20:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/etag_caching/middleware.rb:21:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/batch-loader-2.0.1/lib/batch_loader/middleware.rb:11:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-attack-6.3.0/lib/rack/attack.rb:97:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/apollo_upload_server-2.1.0/lib/apollo_upload_server/middleware.rb:19:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/multipart.rb:173:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-attack-6.3.0/lib/rack/attack.rb:111:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/warden-1.2.8/lib/warden/manager.rb:36:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/warden-1.2.8/lib/warden/manager.rb:34:in `catch'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/warden-1.2.8/lib/warden/manager.rb:34:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-cors-1.0.6/lib/rack/cors.rb:98:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/conditional_get.rb:27:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/http/permissions_policy.rb:22:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/read_only/controller.rb:50:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/read_only.rb:18:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/cookies.rb:689:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/basic_health_check.rb:25:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:37:in `call_app'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:26:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:99:in `block in tagged'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:37:in `tagged'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:99:in `tagged'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:26:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/request_context.rb:21:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/request_store-1.5.0/lib/request_store/middleware.rb:19:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-timeout-0.5.2/lib/rack/timeout/core.rb:123:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-timeout-0.5.2/lib/rack/timeout/support/timeout.rb:19:in `timeout'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-timeout-0.5.2/lib/rack/timeout/core.rb:122:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/config/initializers/fix_local_cache_middleware.rb:11:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/host_authorization.rb:92:in `call'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/requests_rack_middleware.rb:75:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/gitlab-labkit-0.21.1/lib/labkit/middleware/rack.rb:19:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/gitlab-labkit-0.21.1/lib/labkit/context.rb:39:in `with_context'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/gitlab-labkit-0.21.1/lib/labkit/middleware/rack.rb:18:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/request_id.rb:26:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/sentry-raven-3.1.2/lib/raven/integrations/rack.rb:51:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/engine.rb:539:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/railtie.rb:207:in `public_send'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/railties-6.1.4.1/lib/rails/railtie.rb:207:in `method_missing'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/release_env.rb:12:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/configuration.rb:249:in `call'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/request.rb:77:in `block in handle_request'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:338:in `with_force_shutdown'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/request.rb:76:in `handle_request'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/server.rb:438:in `process_client'
  /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/puma-5.3.2/lib/puma/thread_pool.rb:145:in `block in spawn_thread'

==> /var/log/gitlab/gitlab-rails/api_json.log <==
{"time":"2022-10-23T06:39:02.695Z","severity":"INFO","duration_s":0.22617,"db_duration_s":0.20509,"view_duration_s":0.02108,"status":500,"method":"GET","path":"/api/v4/groups/11882/subgroups","params":[{"key":"all_available","value":"false"},{"key":"statistics","value":"true"}],"host":"192.168.1.123","remote_ip":"172.17.0.1, 127.0.0.1","ua":"PostmanRuntime/7.29.2","route":"/api/:version/groups/:id/subgroups","user_id":1,"username":"root","exception.class":"ActiveRecord::StatementInvalid","exception.message":"PG::GroupingError: ERROR:  column \"namespaces.name\" must appear in the GROUP BY clause or be used in an aggregate function\nLINE 9: ...parent_id\" = \"base_and_descendants\".\"id\")) SELECT namespaces...\n                                                             ^\n","exception.backtrace":["lib/gitlab/database/load_balancing/connection_proxy.rb:103:in `block in read_using_load_balancer'","lib/gitlab/database/load_balancing/load_balancer.rb:53:in `read'","lib/gitlab/database/load_balancing/connection_proxy.rb:102:in `read_using_load_balancer'","lib/gitlab/database/load_balancing/connection_proxy.rb:47:in `select_all'","lib/api/groups.rb:108:in `present_groups'","lib/api/groups.rb:349:in `block (2 levels) in <class:Groups>'","ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call'","ee/lib/gitlab/ip_address_state.rb:10:in `with'","ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call'","lib/api/api_guard.rb:213:in `call'","lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'","lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'","lib/gitlab/middleware/speedscope.rb:13:in `call'","lib/gitlab/request_profiler/middleware.rb:17:in `call'","lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'","lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'","lib/gitlab/metrics/web_transaction.rb:21:in `run'","lib/gitlab/metrics/rack_middleware.rb:16:in `call'","lib/gitlab/jira/middleware.rb:19:in `call'","lib/gitlab/middleware/go.rb:20:in `call'","lib/gitlab/etag_caching/middleware.rb:21:in `call'","lib/gitlab/middleware/multipart.rb:173:in `call'","lib/gitlab/middleware/read_only/controller.rb:50:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:21:in `call'","config/initializers/fix_local_cache_middleware.rb:11:in `call'","lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'","lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:75:in `call'","lib/gitlab/middleware/release_env.rb:12:in `call'"],"queue_duration_s":0.004983,"redis_calls":5,"redis_duration_s":0.0012920000000000002,"redis_read_bytes":203,"redis_write_bytes":319,"redis_cache_calls":1,"redis_cache_duration_s":0.000226,"redis_cache_read_bytes":203,"redis_cache_write_bytes":64,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001066,"redis_shared_state_write_bytes":255,"db_count":11,"db_write_count":0,"db_cached_count":3,"db_replica_count":0,"db_replica_cached_count":0,"db_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_count":11,"db_primary_cached_count":3,"db_primary_wal_count":0,"db_primary_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.205,"cpu_s":0.027652,"mem_objects":13932,"mem_bytes":1295025,"mem_mallocs":3412,"mem_total_bytes":1852305,"pid":10322,"correlation_id":"01GG1S1X9ZENDRH96VAMHTPD2M","meta.user":"root","meta.caller_id":"GET /api/:version/groups/:id/subgroups","meta.remote_ip":"172.17.0.1","meta.feature_category":"subgroups","meta.client_id":"user/1"}

==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"application/json","correlation_id":"01GG1S1X9ZENDRH96VAMHTPD2M","duration_ms":233,"host":"192.168.1.123:30000","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/","status":500,"system":"http","time":"2022-10-23T06:39:02Z","ttfb_ms":232,"uri":"/api/v4/groups/11882/subgroups?all_available=false\u0026statistics=true","user_agent":"PostmanRuntime/7.29.2","written_bytes":39}

==> /var/log/gitlab/nginx/gitlab_access.log <==
172.17.0.1 - - [23/Oct/2022:06:39:02 +0000] "GET /api/v4/groups/11882/subgroups?all_available=false&statistics=true HTTP/1.1" 500 39 "" "PostmanRuntime/7.29.2" -

Output of checks

Results of GitLab application Check

Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 13.21.1 ? ... OK (13.21.1)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 1/1

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ...
yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet)
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
Administrator / SI0 ... yes
Redis version >= 5.0.0? ... yes
Ruby version >= 2.7.2 ? ... yes (2.7.4)
Git version >= 2.33.0 ? ... yes (2.33.0)
Git user has default SSH configuration? ... yes
Active users: ... 1
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 7.x (6.4 - 6.x deprecated to be removed in 13.8)? ... skipped (elasticsearch is disabled)

Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished