Skip to content

Create new group value stream

Magdalena Frankiewicz requested to merge create-new-value-stream into master

What does this MR do?

Solves #222337 (closed)

With this MR we will be able to create multiple value streams for a group (till now only one value stream per group possible).

The are changes in the relation: from group -> group stage to group -> group value stream -> group stage

We are supporting "old" route: /groups/gitlab-org/-/analytics/value_stream_analytics/stages stays for now for all stages of a group.

As well as new routes: /groups/gitlab-org/-/analytics/value_stream_analytics/value_streams for value streams and /groups/gitlab-org/-/analytics/value_stream_analytics/value_streams/[value_stream_id]/stages for stages of specific value stream.

For Database review:

  1. CreateAnalyticsCycleAnalyticsGroupValueStreams
== 20200624142107 CreateAnalyticsCycleAnalyticsGroupValueStreams: migrating ===
-- table_exists?(:analytics_cycle_analytics_group_value_streams)
   -> 0.0006s
-- create_table(:analytics_cycle_analytics_group_value_streams)
   -> 0.0082s
-- add_index(:analytics_cycle_analytics_group_value_streams, [:group_id, :name], {:unique=>true, :name=>"index_analytics_ca_group_value_streams_on_group_id_and_name"})
   -> 0.0020s
-- transaction_open?()
   -> 0.0000s
-- execute("ALTER TABLE analytics_cycle_analytics_group_value_streams\nADD CONSTRAINT check_bc1ed5f1f7\nCHECK ( char_length(name) <= 255 )\nNOT VALID;\n")
   -> 0.0003s
-- execute("ALTER TABLE analytics_cycle_analytics_group_value_streams VALIDATE CONSTRAINT check_bc1ed5f1f7;")
   -> 0.0004s
== 20200624142107 CreateAnalyticsCycleAnalyticsGroupValueStreams: migrated (0.0153s)

gitlab:create-new-value-stream+$ rails db:rollback                                      ~/gitlab/gitlab-development-kit/gitlab
== 20200624142107 CreateAnalyticsCycleAnalyticsGroupValueStreams: reverting ===
-- drop_table(:analytics_cycle_analytics_group_value_streams)
   -> 0.0017s
== 20200624142107 CreateAnalyticsCycleAnalyticsGroupValueStreams: reverted (0.0018s)
  1. AddGroupValueStreamToCycleAnalyticsGroupStages
gitlab:create-new-value-stream+?$ rails db:migrate VERSION=20200624142207                                                 ~/gitlab/gitlab-development-kit/gitlab
== 20200624142207 AddGroupValueStreamToCycleAnalyticsGroupStages: migrating ===
-- add_column(:analytics_cycle_analytics_group_stages, :group_value_stream_id, :bigint)
   -> 0.0008s
== 20200624142207 AddGroupValueStreamToCycleAnalyticsGroupStages: migrated (0.0065s)

== 20200624142207 AddGroupValueStreamToCycleAnalyticsGroupStages: reverting ===
-- remove_column(:analytics_cycle_analytics_group_stages, :group_value_stream_id)
   -> 0.0009s
== 20200624142207 AddGroupValueStreamToCycleAnalyticsGroupStages: reverted (0.0072s)
  1. AddNotValidForeignKeyToCycleAnalyticsGroupStages
== 20200701064756 AddNotValidForeignKeyToCycleAnalyticsGroupStages: migrating =
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:analytics_cycle_analytics_group_stages, :group_value_stream_id, {:name=>"index_analytics_ca_group_stages_on_value_stream_id", :algorithm=>:concurrently})
   -> 0.0018s
-- add_index(:analytics_cycle_analytics_group_stages, :group_value_stream_id, {:name=>"index_analytics_ca_group_stages_on_value_stream_id", :algorithm=>:concurrently})
   -> 0.0020s
-- add_foreign_key(:analytics_cycle_analytics_group_stages, :analytics_cycle_analytics_group_value_streams, {:column=>:group_value_stream_id, :name=>"fk_analytics_cycle_analytics_group_stages_group_value_stream_id", :on_delete=>:cascade, :validate=>false})
   -> 0.0007s
== 20200701064756 AddNotValidForeignKeyToCycleAnalyticsGroupStages: migrated (0.0049s)

gitlab:create-new-value-stream+?$ rails db:rollback                                                                       ~/gitlab/gitlab-development-kit/gitlab
== 20200701064756 AddNotValidForeignKeyToCycleAnalyticsGroupStages: reverting =
-- foreign_keys(:analytics_cycle_analytics_group_stages)
   -> 0.0030s
-- remove_foreign_key(:analytics_cycle_analytics_group_stages, {:column=>:group_value_stream_id, :name=>"fk_analytics_cycle_analytics_group_stages_group_value_stream_id"})
   -> 0.0030s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:analytics_cycle_analytics_group_stages, :group_value_stream_id, {:algorithm=>:concurrently})
   -> 0.0027s
-- remove_index(:analytics_cycle_analytics_group_stages, {:algorithm=>:concurrently, :column=>:group_value_stream_id})
   -> 0.0061s
== 20200701064756 AddNotValidForeignKeyToCycleAnalyticsGroupStages: reverted (0.0153s)
  1. AddDefaultValueStreamToGroupsWithGroupStages
== 20200701070435 AddDefaultValueStreamToGroupsWithGroupStages: migrating =====
SELECT DISTINCT "namespaces".* FROM "namespaces" INNER JOIN "analytics_cycle_analytics_group_stages" ON "analytics_cycle_analytics_group_stages"."group_id" = "namespaces"."id" WHERE "namespaces"."type" = 'Group'
-- change_column_null(:analytics_cycle_analytics_group_stages, :group_id, true)
   -> 0.0007s
== 20200701070435 AddDefaultValueStreamToGroupsWithGroupStages: migrated (0.1284s)

== 20200701070435 AddDefaultValueStreamToGroupsWithGroupStages: reverting =====
-- change_column_null(:analytics_cycle_analytics_group_stages, :group_id, false)
   -> 0.0006s
SELECT "analytics_cycle_analytics_group_value_streams".* FROM "analytics_cycle_analytics_group_value_streams" WHERE "analytics_cycle_analytics_group_value_streams"."name" = 'default'
== 20200701070435 AddDefaultValueStreamToGroupsWithGroupStages: reverted (0.0299s)
  1. ValidateForeignKeyOnCycleAnalyticsGroupStages
gitlab:create-new-value-stream+$ rails db:migrate                                                                         ~/gitlab/gitlab-development-kit/gitlab
== 20200701091253 ValidateForeignKeyOnCycleAnalyticsGroupStages: migrating ====
-- foreign_keys(:analytics_cycle_analytics_group_stages)
   -> 0.0029s
-- execute("ALTER TABLE analytics_cycle_analytics_group_stages VALIDATE CONSTRAINT fk_analytics_cycle_analytics_group_stages_group_value_stream_id;")
   -> 0.0057s
== 20200701091253 ValidateForeignKeyOnCycleAnalyticsGroupStages: migrated (0.0088s)

gitlab:create-new-value-stream+$ rails db:rollback                                                                      ~/gitlab/gitlab-development-kit/gitlab
== 20200701091253 ValidateForeignKeyOnCycleAnalyticsGroupStages: reverting ====
-- foreign_keys(:analytics_cycle_analytics_group_stages)
   -> 0.0031s
-- remove_foreign_key(:analytics_cycle_analytics_group_stages, {:column=>:group_value_stream_id, :name=>"fk_analytics_cycle_analytics_group_stages_group_value_stream_id"})
   -> 0.0025s
-- add_foreign_key(:analytics_cycle_analytics_group_stages, :analytics_cycle_analytics_group_value_streams, {:column=>:group_value_stream_id, :name=>"fk_analytics_cycle_analytics_group_stages_group_value_stream_id", :on_delete=>:cascade, :validate=>false})
   -> 0.0008s
== 20200701091253 ValidateForeignKeyOnCycleAnalyticsGroupStages: reverted (0.0065s)

Queries:

SELECT DISTINCT
    "namespaces".*
FROM
    "namespaces"
    INNER JOIN "analytics_cycle_analytics_group_stages" ON "analytics_cycle_analytics_group_stages"."group_id" = "namespaces"."id"
WHERE
    "namespaces"."type" = 'Group'

Plan:

Unique  (cost=2014.89..2028.41 rows=115 width=331) (actual time=442.228..443.709 rows=68 loops=1)
   Buffers: shared hit=2218 read=197 dirtied=3
   I/O Timings: read=418.781
   ->  Sort  (cost=2014.89..2015.18 rows=115 width=331) (actual time=442.224..442.302 rows=588 loops=1)
         Sort Key: namespaces.id, namespaces.name, namespaces.path, namespaces.owner_id, namespaces.created_at, namespaces.updated_at, namespaces.description, namespaces.avatar, namespaces.membership_lock, namespaces.share_with_group_lock, namespaces.visibility_level, namespaces.request_access_enabled, namespaces.ldap_sync_status, namespaces.ldap_sync_error, namespaces.ldap_sync_last_update_at, namespaces.ldap_sync_last_successful_update_at, namespaces.ldap_sync_last_sync_at, namespaces.lfs_enabled, namespaces.description_html, namespaces.parent_id, namespaces.shared_runners_minutes_limit, namespaces.repository_size_limit, namespaces.require_two_factor_authentication, namespaces.two_factor_grace_period, namespaces.cached_markdown_version, namespaces.project_creation_level, namespaces.runners_token, namespaces.file_template_project_id, namespaces.saml_discovery_token, namespaces.runners_token_encrypted, namespaces.custom_project_templates_group_id, namespaces.auto_devops_enabled, namespaces.extra_shared_runners_minutes_limit, namespaces.last_ci_minutes_notification_at, namespaces.last_ci_minutes_usage_notification_level, namespaces.subgroup_creation_level, namespaces.emails_disabled, namespaces.max_pages_size, namespaces.max_artifacts_size, namespaces.mentions_disabled, namespaces.default_branch_protection, namespaces.unlock_membership_to_ldap, namespaces.max_personal_access_token_lifetime, namespaces.push_rule_id, namespaces.shared_runners_enabled, namespaces.allow_descendants_override_disabled_shared_runners
         Sort Method: quicksort  Memory: 233kB
         Buffers: shared hit=2218 read=197 dirtied=3
         I/O Timings: read=418.781
         ->  Nested Loop  (cost=0.71..2010.96 rows=115 width=331) (actual time=9.547..437.218 rows=588 loops=1)
               Buffers: shared hit=2195 read=197 dirtied=3
               I/O Timings: read=418.781
               ->  Index Only Scan using index_analytics_ca_group_stages_on_group_id on public.analytics_cycle_analytics_group_stages  (cost=0.28..16.46 rows=579 width=8) (actual time=2.834..18.825 rows=588 loops=1)
                     Heap Fetches: 179
                     Buffers: shared hit=29 read=8 dirtied=2
                     I/O Timings: read=9.890
               ->  Index Scan using namespaces_pkey on public.namespaces  (cost=0.43..3.44 rows=1 width=331) (actual time=0.709..0.709 rows=1 loops=588)
                     Index Cond: (namespaces.id = analytics_cycle_analytics_group_stages.group_id)
                     Filter: ((namespaces.type)::text = 'Group'::text)
                     Rows Removed by Filter: 0
                     Buffers: shared hit=2166 read=189 dirtied=1
                     I/O Timings: read=408.891

Summary:

Time: 444.749 ms
  - planning: 0.817 ms
  - execution: 443.932 ms
    - I/O read: 418.781 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 2218 (~17.30 MiB) from the buffer pool
  - reads: 197 (~1.50 MiB) from the OS file cache, including disk I/O
  - dirtied: 3 (~24.00 KiB)
  - writes: 0
SELECT
    "analytics_cycle_analytics_group_value_streams".*
FROM
    "analytics_cycle_analytics_group_value_streams"
WHERE
    "analytics_cycle_analytics_group_value_streams"."name" = 'default'

Plan - not possible to run in #database-lab channel:

ERROR: failed to run explain without execution: ERROR: relation "analytics_cycle_analytics_group_value_streams" does not exist (SQLSTATE 42P01)

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Magdalena Frankiewicz

Merge request reports