Add Elastic Stack cluster integration
What does this MR do?
Allows GitLab to continue querying Elasticsearch for pod logs. The main idea is
for existing integrations to continue working without user intervention once we
stop using the cluster_applications_elastic_stack
table in %14.0.
In this first iteration, newly created integrations are expected to expose pod
logs in a certain format over http on a service called
elastic-stack-elasticsearch-master
, on port 9200
, in the Kubernetes namespace gitlab-managed-apps
.
See #326560 (closed)
Screenshots (strongly suggested)
Form | State sync | Logs |
---|---|---|
multiple_forms | state_sync_720p | logs |
Edit: Changed approach to multiple forms. Updated only the "Form" recording to save time, as only that part of the functionality was affected.
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. -
I have not included a changelog entry because _____.
-
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides - [-] Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec
- [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Migration output
➜ gitlab git:(add-elasticstack-cluster-integration) ✗ bin/rails db:migrate
== 20210504153354 CreateClustersIntegrationElasticstack: migrating ============
-- create_table(:clusters_integration_elasticstack, {:id=>false})
-- quote_column_name(:chart_version)
-> 0.0000s
-> 0.0075s
-- quote_table_name("check_f8d671ce04")
-> 0.0000s
-- quote_table_name(:clusters_integration_elasticstack)
-> 0.0000s
-- execute("ALTER TABLE \"clusters_integration_elasticstack\"\nADD CONSTRAINT \"check_f8d671ce04\" CHECK (char_length(\"chart_version\") <= 10)\n")
-> 0.0009s
== 20210504153354 CreateClustersIntegrationElasticstack: migrated (0.0162s) ===
➜ gitlab git:(add-elasticstack-cluster-integration) ✗ bin/rails db:rollback
== 20210504153354 CreateClustersIntegrationElasticstack: reverting ============
-- drop_table(:clusters_integration_elasticstack, {:id=>false})
-> 0.0020s
== 20210504153354 CreateClustersIntegrationElasticstack: reverted (0.0105s) ===