Updates for support of ES 8 & OpenSearch
What does this Merge Request do and why?
There was a recent change to upgrade the rails dependency used to communicate with Elasticsearch. The new dependency throws warnings if Elasticsearch security is disabled because that would be a risk for production. However, this is the default behavior for Elasticsearch instances and what we want for gdk and development use. Also, it's pretty annoying when running rspec tests.
This MR changes:
-
disable security using the elasticsearch/config/elasticsearch.yml configuration file on
gdk install
andgdk reconfigure
- Note: if security is already set in the yml file, this will not override that
- updates documentation for how to run the now supported Elasticsearch v8.X and OpenSearch v1.X and removes mention of Elasticsearch 6.X which is not supported anymore
Log example
➜ bin/rspec ee/spec/workers/elastic/migration_worker_spec.rb
DEPRECATION WARNING: Database connection should not be called during initializers. Read more at https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers (called from connected? at ee/lib/gitlab/geo.rb:79)
Run options: include {:focus=>true}
All examples were filtered out; ignoring {:focus=>true}
Test environment set up in 9.235826 seconds
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
To test
I tested this in a working instance of gdk with Elasticsearch enabled and setup. You can run the spec mentioned above to verify the message happens. Not sure if there is a better way to test this out.
- look at the elasticsearch config file
cat ../elasticsearch/config/elasticsearch.yml
- run the rspec test and verify you see the warning messages:
bin/rspec ee/spec/workers/elastic/migration_worker_spec.rb
- replicate the two file changes in the working instance of gdk
- run the make command
support/elasticsearch-config
- look at the elasticsearch config file:
cat ../elasticsearch/config/elasticsearch.yml
- verify the config file has
xpack.security.enabled: false
at the end - restart elasticsearch
gdk restart elasticsearch
- run the rspec test and verify there are NO warning messages:
bin/rspec ee/spec/workers/elastic/migration_worker_spec.rb
- run the make command
support/elasticsearch-config
- look at the elasticsearch config file
cat ../elasticsearch/config/elasticsearch.yml
- verify the config file has only has one instance
xpack.security.enabled: false
at the end (we do not want to duplicate it every time reconfigure is run)
Merge Request checklist
-
This change is backward compatible. If not, please include steps to communicate to our users. -
Tests added for new functionality. If not, please raise Issue to follow-up. -
Documentation added/updated, if needed. -
gdk doctor
test added, if needed. -
Add the ~highlight
label if this MR should be included in theCHANGELOG.md
.
Edited by Terri Chu