Technical Research on Elasticsearch 7.x

Elasticsearch 7.0.0 was released in April of 2019 by Elastic. Currently GitLab supports Elasticsearch 5.6 and 6.x, but will be dropping support for 5.6 with the release of GitLab 12.7.

GitLab needs to investigate what our options are for supporting Elasticsearch 7.x. Things to think about here:

• Can we run 6.x and 7.x at the same time?
• How would the upgrade to 7.x work?
• Can we ship two versions of the indexer?
  • Would we provide an option to switch/select?
• Do we need two paths in the rails code base?

There may be other areas of interest to investigate, but the intent of this issue is to determine what options may exist.

TODO

• Get the elasticsearch-rails, elasticsearch-model and elasticsearch-api gems all running the v6 version to confirm they're working since they are quite old
• See whether we can create index on both v6 and v7 with the same gem using bundle exec rake gitlab:elastic:index.
  • (Probably creating the index will reveal most of the incompatibilities we have since querying doesn't seem to use many features and the join stuff we're using seems to already be the correct way to join in v7.)
• Update CI for gitlab-elasticsearch-indexer to test with ES v7 https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer/blob/master/.gitlab-ci.yml
• Upgrade Elasticsearch version to test against ES v6 in https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml#L216 then we'll need to to add also a test for v7
• Run same QA steps as in !22645 (merged)