Add clients for Elasticsearch to ActiveContext gem
What does this MR do and why?
The ActiveContext gem deals with interactions with vector stores. This MR allows configuring Elasticsearch as a vector store by setting up a client to connect with Elasticsearch.
Changes in this MR:
- Adapter class
- Client class:
- the config comes from an initializer which can be
::Gitlab::CurrentSettings.elasticsearch_config
to match the current connection options for Elasticsearch. - at the moment it only has a search method which finds all documents. This will later be changed to accept an index and body.
- the config comes from an initializer which can be
- QueryResult class
- Adds
'elasticsearch'
gem as a dependency for the ActiveContext gem. The elasticsearch gem is already in use in the monolith.
Note: this does not support OpenSearch connections which is coming in a separate MR. The reason for keeping it separate is that we want to remove the dependency of the elasticsearch gem for using opensearch so that the gem can be upgraded in the future.
References
Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
- Add an initializer in the monolith
config/initializers/active_context.rb
# frozen_string_literal: true
ActiveContext.configure do |config|
config.databases = {
es1: {
adapter: 'ActiveContext::Databases::Elasticsearch::Adapter',
options: ::Gitlab::CurrentSettings.elasticsearch_config
}
}
end
bundle install
- Start a rails console
-
ActiveContext.adapter
should be anActiveContext::Databases::Elasticsearch::Adapter
-
ActiveContext.adapter.client.client.ping
should resolve -
ActiveContext.adapter.search(nil).count
should return the number of docs in your cluster -
ActiveContext.adapter.search(nil).first
should return the first document in your cluster
Related to #507433