Skip to content

Add clients for Elasticsearch to ActiveContext gem

Madelein van Niekerk requested to merge 507433-search-clients into master

What does this MR do and why?

The ActiveContext gem deals with interactions with vector stores. This MR allows configuring Elasticsearch as a vector store by setting up a client to connect with Elasticsearch.

Changes in this MR:

  • Adapter class
  • Client class:
    • the config comes from an initializer which can be ::Gitlab::CurrentSettings.elasticsearch_config to match the current connection options for Elasticsearch.
    • at the moment it only has a search method which finds all documents. This will later be changed to accept an index and body.
  • QueryResult class
  • Adds 'elasticsearch' gem as a dependency for the ActiveContext gem. The elasticsearch gem is already in use in the monolith.

Note: this does not support OpenSearch connections which is coming in a separate MR. The reason for keeping it separate is that we want to remove the dependency of the elasticsearch gem for using opensearch so that the gem can be upgraded in the future.

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

  • Add an initializer in the monolith config/initializers/active_context.rb
# frozen_string_literal: true

ActiveContext.configure do |config|
  config.databases = {
    es1: {
      adapter: 'ActiveContext::Databases::Elasticsearch::Adapter',
      options: ::Gitlab::CurrentSettings.elasticsearch_config
    }
  }
end
  • bundle install
  • Start a rails console
  • ActiveContext.adapter should be an ActiveContext::Databases::Elasticsearch::Adapter
  • ActiveContext.adapter.client.client.ping should resolve
  • ActiveContext.adapter.search(nil).count should return the number of docs in your cluster
  • ActiveContext.adapter.search(nil).first should return the first document in your cluster

Related to #507433

Edited by Madelein van Niekerk

Merge request reports

Loading