Skip to content

Add client for OpenSearch to ActiveContext gem

What does this MR do and why?

The ActiveContext gem deals with interactions with vector stores. This MR allows configuring OpenSearch as a vector store by setting up a client to connect with OpenSearch.

Changes in this MR:

  • Adapter class
  • Client class:
    • the config comes from an initializer which can be ::Gitlab::CurrentSettings.elasticsearch_config to match the current connection options for OpenSearch. It mirrors the current implementation in the monolith for opensearch to get AWS credentials.
    • at the moment it only has a search method which finds all documents. This will later be changed to accept an index and body.
  • QueryResult class
  • Adds 'opensearch-ruby' gem as a dependency for the ActiveContext gem.
  • Adds 'aws-sdk-core' and 'faraday_middleware-aws-sigv4' as development dependencies for the ActiveContext gem. This is needed for tests, unless there's another way.

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

  • Update your ::Gitlab::CurrentSettings.elasticsearch_config to point to an OpenSearch cluster on AWS.
  • Add an initializer in the monolith config/initializers/active_context.rb
# frozen_string_literal: true

ActiveContext.configure do |config|
  config.databases = {
    os1: {
      adapter: 'ActiveContext::Databases::OpenSearch::Adapter',
      options: ::Gitlab::CurrentSettings.elasticsearch_config
    }
  }
end
  • bundle install
  • Start a rails console
  • ActiveContext.adapter should be an ActiveContext::Databases::Opensearch::Adapter
  • ActiveContext.adapter.client.client.ping should resolve
  • ActiveContext.adapter.search(nil).count should return the number of docs in your cluster
  • ActiveContext.adapter.search(nil).first should return the first document in your cluster
  • [Optional] Update your ::Gitlab::CurrentSettings.elasticsearch_config to point to a docker version of opensearch without credentials and repeat.

Related to #507433

Edited by Madelein van Niekerk

Merge request reports

Loading