Add client for OpenSearch to ActiveContext gem
What does this MR do and why?
The ActiveContext gem deals with interactions with vector stores. This MR allows configuring OpenSearch as a vector store by setting up a client to connect with OpenSearch.
Changes in this MR:
- Adapter class
- Client class:
- the config comes from an initializer which can be
::Gitlab::CurrentSettings.elasticsearch_config
to match the current connection options for OpenSearch. It mirrors the current implementation in the monolith for opensearch to get AWS credentials. - at the moment it only has a search method which finds all documents. This will later be changed to accept an index and body.
- the config comes from an initializer which can be
- QueryResult class
- Adds
'opensearch-ruby'
gem as a dependency for the ActiveContext gem. - Adds
'aws-sdk-core'
and'faraday_middleware-aws-sigv4'
as development dependencies for the ActiveContext gem. This is needed for tests, unless there's another way.
References
Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
- Update your
::Gitlab::CurrentSettings.elasticsearch_config
to point to an OpenSearch cluster on AWS. - Add an initializer in the monolith
config/initializers/active_context.rb
# frozen_string_literal: true
ActiveContext.configure do |config|
config.databases = {
os1: {
adapter: 'ActiveContext::Databases::OpenSearch::Adapter',
options: ::Gitlab::CurrentSettings.elasticsearch_config
}
}
end
bundle install
- Start a rails console
-
ActiveContext.adapter
should be anActiveContext::Databases::Opensearch::Adapter
-
ActiveContext.adapter.client.client.ping
should resolve -
ActiveContext.adapter.search(nil).count
should return the number of docs in your cluster -
ActiveContext.adapter.search(nil).first
should return the first document in your cluster - [Optional] Update your
::Gitlab::CurrentSettings.elasticsearch_config
to point to a docker version of opensearch without credentials and repeat.
Related to #507433
Edited by Madelein van Niekerk