Add embeddings to issues if on Elasticsearch 8+
What does this MR do and why?
Creates a new Elastic migration which adds a vector field to store embeddings in the issues index.
This migration is skippable. It will only be executed if the platform is elasticsearch and the version is 8+.
As per this thread, the initial implementation will not used quantized vectors but once this is available on the AI Gateway, we'll need another migration to update the mappings.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Supported platform and version
curl http://localhost:9200
{
...
"cluster_name" : "elasticsearch",
"version" : {
"number" : "8.11.4",
Elastic::MigrationWorker.new.perform
MigrationWorker: migration[AddEmbeddingToIssues] executing migrate method
[Elastic::Migration: 20240410193847] Adding {:embedding=\u003e{:type=\u003e\"dense_vector\", :dims=\u003e768, :similarity=\u003e\"dot_product\", :index=\u003etrue}} to gitlab-development-issues mapping
MigrationWorker: migration[AddEmbeddingToIssues] updating with completed: true
MigrationWorker: no migration available
Unsupported platform and version
curl http://localhost:9200
{
...
"cluster_name" : "elasticsearch",
"version" : {
"number" : "7.17.20"
...
Elastic::MigrationWorker.new.perform
MigrationWorker: no migration available
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
Related to #454251 (closed)
Edited by Madelein van Niekerk