Hybrid issue search results in Advanced Search
Problem to solve
Search results in Global Search are currently only keyword matched and by leveraging newly introduced Elastic AI tooling, we can combine semantic and keyword matches to provide accurate results but also broaden the result context.
A bonus is that groupglobal search can introduce an AI-powered feature which other teams can follow and leverage for introducing other embedding features.
Proposal
Show search results that originate from semantic search in Global Search. Start with issue search and rollout to other scopes in follow-up iterations.
How to get there
- Upgrade Elastic to 8.12+ if models are being stored in the cluster
- Generating issue embeddings in rails using an embedding model
- Combine title and description into one chunk so that the text contains maximum context
- Store embeddings for selected scopes and fields in Elastic as part of the indexing process
- Follow guidance on using the HNSW in tact to keep latency minimal
- Add a query to combine kNN and keyword search with reranking behind a feature flag
Details
The embedding model chosen by Duo and that will be used here is Vertex textembedding-gecko
. It has 768 dimensions and max tokens of 3,072 per request. https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings. The rate limit is 1500 requests per minute in the Iowa region. https://cloud.google.com/vertex-ai/generative-ai/docs/quotas. The QPM applies to every model so if we expect more requests for generating embeddings, the total should be below the limit.