Remove issue project joins for global scoped searches
The issue data stored in Elasticsearch will now contain access_level
field to determine permissions.
This should allow global scoped searches to use the access_level
field instead of doing a join on the parent field.
We expect this to reduce the complexity and improve performance.
Example globally scoped issues Elasticsearch query generated
{
"query": {
"bool": {
"must": [
{
"simple_query_string": {
"_name": "issue:match:search_terms",
"fields": [
"title^2",
"description"
],
"query": "test",
"default_operator": "and"
}
}
],
"filter": [
{
"term": {
"type": {
"_name": "doc:is_a:issue",
"value": "issue"
}
}
},
{
"has_parent": {
"_name": "issue:authorized:project",
"parent_type": "project",
"query": {
"bool": {
"should": [
{
"bool": {
"filter": [
{
"term": {
"visibility_level": {
"_name": "issue:authorized:project:any",
"value": 0
}
}
},
{
"terms": {
"_name": "issue:authorized:project:issues:enabled_or_private",
"issues_access_level": [
20,
10
]
}
}
]
}
},
{
"bool": {
"_name": "issue:authorized:project:visibility:10:issues:access_level",
"filter": [
{
"term": {
"visibility_level": {
"_name": "issue:authorized:project:visibility:10",
"value": 10
}
}
},
{
"terms": {
"_name": "issue:authorized:project:visibility:10:issues:access_level:enabled_or_private",
"issues_access_level": [
20,
10
]
}
}
]
}
},
{
"bool": {
"_name": "issue:authorized:project:visibility:20:issues:access_level",
"filter": [
{
"term": {
"visibility_level": {
"_name": "issue:authorized:project:visibility:20",
"value": 20
}
}
},
{
"terms": {
"_name": "issue:authorized:project:visibility:20:issues:access_level:enabled_or_private",
"issues_access_level": [
20,
10
]
}
}
]
}
}
]
}
}
}
}
]
}
},
"highlight": {
"fields": {
"title": {},
"description": {}
},
"number_of_fragments": 0,
"pre_tags": [
"gitlabelasticsearch→"
],
"post_tags": [
"←gitlabelasticsearch"
]
}
}
Epic order
- Remove joins from the Elasticsearch query for project/group scoped issues search
- De-normalize the issue permission data. <-- will include migration and can be done in multiple MRs
- Remove joins from the Elasticsearch query for globally scoped issues search
- Migration to copy issue data to new index
- Migration to remove issue data from combined index **
- Rake task changes to support new index **
- Backend changes for supporting new indexes during Zero Downtime Re-indexing **
- Admin UI changes to support new index (any time after new index created/data migrated to it)
** must be released in conjunction with the Migration to copy issue data to new index
Edited by Terri Chu