WIP: Set Filebeat to use daily indices
What does this MR do?
After the most recent ELK version upgrade, indices changed from daily to a different heuristic as defined by ILM by default.
Related issue: #219643 (comment 359787092)
Current scenario:
-
GKE setup on 2020.06.09
-
On 2020.06.12 those are the available indices:
green open filebeat-7.6.2-2020.06.09-000001 Ku7WxTnwS1SprDBTiPfCYw 1 1 671424 0 1017.9mb 541.3mb
green open ilm-history-1-000001 z0vT1PgbRRSXaBX85UE2nQ 1 1 18 0 50.6kb 25.3kb
For any data range which doesn't not include 2020.06.09, the search (index based) will return empty.
for example:
While this one would accidentally return the 'right' results:
#input {"from":"2020-06-09T02:56:23.293Z","to":"2020-06-12T02:56:23.293Z"}
(from.to_date..to.to_date).map { |day| puts "filebeat-*-#{day.strftime('%Y.%m.%d')}-*"}
filebeat-*-2020.06.09-*
filebeat-*-2020.06.10-*
filebeat-*-2020.06.11-*
filebeat-*-2020.06.12-*
This one would return nothing (because 2020.06.09 contains all the data)
#input {"from":"2020-06-11T02:56:55.586Z","to":"2020-06-12T02:56:55.586Z"}
(from.to_date..to.to_date).map { |day| puts "filebeat-*-#{day.strftime('%Y.%m.%d')}-*"}
filebeat-*-2020.06.11-*
filebeat-*-2020.06.12-*
After additional testing, I will be closing this MR as the proposed changes still do not fix the issue. More investigation will be required.
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
Edited by Zamir Martins