Log extra metadata about searches
What does this MR do?
Logging these extra fields as top level keys will ensure that we can query/filter them in Kibana which will help us to answer questions about common search terms or understand usage of search within individual groups. Currently these query params are already logged but they are nested keys they are not indexed and cannot be filtered in any way in Kibana.
This change adds to the existing logic in ApplicationController
which in term relies on the append_info_to_payload
being invoked after a request. This metadata is finally extracted and logged by our lograge configuration.
Screenshots
{"method":"GET","path":"/search","format":"*/*","controller":"SearchController","action":"show","status":200,"unpermitted_params":["utf8","search_code","nav_source","utf8","search_code","nav_source","utf8","search_code","nav_source","utf8","search_code","nav_source","utf8","search_code","nav_source","utf8","search_code","nav_source","utf8","search_code","nav_source","utf8","search_code","nav_source"],"time":"2020-07-28T05:18:54.081Z","params":[{"key":"utf8","value":"✓"},{"key":"search","value":"hello world"},{"key":"group_id","value":"456"},{"key":"project_id","value":"13"},{"key":"scope","value":""},{"key":"search_code","value":"true"},{"key":"snippets","value":"false"},{"key":"repository_ref","value":"master"},{"key":"nav_source","value":"navbar"}],"remote_ip":"127.0.0.1","user_id":null,"username":null,"ua":"curl/7.64.1","correlation_id":"pGc23kDjaia","meta.project":"root/wget2","meta.root_namespace":"root","meta.caller_id":"SearchController#show","meta.search.group_id":"456","meta.search.project_id":"13","meta.search.search":"hello world","gitaly_calls":5,"gitaly_duration_s":0.204777,"redis_calls":18,"redis_duration_s":0.034875,"redis_read_bytes":2041,"redis_write_bytes":90337,"redis_cache_calls":16,"redis_cache_duration_s":0.028589,"redis_cache_read_bytes":2039,"redis_cache_write_bytes":90290,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006286,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":47,"queue_duration_s":0.288697,"cpu_s":3.05,"db_duration_s":0.10424,"view_duration_s":1.87052,"duration_s":3.29513,"db_count":12,"db_write_count":0,"db_cached_count":3}
Does this MR meet the acceptance criteria?
Conformity
- [-] Changelog entry
- [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Closes #227884 (closed)