Skip to content

Validating llm logs

Gosia Ksionek requested to merge mk-prevent-logging-interactions into master

What does this MR do and why?

In order to make sure we have all data we need in Kibana, and this data is easy to explore and filter, we need to make sure data is always passed as the same type. This MR adds soft-validation on params passed to llm.log. We check if the param is defined (if not, the error is raised) and if the param passed has the correct type. Type mismatch is not causing error, but the special log on warn level is added. This way we can spot and fix mismatching errors and in the future also guard this by raising error. As at this moment that can have unpredicted consequences, I opted for "soft" approach.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. To any place we are adding llm-related logs, add any new param and observe error when this part of the code is run.
  2. In any place we are adding llm-related logs, change type of param (for example: from string to integer), run code and observe warn log being added to llm.log.

solves #495391 (closed) #495399 (closed)

Edited by Shinya Maeda

Merge request reports

Loading