Skip to content

Fix JSON error that occurs when X-Ray config file contains NULL byte

Leaminn Ma requested to merge fix-json-postgres-null-byte-unicode-error into master

What does this MR do and why?

We are rolling out the feature flag ai_enable_internal_repository_xray_service in #483928 (closed). It's at 25% roll out as of 2024-10-09 6:46PM PT. The feature is a new in-monolith service that scans a project's repository for dependency config files and then parses a list of libraries from the file content. This service is executed by a Sidekiq job that is triggered upon a new commit to a Duo-enabled project's default branch.

While monitoring the logs, we noticed an error occurring in one of the projects. See kibana logs: https://log.gprd.gitlab.net/app/r/s/heZdD It appears to be due to a unique case where the customer's dependency file contains an invalid byte sequence. This causes the job fail. See #483928 (comment 2150918354) for details.

The error occurs during .upsert_all where the JSON module is utilized. Since JSON doesn't support the NULL byte, it raises the following error:

Screenshot_2024-10-09_at_6.57.31_PM

This MR fixes the error above by sanitizing the file content and removing NULL bytes prior to parsing.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Error reproduced locally with new spec test Spec test passes after content is sanitized
Screenshot_2024-10-09_at_6.59.31_PM Screenshot_2024-10-09_at_7.00.47_PM

Related to #483928 (closed)

Edited by Leaminn Ma

Merge request reports

Loading