Force convert invalid ASCII 8bit diffs and use a replacement ASCII character
What does this MR do and why?
As described in #207124 (closed), invalid characters in an ASCII-8bit encoded diff were causing the entire diff to not load. In order to at least mitigate that portion of it, this MR will determine if there is an invalid character in the diff and replace it with a generic ASCII snowman in the text.
Since the actual diff highlighting uses the file blob, the ascii snowman is really only seen in the text
field in diffs_batch.json
"highlighted_diff_lines": [
{
"line_code": "f9abdc73eba4e926645a8761ea178b0ca630a28b_1_1",
"type": null,
"old_line": 1,
"new_line": 1,
"text": " ☃☃#language fr-FR \"☃\"\r",
"meta_data": null,
"rich_text": " <span id=\"LC1\" class=\"line\" lang=\"plaintext\"><feff>#language fr-FR \"è\"</span>\n",
"can_receive_suggestion": true
},
The text
field doesn't seem to be used in this situation at all, so I think it's a definite improvement over the current functionality of a complete diff failure :)
I'm introducing a feature flag so we can cautiously enable it and check any other unforseen consequences.
Screenshots or screen recordings
These are strongly recommended to assist reviewers and reduce the time to merge your change.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #207124 (closed)