Do not escape double quote during po to json
relate #417128
I found one issue related i18n, when compile po to json in gitlab:assets:compile
, it will escape double quote msgid in po file, which will cause app.js NOT match with original msgid. So all msgid contains double quote ARE not working currently.
Behavior
let's say we have a i18n template
considerFlexibleRollout: s__(
'FeatureFlags|Consider using the more flexible "Percent rollout" strategy instead.',
),
after extract task, po file will be like below, msgid is in double quote, so \"
means "
, the msgid is same with js file, which is correct
msgid "FeatureFlags|Consider using the more flexible \"Percent rollout\" strategy instead."
msgstr ""
however convert po to json, the app.js is
"FeatureFlags|Consider using the more flexible \\\"Percent rollout\\\" strategy instead.":["请考虑使用更灵活的“百分比上线”策略。"]
please pay attention, the string in app.js is FeatureFlags|Consider using the more flexible \"Percent rollout\" strategy instead.
, which add extra \
in msgid, mismatch with js file, break the i18n functionality.
Analyze
there are two ways to translate po to json, one is po_to_json
gem, another is gitlab/scripts/frontend/po_to_json.js
written by nodejs.
At first, the bug was introduce in po_to_json
gem, line 2 in method push_buffer
, value.gsub(/\"/, """)
will try to replace \"
to "
, which should not escape double quote, however gsub method didn't change value
variable, so it should be value = value.gsub
or value.gsub!
(I will create pr to fix this gem)
def push_buffer(value, key = nil)
value = ::Regexp.last_match(1) if value =~ /^"(.*)"/
value.gsub(/\\"/, "\"")
if key.nil?
buffer[lastkey] = [
buffer[lastkey],
value
].join("")
else
buffer[key] = value
@lastkey = key
end
end
Currently, gitlab/scripts/frontend/po_to_json.js
is introduced at 2023.03, to replace po_to_json
task for better performance, and funny thing is, we reproduce the bug logic in the task, just to make sure app.js generated by old way and new way are identical, and we know it may be a bug. read the following code in po_to_json.js
, we escape msgid which change "
to \"
, which is wrong.
/**
* TODO: This replacer might be unnecessary _or_ even cause bugs.
* due to potential unnecessary double escaping.
* But for now it is here to ensure that the old and new output
* are equivalent.
* @param str
* @returns {string}
*/
function escapeMsgid(str) {
return `${str}`.replace(/([\\"])/g, '\\$1');
}
Action
- since we use
po_to_json.js
instead ofpo_to_json
gem, so currently we can only focus onpo_to_json.js
, we should remove the function escapeMsgid, then msgid will be correct. - I will create PR to
po_to_json
gem