Skip to content

Convert PO files to JS in node (14x faster)

Lukas Eipert requested to merge leipert-po-is-a-river-in-italy into master

What does this MR do and why?

We are currently using the gettext_i18n_rails_js gem to convert our translations from PO files to JavaScript. The process takes about 28 seconds on my machine. This is particularly painful during gdk update and in CI where it is run every time we compile our assets.

With NodeJS we can compile the resulting file about twice as fast. If we make use of parallel execution with the help of the battle-tested jest-worker, we can bring down the execution time to just about two seconds, which is at least a 14x improvement and takes it from "what is this doing" to barely noticing it.

Bringing this process into the Node world also will allow us:

  1. To potentially drop gettext_i18n_rails_js gem, as this is basically the only thing that was being used from the gem
  2. Iterate on the resulting format and our usage of the jed library, as there is potential for size gains and performance gains.
  3. Potentially fix some bugs. The conversion seems to escape a few characters which do not seem to need escapeing or where the escapes seem to be plain wrong.

For now none of these things have been changed and we focus on parity of the results of the two processes. In order to verify the results, please check out this commit and run bash scripts/frontend/po_to_json_spec.sh

Screenshots or screen recordings

N/A

How to set up and validate locally

Pinning test

  1. Check out 4cb85bf073f61350e2553fd24ed910b90ed2a97c, the commit with the pinning tests.
  2. Run yarn install
  3. Run bash ./scripts/frontend/po_to_json_spec.sh
  4. The tests generates the files the "old" and the "new" way and will compare them with a jest integration test

To check it end-to-end in the browser

  1. Go to the latest commit on the branch
  2. Remove compiled app.js files rm -f app/assets/javascripts/locale/**/app.js
  3. Run node scripts/frontend/po_to_json.js
  4. Start GDK and use a different language in the user profile settings (e.g. German or French)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Lukas Eipert

Merge request reports