Skip to content

Force gitaly-ruby to use UTF-8 encoding

Patrick Steinhardt requested to merge pks-ruby-utf8-encoding into master

When starting the gitaly-ruby sidecar, from the main Go server, we always set the "LANG=en_US.UTF-8" environment variable. The purpose of this is that we always want Git to output error messages in English, and not the native language the Gitaly server is running on. The downside though is that on some servers configured to use a different language, the locale may not be available at all. While Git itself will know how to handle this just fine by falling back to the C or POSIX locales, Ruby doesn't as it will use ASCII as external encoding. As a result, it will raise an encoding error as soon as it encounters any non-ASCII character in spawned commands.

There's multiple ways to approach this issue:

- Stop fiddling with the locale in Go code. This is probably nothing
  we want to do, as it would mean that Git's error messages would be
  in the machines native language and thus harder to understand and
  debug for us.

- Try a range of locales in Go and forward the first available
  language, e.g. by checking for en_US.UTF-8, C.UTF-8 and finally
  POSIX.UTF-8. While this would still ensure English error messages,
  the downside is that a system may not have any of these available.
  In that case, we'd have to use a non-UTF-8 locale, causing Ruby to
  choke on the encoding again.

So fiddling with the locale won't help us in all cases, which is why we should just keep it the way it is and continue passing LANG=en_US.UTF-8. Instead, let's just force Ruby itself to use UTF-8 as encoding by setting the default external encoding to UTF-8 on startup.

Fixes #2852 (closed).

Unfortunately, I found it quite hard to either reproduce the error or write any tests for this. @wchandler, if you've got any hint to that regard I'd be happy to test.

Edited by GitLab Release Tools Bot

Merge request reports