Skip to content

Error on newlines in sidekiq-cluster arguments

Sean McGivern requested to merge fix-sidekiq-cluster-dryrun-newlines into master

When using Helm charts, which are YAML files, it's relatively easy to accidentally include a newline in the Sidekiq arguments. This will typically be a trailing newline, which shouldn't be a problem, except that in our cloud-native deployment we launch Sidekiq using the command given by sidekiq-cluster --dryrun (to avoid having the wrapper process), so we effectively do:

$(bin/sidekiq-cluster --dryrun $ARGS)

(See https://gitlab.com/gitlab-org/build/CNG/-/blob/68c9feba11e1f37aea0d837cc61868e16445bb4b/gitlab-sidekiq/scripts/process-wrapper#L46-57.)

Ruby's Shellwords.escape quotes newlines as '\n': https://github.com/ruby/shellwords/blob/v0.1.0/lib/shellwords.rb#L161-L163

When those are parsed back in the arguments array we get foo'\n' for an original argument of foo\n. Then, when we apply splitting, we get foo' and ' as two separate arguments.

The quoting here is quite complicated but the below demonstrates (in Bash) what's happening:

$ ruby -e 'p ARGV; p ARGV.first.split' "foo'"$'\n'"'"
["foo'\n'"]
["foo'", "'"]

If we did care about newlines in this argument array, the solution would be more involved. But we don't, because they are never valid, so we can just error when we encounter them.

This came from https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/14002.

Edited by Sean McGivern

Merge request reports

Loading