Use bundler to install Omnibus gems

What does this MR do?

Instead of calling gem install <x> many times, let's define the Gemfile and Gemfile.lock and run bundle install. This ensures dependencies are pinned and don't quietly move with a new invocation of gem install. There is also a side benefit to using bundler: we can later use the newly-introduced feature to include and verify checksums (https://github.com/rubygems/rubygems/pull/6374).

bundler generally installs everything in the expected gem directory that is used by ruby, but for gems installed from a Git source, bundler puts those gems in a special bundler/gems directory: https://github.com/rubygems/rubygems/blob/5712b5229dc5845a163d1806070503ad64b076a9/bundler/lib/bundler/plugin/installer/git.rb#L19. Since we don't use any gems from Git sources now--gitlab-ruby-shadow is now used instead of ruby-shadow--there is no issue.

Also, note that if we want to ensure gitlab-ctl doesn't try to pull in gems from GitLab Rails, we could invoke bundler, but this is a slightly riskier change that we can consider later:

diff --git a/config/software/gitlab-ctl.rb b/config/software/gitlab-ctl.rb
index 025c164d5..a508ea11e 100644
--- a/config/software/gitlab-ctl.rb
+++ b/config/software/gitlab-ctl.rb
@@ -73,7 +73,8 @@ if [ "$1" == "reconfigure" ] && [ "$UID" != "0" ]; then
   exit 1
 fi

-#{install_dir}/embedded/bin/omnibus-ctl #{File.basename(install_dir)} '#{install_dir}/embedded/service/omnibus-ctl*' "$@"
+export BUNDLE_GEMFILE=#{install_dir}/embedded/service/omnibus-gitlab/Gemfile
+#{install_dir}/embedded/bin/ruby #{install_dir}/embedded/bin/bundle exec #{install_dir}/embedded/bin/omnibus-ctl #{File.basename(install_dir)} '#{install_dir}/embedded/service/omnibus-ctl*' "$@"
       EOH
     end
   end
stanhu@jet-arm omni

Relates to #8368

Related issues

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

• MR title and description are up to date, accurate, and descriptive.
• MR targeting the appropriate branch.
• Latest Merge Result pipeline is green.
• When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

• The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
• If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
• If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
• If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes.
• Documentation created/updated.
• Tests added.
• Integration tests added to GitLab QA.
• Equivalent MR/issue for the GitLab Chart opened.
• Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.