Configure rails db_extra_config_command attribute
What does this MR do?
Configure gitlab_rails['db_extra_config_command'] in gitlab.rb
which is used to specify a script to
retrieve rails database password versus keeping the unencrypted password in the gitlab.rb
file or in a disk file.
The contents of this attribute will be added to the rails database.yml
file like so:
The gitlab.rb
:
gitlab_rails['db_extra_config_command'] = '/opt/database-config.sh'
Resulting entry in database.yml
production:
config_command: '/opt/database-config.sh'
main:
.
.
.
There is no modification to how we generate default passwords for gitlab-rails['db_password'] as
gitlab_rails['db_extra_config_command']` may be used for other than just credentials.
Related issues
- Related #6274 (closed)
- gitlab!141818 (merged)
Checklist
See Definition of done.
For anything in this list which will not be completed, please provide a reason in the MR discussion.
Required
-
MR title and description are up to date, accurate, and descriptive. -
MR targeting the appropriate branch. -
Latest Merge Result pipeline is green. -
When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.
For GitLab team members
If you don't have access to this, the reviewer should trigger these jobs for you during the review process.
-
The manual Trigger:ee-package
jobs have a green pipeline running against latest commit. -
If config/software
orconfig/patches
directories are changed, make sure thebuild-package-on-all-os
job within theTrigger:ee-package
downstream pipeline succeeded. -
If you are changing anything SSL related, then the Trigger:package:fips
manual job within theTrigger:ee-package
downstream pipeline must succeed. -
If CI configuration is changed, the branch must be pushed to dev.gitlab.org
to confirm regular branch builds aren't broken.
Expected (please provide an explanation if not completing)
-
Test plan indicating conditions for success has been posted and passes. -
Documentation created/updated. -
Tests added. -
Integration tests added to GitLab QA. -
Equivalent MR/issue for the GitLab Chart opened. -
Validate potential values for new configuration settings. Formats such as integer 10
, duration10s
, URIscheme://user:passwd@host:port
may require quotation or other special handling when rendered in a template and written to a configuration file.
Test plan
-
Green pipeline including triggering Trigger::ee-package
. Pipeline at https://gitlab.com/gitlab-org/omnibus-gitlab/-/pipelines/1142077455 fails QAdecomposition-multiple-db
jobs. Do our QA tests need to be modified to handle the newdatabase.yml
attribute? -
Install and reconfigure using database defaults. Check for config_command: nil
indatabase.yml
file:docker compose file
version: '3.8' name: 'gitlab-test' x-default_settings: &default_settings image: "${GITLAB_IMAGE:-registry.gitlab.com/gitlab-org/omnibus-gitlab/gitlab-ee:configure-rails-extra-db-config-command}" pull_policy: always privileged: true shm_size: '256m' restart: always services: postgresql: <<: *default_settings image: postgres:14 environment: - POSTGRES_USER=gitlab - POSTGRES_PASSWORD=secret_password - POSTGRES_DB=gitlabhq_production ports: - '5432:5432' networks: default: ipv4_address: 10.0.0.2 healthcheck: test: pg_isready -U gitlab -d gitlabhq_production interval: 10s timeout: 3s retries: 3 gitlab: <<: *default_settings environment: GITLAB_OMNIBUS_CONFIG: | external_url "http://10.0.0.3" gitlab_rails['db_database'] = "gitlabhq_production" gitlab_rails['db_username'] = "gitlab" gitlab_rails['db_password'] = "secret_password" gitlab_rails['db_host'] = '10.0.0.2' # Disable PG postgresql['enable'] = false ports: - "80:80" - "443:443" - "22:22" networks: default: ipv4_address: 10.0.0.3 healthcheck: disable: true depends_on: postgresql: condition: service_healthy networks: default: ipam: config: - subnet: 10.0.0.0/24
-
Gitlab container comes up with no issue. $ docker exec -it gitlab-test-gitlab-1 gitlab-ctl status run: alertmanager: (pid 1529) 408s; run: log: (pid 1363) 423s run: gitaly: (pid 1479) 410s; run: log: (pid 575) 500s run: gitlab-exporter: (pid 1489) 409s; run: log: (pid 815) 439s run: gitlab-kas: (pid 597) 498s; run: log: (pid 620) 495s run: gitlab-workhorse: (pid 1412) 410s; run: log: (pid 778) 448s run: logrotate: (pid 498) 516s; run: log: (pid 506) 515s run: nginx: (pid 1448) 410s; run: log: (pid 791) 446s run: prometheus: (pid 1498) 409s; run: log: (pid 1342) 427s run: puma: (pid 650) 461s; run: log: (pid 657) 460s run: redis: (pid 514) 510s; run: log: (pid 523) 509s run: redis-exporter: (pid 1491) 409s; run: log: (pid 1104) 433s run: sidekiq: (pid 669) 455s; run: log: (pid 689) 454s run: sshd: (pid 37) 526s; run: log: (pid 36) 526s
-
Correct config_command
entry indatabase.yml
file.$ docker exec -it gitlab-test-gitlab-1 head /opt/gitlab/embedded/service/gitlab-rails/config/database.yml # This file is managed by gitlab-ctl. Manual changes will be # erased! To change the contents below, edit /etc/gitlab/gitlab.rb # and run `sudo gitlab-ctl reconfigure`. production: config_command: nil main: adapter: postgresql encoding: unicode collation:
-
-
Set gitlab_rails['db_extra_config_command
]-
Bring up new docker compose using the above docker compose file. -
Create configure command at /opt/database-config.sh
in thegitlab-test-gitlab-1
container using the following content:#!/bin/bash cat << EOF --- main: password: 'secret_password' EOF
-
Set permissions on the configuration command: chmod 755 /opt/database-config.sh
-
Edit the /etc/gitlab/gitlab.rb
file and set:gitlab_rails['db_password'] = nil gitlab_rails['db_extra_config_command'] = '/opt/database-config.sh'
-
Run gitlab-ctl reconfigure
and check status$ gitlab-ctl reconfigure $ gitlab-ctl status ok: run: alertmanager: (pid 3602) 1s ok: run: gitaly: (pid 3617) 0s ok: run: gitlab-exporter: (pid 3653) 0s ok: run: gitlab-kas: (pid 3663) 0s ok: run: gitlab-workhorse: (pid 3678) 1s ok: run: logrotate: (pid 3697) 0s ok: run: nginx: (pid 3703) 1s ok: run: prometheus: (pid 3735) 0s ok: run: puma: (pid 3763) 1s ok: run: redis: (pid 3768) 0s ok: run: redis-exporter: (pid 3777) 1s ok: run: sidekiq: (pid 3795) 0s ok: run: sshd: (pid 3803) 1s curl http://10.0.0.3 <html><body>You are being <a href="http://10.0.0.3/users/sign_in">redirected</a>.</body></html>
-
Check contents of database.yml
to check ifconfig_command
is populated with the script path.cat /opt/gitlab/embedded/service/gitlab-rails/config/database.yml | head -n 8 # This file is managed by gitlab-ctl. Manual changes will be # erased! To change the contents below, edit /etc/gitlab/gitlab.rb # and run `sudo gitlab-ctl reconfigure`. production: config_command: "/opt/database-config.sh" main: adapter: postgresql
-
Remove old passwords from database.yml
to make sure it is actually using the command.$ sed -ri -e 's/.*password\:.*/ password:/' /opt/gitlab/embedded/service/gitlab-rails/config/database.yml
-
Restart gitlab and check operation. $ gitlab-ctl restart $ ok: run: alertmanager: (pid 4542) 0s ok: run: gitaly: (pid 4559) 0s ok: run: gitlab-exporter: (pid 4595) 0s ok: run: gitlab-kas: (pid 4604) 1s ok: run: gitlab-workhorse: (pid 4619) 0s ok: run: logrotate: (pid 4639) 0s ok: run: nginx: (pid 4645) 1s ok: run: prometheus: (pid 4677) 0s ok: run: puma: (pid 4705) 0s ok: run: redis: (pid 4710) 1s ok: run: redis-exporter: (pid 4719) 0s ok: run: sidekiq: (pid 4733) 0s ok: run: sshd: (pid 4744) 0s $ curl http://10.0.0.3 <html><body>You are being <a href="http://10.0.0.3/users/sign_in">redirected</a>.</body></html>
-