SharedModel.using_connection doesn't override connection_db_config

Summary

When used within SharedModel.using_connection, connection_db_config still returns the original connection database config, and not the one for the overriding connection.

This leads to incorrect lease keys generated, for example in https://gitlab.com/gitlab-org/gitlab/-/blob/caf64fec980ce1161226e88142da735d3048eb59/lib/gitlab/database/reindexing/coordinator.rb#L77.

Here are all usages of connection_db_config as of now:

$ git grep --line-number "connection_db_config.name"
lib/gitlab/database.rb:52:    PRIMARY_DATABASE_NAME = ActiveRecord::Base.connection_db_config.name.to_sym # rubocop:disable Database/MultipleDatabases
lib/gitlab/database/async_indexes/index_creator.rb:51:          [super, async_index.connection_db_config.name].join('/')
lib/gitlab/database/async_indexes/index_destructor.rb:57:          [super, async_index.connection_db_config.name].join('/')
lib/gitlab/database/count/reltuples_count_strategy.rb:60:          models.group_by { |model| model.connection_db_config.name }.map do |db_name, models_for_db|
lib/gitlab/database/each_database.rb:64:          connection_name = model.connection_db_config.name
lib/gitlab/database/load_balancing/configuration.rb:66:          @model.connection_db_config.name.to_sym
lib/gitlab/database/partitioning.rb:41:                if connection_name != model.connection_db_config.name
lib/gitlab/database/reindexing/coordinator.rb:77:          [super, index.connection_db_config.name].join('/')
lib/gitlab/metrics/samplers/database_sampler.rb:67:            db_config_name: klass.connection_db_config.name
spec/lib/gitlab/database/each_database_spec.rb:132:        allow(main_model).to receive_message_chain('connection_db_config.name').and_return('main')
spec/lib/gitlab/database/each_database_spec.rb:133:        allow(ci_model).to receive_message_chain('connection_db_config.name').and_return('ci')
spec/lib/gitlab/database/each_database_spec.rb:154:        allow(main_model).to receive_message_chain('connection_db_config.name').and_return('main')
spec/lib/gitlab/database/each_database_spec.rb:155:        allow(ci_model).to receive_message_chain('connection_db_config.name').and_return('ci')
spec/support/db_cleaner.rb:50:      puts "The #{result['table']} (#{connection_class.connection_db_config.name}) table has #{result['column_count']} columns."

Steps to reproduce

Gitlab::Database::SharedModel.using_connection(Ci::ApplicationRecord.connection) do
  index = Gitlab::Database::Reindexing::QueuedAction.last.index
  
  puts "Current database: #{index.connection.execute('select current_database()').to_a[0]['current_database']}"
  puts "Index connection name: #{index.connection_db_config.name}" # <-- should be `ci`, but is `main`

  config = Gitlab::Database.db_config_for_connection(index.connection)
  puts "Config for connection name: #{config.name}" # <-- correctly returns `ci`
end

What is the current bug behavior?

When used within using_connection, connection_db_config still returns the original connection config.

What is the expected correct behavior?

It should return the overriding connection config.

Possible fixes

Short term we can replace all usage with Gitlab::Database.db_config_for_connection(index.connection).name.

Long term we may change using_connection to override the database config too.

Use native AR multiple database support for Git... (#352333) is related too.