Skip to content

Enable cross-database query check

Dylan Griffith requested to merge build-allow-list-for-ci-cross-joins into master

Hello developers,

If you can encounter this CrossJoinAcrossUnsupportedTablesError exception in your specs, see the docs on how to remove the join https://docs.gitlab.com/ee/development/database/multiple_databases.html#removing-joins-between-ci_-and-non-ci_-tables

Additionally, for feature specs, you may need to check log/test.log if asynchronous requests fail with 500 to see if you can spot CrossJoinAcrossUnsupportedTablesError errors.


What does this MR do?

Enable cross-database query check, while allowing existing cross-database queries to pass. For any new SQL queries that cross databases, specs will now raise a CrossJoinAcrossUnsupportedTablesError exception

Related issue : #337077 (closed)

The allowlist was generated from CI data with this patch:

  config.around do |example|
    example.run
    capybara_error = Capybara.current_session && Capybara.current_session.server && Capybara.current_session.server.error
    has_capybara_error = capybara_error && capybara_error.message.include?("Unsupported cross-join")
    has_exception = example.exception && example.exception.message.include?("Unsupported cross-join")
    if Thread.current[:has_cross_join_exception] || has_exception || has_capybara_error
      # example.location contains location and lineno
      File.open(Rails.root.join("log/ci-cross-join-allowlist.log"), 'a') { |f| f.puts(example.location) }
    end
  end

We then download the CI artifacts using !69032 (closed), and:

mkdir allowlists
cd allowlists
../scripts/api/download_pipeline_artifact.rb -p 278964 --pipeline_id 357551673 -a log/ci-cross-join-allowlist.log
cat *.log | cut -d ":" -f 1 | sort | uniq | ruby -e 'require "yaml"; list=ARGF.read.split("\n"); puts YAML.dump(list)' > unique_files.yml

Screenshots or Screencasts (strongly suggested)

How to setup and validate locally (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Edited by Thong Kuah

Merge request reports

Loading