Skip to content

Add query analyzers framework and multiple partition scan detector

Matt Kasa requested to merge 427810-identify-multiple-partition-scan-queries into master

What does this MR do and why?

This adds a framework for additional query analyzers that use our auto_explain logs to allow us to analyze unique queries executed by our test suites.

The first analyzer is a multiple partition scan detector meant to help identify queries on partitioned tables that do not correctly use the partition key to avoid scanning multiple partitions.

Relates to #427810 (closed)

How to set up and validate locally

  1. Check out this MR branch locally
  2. Download a set of auto_explain artifacts from a merged result pipeline
  3. Run scripts/merge-auto-explain-logs: 
    env RSPEC_AUTO_EXPLAIN_LOG_PATH=path/to/auto_explain/auto_explain.ndjson.gz ./scripts/merge-auto-explain-logs
  4. Inspect the contents of p_ci_builds_multiple_partition_scans.ndjson.gz

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #427810 (closed)

Merge request reports