Skip to content

Remove upper index size limit for reindexing

Andreas Brandl requested to merge ab/reindexing-limits into master

What does this MR do and why?

This change:

  1. Removes the upper size limit for indexes for automatic reindexing, which previously was set at 100 GB.
  2. Increases the timeout for a reindexing operation from 9 hours to 24 hours.

We've previously limited the size as we didn't want to automatically perform reindexing on very large btrees. At the time we didn't have indexes this large, now we have.

Large data migrations can significantly increase the bloat level and total size for indexes as seen in gitlab-org/database-team/team-tasks#208 (closed). We actually want to rebuild those indexes automatically, so we can reduce their size again.

As for the timeout increase: Large GIN indexes take a lot longer than 9 hours to rebuild. The example we've seen recently is the GIN index on notes, which ran into the 9 hours limit. We'll retry rebuilding this index in gitlab-org/database-team/team-tasks#212 (closed), after having increased the timeout.

For more background see #345134 (closed).

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Andreas Brandl

Merge request reports