Skip to content

Use alter table when deleting records

Adam Hegyi requested to merge 428260-use-alter-table-when-deleting-records into master

What does this MR do and why?

When ensuring consistency in ClickHouse, for the EventAuthorsConsistencyCronWorker we used lightweight deletes: https://clickhouse.com/docs/en/guides/developer/lightweight-delete

Unfortunately, the record marking phase of these delete queries are executed synchronously which fails the background job (network timeout) for very large tables. The only option to deal with this is to use ALTER TABLE DELETE ... statement which executes the whole deletion asynchronously.

The deletion caused Net::ReadTimeout with #<TCPSocket:(closed)> exception on PRD.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #428260 (closed)

Edited by Adam Hegyi

Merge request reports