Add timeout status to bulk importer
- What does this MR do and why?
- Scope database review
- How to set up and validate locally
- MR acceptance checklist
What does this MR do and why?
- Adds a new status to
BulkImport
andBulkImports::Entity
to denote a timeout. - Adds a new scope;
stale
to both models which returns objects that are not complete, or failed, and have been created or running for over 1 day. - Adds a new daily cron worker to set
stale
objects to have timed out. - Bonus: Fixes documentation examples for troubleshooting steps and adds correct syntax highlighting.
Follow-up MRs are coming in %14.10 for other BulkImport
classes.
Scope database review
BulkImport
SELECT "bulk_imports".* FROM "bulk_imports" WHERE (created_at > '2022-03-20 11:08:19.041942') AND "bulk_imports"."status" IN (0, 1)
Seq Scan on public.bulk_imports (cost=0.00..102.88 rows=1 width=39) (actual time=8.346..8.347 rows=1 loops=1)
Filter: ((bulk_imports.created_at > '2022-03-20 11:08:19.041942+00'::timestamp with time zone) AND (bulk_imports.status = ANY ('{0,1}'::integer[])))
Rows Removed by Filter: 2061
Buffers: shared read=18 dirtied=5
I/O Timings: read=5.415 write=0.000
Time: 9.102 ms
- planning: 0.728 ms
- execution: 8.374 ms
- I/O read: 5.415 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 0 from the buffer pool
- reads: 18 (~144.00 KiB) from the OS file cache, including disk I/O
- dirtied: 5 (~40.00 KiB)
- writes: 0
BulkImports::Entity
SELECT "bulk_import_entities".* FROM "bulk_import_entities" WHERE (created_at < '2022-03-20 11:16:25.948137') AND "bulk_import_entities"."status" IN (0, 1)
Index Scan using index_bulk_import_entities_on_bulk_import_id_and_status on public.bulk_import_entities (cost=0.28..248.62 rows=88 width=258) (actual time=4.001..37.418 rows=68 loops=1)
Index Cond: (bulk_import_entities.status = ANY ('{0,1}'::integer[]))
Filter: (bulk_import_entities.created_at < '2022-03-20 11:16:25.948137+00'::timestamp with time zone)
Rows Removed by Filter: 1
Buffers: shared hit=42 read=58 dirtied=10
I/O Timings: read=34.659 write=0.000
Time: 38.211 ms
- planning: 0.745 ms
- execution: 37.466 ms
- I/O read: 34.659 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 42 (~336.00 KiB) from the buffer pool
- reads: 58 (~464.00 KiB) from the OS file cache, including disk I/O
- dirtied: 10 (~80.00 KiB)
- writes: 0
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #352985 (closed)
Edited by Max Woolf