Direct Transfer - Change the logic to detect stale trackers
BulkImports::FinishBatchedPipelineWorker fails all BulkImports::BatchTracker
associated to a tracker if the tracker is considered stale. To determine if a tracker is stale, the formula tracker.created_at < 4.hours.ago
is used. And because all trackers are created when the entity migration starts, all batch migrations must be completed within 4 hours; otherwise, the migration fails.
Problem
Having a hardcoded period is problematic because the time could not be enough for large projects.
Proposed solution
As mentioned on #428110 (comment 1620770037), for batched migration, it's fine to hardcode a period because a batches process the same number of records no matter the size of the project.
So, to fix the described problem, we should remove the tracker.stale?
condition and update FinishBatchedPipelineWorker
worker to fail batches that didn't finish in a specific time.