A nightly batch job processes 10,000 order records. 9,997 succeed. Three fail — one duplicate key, one malformed field, one downstream rate limit hit at the wrong moment. On a platform that only understands retrying at the level of “the job,” the recovery options are limited: rerun the whole batch, or leave three records permanently unresolved.
Neither is a good answer. Rerunning 10,000 records to fix three creates its own risk if any of the 9,997 successful records aren’t safely reprocessable — duplicate writes, double-counted totals, a downstream system that doesn’t expect to see the same record twice. Leaving three records unresolved means a customer order silently never completes, discovered only when someone notices later, if anyone does.
The unit of recovery should match the unit of failure. When a job of 10,000 fails because of 3 records, the job didn’t fail — 3 records did. Recovery that can’t see that distinction pays the cost of the whole batch every time.
The most common failure isn’t a broken integration — it’s a broken record inside one that’s working
It’s tempting to think of integration failures as “the integration is down” events. In practice, the far more common failure mode is a working integration processing a batch where a small number of records don’t fit the expected shape: a duplicate key, a field that exceeds a length limit, a reference to a record that hasn’t been created yet on the destination side.
The integration itself isn’t broken. The mapping is correct, the connection is healthy, the vast majority of the batch processes exactly as intended. One row is the problem, and it’s buried inside a run that otherwise looks completely successful.
Platforms without record-level visibility handle this the same way regardless of scale: the job either succeeds or fails as a unit, and one bad record can flip a 10,000-record run to “failed” even though 9,999 of them are fine.
Rerunning the whole batch is expensive in three different ways
The naive fix — rerun the entire job — has a cost that scales with batch size, and that cost shows up in three places. First, compute and time: reprocessing 10,000 records to fix 3 is 3,333 times more work than the fix actually requires, and at scale that’s not a rounding error.
Second, risk to records that already succeeded. If the destination system doesn’t reject records it’s already seen, a full rerun creates duplicates: a customer billed twice, an inventory count adjusted twice, a record inserted twice into a system with no dedup logic of its own. “Just rerun it” assumes the successful records are safely repeatable, and that assumption doesn’t always hold.
Third, delay. A large batch rerun takes real time to complete, and during that window the three records that actually needed fixing are still unresolved, now waiting behind 9,997 records that didn’t need reprocessing in the first place.
Reprocessing 10,000 records to fix 3 isn’t caution. It’s paying the cost of the whole batch for a problem that lives in a tenth of a percent of it.
Record-level recovery treats the row as the unit of work
Record-level recovery isolates the failed row: correct the source data or the mapping, retry that specific record, leave the other 9,999 untouched. The distinction that matters is that the retry targets the actual unit of failure, not the container it happened to fail inside of.
This requires the platform to track failure at record granularity in the first place — which record, which field, which error — not just “this job run had a nonzero failure count.” Without that, record-level recovery isn’t possible even in principle; the information needed to isolate the fix doesn’t exist.
A platform that can only tell you the job failed can only offer you the job as the unit of recovery.
Bulk remediation is the other half of the same capability
Record-level precision solves the single-bad-record case. It doesn’t solve the case where 300 records across several batches failed for the same underlying reason — a schema change on the destination side, a rate limit that triggered repeatedly across a high-volume window. Retrying those one at a time defeats the purpose of having record-level granularity in the first place.
Bulk fallout remediation applies a single recovery action across a queue of similarly-failed records at once — same error classification, same fix, one operation instead of 300 individual retries. Precision and throughput here aren’t a tradeoff: record-level accuracy in identifying what failed, bulk-level efficiency in resolving the records that failed the same way.
| Run-level only | Record-level + bulk | |
|---|---|---|
| 3 bad records in a 10,000-row batch | Rerun all 10,000, or leave 3 unresolved | Retry the 3, leave 9,997 untouched |
| 300 records fail the same way across batches | 300 individual manual fixes | One remediation action applied to the queue |
| Risk to already-successful records | Rerun risks duplicates on reprocessable systems | Zero — untouched records stay untouched |
Where a full rerun is still the right call
Record-level recovery isn’t a universal replacement for reprocessing. If the failure traces back to something systemic — a mapping bug that affected every record, a connection that was down for the entire run — the fix belongs at the integration level, and a full rerun after the fix is the correct response. Record-level recovery is the right tool when the integration is healthy and a minority of records don’t fit; it’s the wrong tool when the integration itself is what broke.
The judgment call is diagnosing which situation you’re in before picking a recovery path, not defaulting to one or the other regardless of what actually failed.
Precision as a design decision, not an edge case
Koodisi Engage tracks failures at the record level by default and pairs that with bulk remediation for cases where many records share one root cause. That’s a deliberate stance: most platforms in this category treat record-level recovery as a manual afterthought layered on top of run-level retry, rather than the default unit of recovery.
If a single bad record in your integrations currently means rerunning the whole batch, or means writing custom scripts to isolate and fix it, it’s worth testing what record-level recovery looks like against your own data on Koodisi’s Community tier.