AN EXPERIMENT IN DATA RELIABILITY
Break it.
Then make it better.
Good pipelines expect imperfect inputs. This one gives you the controls. Run a batch, inspect what failed, and see why a safe retry matters.
01 / CHOOSE A SCENARIO
Two events rename amount_cents to total_cents. The contract should catch them.
READY WHEN YOU ARESourceContractWarehouse
| Event ID | Amount · cents | Latest result |
|---|---|---|
| evt_001 | 12500 | waiting |
| evt_002 | 8900 | waiting |
| evt_003 | field missing | waiting |
| evt_004 | 5600 | waiting |
| evt_005 | field missing | waiting |
| evt_006 | 3200 | waiting |
—Inserted this run
—Duplicates skipped
—Quarantined
0/6Warehouse events
›What happens when the upstream schema changes? Run the pipeline to find out.
A small model of validation and idempotency, built for exploration.
01 / VALIDATION
Protect the boundary.
A required field disappears or arrives with the wrong type. Quarantine that event with a reason, while valid events continue.
02 / IDEMPOTENCY
Make repetition safe.
The event ID is the key. An identical replay is skipped; a conflicting payload is quarantined. Existing events stay unchanged.
03 / RECOVERY
Close the loop.
Correct the contract and replay the original batch. Only the missing events are added. The final six events total $720.00.