Overview
Data work is not complete when code runs. It is complete when source assumptions, output contracts, and quality checks prove the behavior is correct.
When to Use
- new source ingestion
- schema changes
- transformation logic updates
- new or changed published tables
- bug fixes involving bad data or broken metrics
Do not use this only as a final cleanup step. It should guide implementation from the start.
Workflow
-
Define the contract before implementation. Capture:
- required fields
- key constraints
- expected types
- allowed null behavior
- freshness expectations
- reconciliation rules
-
Write the validation plan first. Common validations:
- uniqueness
- non-null thresholds
- referential integrity
- accepted values
- row count deltas
- source-to-target totals
-
Reproduce data bugs with a failing check. If an incident or defect exists, write the failing validation or test before changing the pipeline.
-
Implement the smallest change that satisfies the contract.
-
Run the relevant validations and capture evidence. Evidence may include:
- test output
- query results
- sample reconciliation output
- dry-run logs
Common Rationalizations
| Rationalization | Reality |
|---|---|
| "The warehouse query looks right." | Visual inspection does not scale and misses edge cases. |
| "We will add checks after the model stabilizes." | Unchecked pipelines create low-trust data and harder incident response. |
| "A successful job means the data is valid." | Jobs succeed while still producing incorrect or incomplete data. |
Red Flags
- no contract exists for a published dataset
- an incident fix ships without a failing reproduction check
- only happy-path sample data is validated
- freshness or completeness expectations are absent
Verification
- Contracts are written before or alongside implementation
- Relevant checks exist for correctness, completeness, and freshness
- Defects are reproduced with a failing validation before the fix
- Evidence from validation is captured and reviewable