Here is what nobody asks: does that number still mean what we think it means?
Most businesses are running on data that was accurate two years ago, before the workaround, before the field got repurposed, before the integration started silently failing. The dashboard still updates, so everyone assumes it is fine. It usually is not.
The four ways data quietly goes wrong
Data does not corrupt all at once. It drifts, and each type of drift has a pattern.
1. Definition drift
A field meant one thing when it was created, then someone needed to track something adjacent, and the easiest place to put it was the existing field. Now the same column means two things depending on context, and every query has to know which context it is looking at.
The classic version: a status field with values like pending, approved, completed — then someone adds pending_review and approved_with_exceptions. Six months later, half your reports filter for approved and miss the exceptions. The other half include them. Nobody agrees on the actual approval rate because the field now encodes two concepts.
How to spot it: Look for fields with more than five distinct values, or values that include separators (underscores, hyphens, slashes). Both are symptoms that a single field is doing multiple jobs.
2. Process drift
The data model reflects a process that no longer happens the way it used to. The business adapted — a step got skipped, a handoff moved, a approval layer was added — but the system did not, so people work around it.
Example: a CRM where deals are supposed to move through five stages, but sales learned that stage three always gets approved, so they skip straight from two to four. The system still has stage three, reports still measure time-in-stage-three, and that metric is now always zero. Nobody looks at it anymore, but it is still on the executive dashboard.
How to spot it: Find metrics that never change, or fields that are always the same value. Then ask someone who does the work daily whether that step still happens.
3. Integration drift
Two systems used to sync cleanly. Then one of them changed — a field was renamed, a API endpoint deprecated, a required parameter added — and the integration kept running but started failing silently.
The failure mode is specific: the sync process does not error, it just stops moving certain records. New customers sync fine, but updates to existing ones do not. Or it works for records created after a certain date, and everything before that is frozen. The symptom is data that is accurate for some things and subtly stale for others, which is worse than comprehensively broken because it is harder to notice.
How to spot it: Compare record counts and last-updated timestamps across systems that should match. A growing discrepancy means something is not syncing.
4. Assumption drift
The data is fine. The metric is wrong.
A report was built on an assumption — \"we close deals in 30 days on average\" — and the logic reflects that. Then the market changed, deal cycles lengthened, and now the metric excludes everything over 45 days as an outlier. The number on the dashboard is still labelled \"average deal cycle\", but it is actually \"average deal cycle for deals that close fast\", which is not the same thing.
How to spot it: Any metric with a filter or exclusion that was added after the original report. Ask what it is filtering out, and whether that exclusion still makes sense.
The audit that actually finds this
The usual data audit checks for missing values and duplicates. Those matter, but they are not where the lies are. The lies are in the fields that look fine but mean something different than they used to.
Here is the audit that surfaces it:
- List your top ten decisions. The ones with budget attached. Hiring plans, inventory orders, marketing spend, pricing changes.
- For each decision, name the number that drove it. Revenue per region, cost per acquisition, inventory turn, whatever it was.
- Trace that number back to the query. Not the dashboard — the actual SQL or report definition that produces it.
- Read the query with someone who does the work. Not the analyst who wrote it — the person in operations or sales who creates the records. Ask them to explain what each field actually represents today.
You will find drift in at least three of the ten. The question is whether any of those three influenced a decision that is now hard to reverse.
The fix is not always technical
Sometimes the drift is in the process and the data is just reflecting it honestly. A field that contains eight different concepts is a problem, but the problem is not the field — it is that your process has eight edge cases and nobody decided how to handle them consistently.
Cleaning the data without fixing the process just means it will drift again in six months. The sequence that sticks:
- Find the drift (the audit above).
- Decide what the field should mean going forward.
- Update the process so people can actually record it that way.
- Migrate the old data to match, or mark it clearly as legacy.
- Update every report that touches it.
Step three is the one that gets skipped. If the system cannot express the thing people need to track, they will find a workaround, and that workaround will become the next drift.
When to do this
Three moments when a data audit is worth the time:
- Before a big decision. If you are about to commit six months of engineering effort or a material budget based on a metric, spend two days checking whether that metric still reflects reality.
- Before a consolidation. Migrating bad data into a new system just means the new system is authoritative about the wrong thing. The sample migration pass mentioned in the consolidation article is where you find this.
- When a report stops making sense. If the numbers feel wrong, or two reports that should agree do not, that is not a bug in the query — it is a symptom that the data model and the reality have diverged.
What good data reliability looks like
It is not zero drift — that is unrealistic. Businesses change, and data models lag. What matters is knowing where the drift is.
A reliable data system is one where:
- Every field has a documented definition, and that definition is reviewed when the process changes.
- Integrations log their failures visibly, not silently.
- Metrics include their assumptions and exclusions in the label, not just in the SQL.
- Someone is responsible for noticing when a dashboard number stops changing.
That last one is the hardest to assign, because it sits between data and operations and most org charts do not have a box for it. But if nobody owns it, it does not happen, and you are back to making decisions on numbers that look authoritative because they came from a system.
The most dangerous data is not missing — it is present, formatted, and wrong.
If you are about to consolidate systems or migrate data and want a second opinion on where the drift might be hiding, that is a reasonable thing to cover in a first call. Book one, or read the FAQ first if you want to know what it involves.
]]>