Data & decisions · 15 October 2022
Data exploration — why the first hour with the data decides the whole analysis
Four datasets, identical statistics, four completely different stories. The case for looking before calculating, in one famous picture.
Every failed analysis I've ever audited failed early — in the first hour, when nobody looked at the data before calculating with it.
Chapter 1 · Data & decisions
The famous proof that looking matters
A statistician named Frank Anscombe built four small datasets with a mischievous property: their summary statistics are practically identical. Same averages. Same spread. Same correlation. Feed them into the standard formulas and you get the same trend line, — in plain words, "start at 3 and add half a unit of y for each unit of x."
Then you plot them. One is a clean straight-line relationship. One is a smooth curve — a completely different story. One is a perfect line ruined by a single rogue point. And in one, there's no relationship at all — a single extreme point manufactures the whole trend by itself.
Four identical sets of statistics. Four different truths. Only the pictures can tell them apart. That's the entire argument for exploration, in one image.
Chapter 2 · Data & decisions
What the first hour should cover
- Shape. Draw the histogram of every number that matters. Waiting times are almost never symmetric — and when the data is lopsided, the average and the "typical case" part company. (The average gets dragged by extremes; the median — the middle value — doesn't.)
- Impossibilities. Negative durations. Dates from 1970. The same category spelt three ways. Every real dataset contains at least one impossibility; find it before it finds your conclusion.
- Missing data, with a reason. When data is missing because of what it would have said — abandoned calls that never logged a duration — everything you calculate from what remains is biased. Ask why each gap exists.
- Time order. A calm-looking histogram can hide a steady trend or a sudden step. Plot the values in the order they happened before trusting any summary.
Chapter 3 · Data & decisions
The point
Exploration isn't the warm-up before the analysis. It's the part of the analysis where the data gets a chance to object. Skip it, and every impressive formula downstream calculates confidently on assumptions nobody checked.
Talk this through with me
A free 30-minute discovery call — no commitment, no jargon.
Book a discovery call