How We KnowAnswer
Why Can We Read Mammoth DNA but Not Dinosaur DNA?
A frozen mammoth tooth can hold readable DNA. A dinosaur fossil can preserve the shape of a bone for far longer. Here’s why those are different feats.
THE QUESTION LIBRARY HOW WE KNOW
How can I tell if a number, chart or headline in the news is misleading?
A few concrete questions help you judge a striking number without dismissing useful evidence.
THE SHORT ANSWER
You can often spot a misleading claim by checking what was counted, the comparison, the chart scale and the uncertainty. Follow the number to its original source. A large percentage change can begin from a small amount. A graph can exaggerate proportions. Statistically significant does not mean large or important, and it does not settle a conclusion. These checks help you judge what the evidence supports.
- A percentage needs a clear denominator: the total it is a share of.
- Percentage changes and changes in percentage points are different comparisons.
- Chart scales can change how a difference looks without changing the underlying values.
- A p-value is calculated under assumptions and does not measure an effect’s importance.
About this picture
An illustration of hits recorded on aircraft that returned. A chart needs to make clear which observations were counted and which are missing; this drawing is not a complete aircraft damage survey.
WHERE THE EVIDENCE STANDS
How sure are we?
- Percentage changes and changes in percentage points express different comparisons.The ONS style guide explains the distinction; the fictional school example is independently checked arithmetic.
- Shortening a bar chart’s numerical scale can distort the proportions of its bars.The Government Analysis Function explains this effect and distinguishes bar charts from line charts.
- Statistical significance means an effect is large or important.The ASA statement explicitly separates statistical significance from effect size and importance.
- A p-value is the probability that a claim is correct.Greenland and colleagues explain that it is conditional on a model and its assumptions, rather than a hypothesis probability.
WHAT WE DON’T KNOW YET
An incomplete headline cannot tell us how the data were collected or which explanations were tested. A checklist may reveal missing information without settling the quality of a complex analysis.
WHAT WOULD CHANGE THIS ANSWER
Original data, clear counting rules and appropriate uncertainty can change our assessment of a particular story. Independent analysis may show that a numerical comparison was correct, incomplete or mistaken.
You can often spot a misleading number by asking what was counted, what it was compared with and what the report leaves out. You do not need to distrust every statistic. You need enough context to understand what the number can support.
Consider a made-up example. A school loses two library books out of every hundred loans one month and four out of every hundred the next. A headline could call that a 100% increase. The arithmetic is correct: four is twice two. It is also an increase of two books per hundred loans, or two percentage points. Both descriptions belong in the picture. Neither tells us why the change happened.
A graph adds another choice. How much of the vertical scale does it show? Are its labels clear? Does it compare the same kind of measurement over the same time span? The UK Government Analysis Function's chart guidance explains why a shortened scale can distort the proportions of bars. A line chart may reasonably focus on a narrow range, provided the scale is clear.
Some headlines add the words statistically significant. That phrase describes a calculation under assumptions. It does not mean the finding is large, important or beyond question. The American Statistical Association warns against using a single numerical threshold as a verdict.
A useful first response is to follow the number back to its source. Look for the original table, the measured group and the uncertainty around the estimate. Separate what the data show from the explanation offered for them. If that information is missing, you have found a reason to pause. You have not yet found evidence that the number was made up.
THE LONG ANSWER
THE LONG ANSWER · ABOUT 5 MINUTES
You’ve read your 3 free long answers this month.
The short answer, how sure we are, what we don’t know, what would change our mind and every source stay free on every page, for everyone.
All Access members read the long answer: the evidence piece by piece, the strongest objections, and how to check it yourself.
Join All Access · $1 a week Or $6 a month
Billed $52 once a year. Cancel anytime in your account.
Until December 4, 11:59 pm Pacific: the Founding rate is $39 a year, and it stays $39 while you stay a member.
Create a free account: 6 long answers a month, email only.
Paying never buys a different answer. Our Paywall Promise
Keep asking
What makes a source trustworthy?
Good science makes a clear, testable claim; shows how evidence was gathered and analyzed; reports uncertainty; and lets others check the work.
New here?
God Plays Dice answers big questions in plain words, with sources and one of six honest labels.
Words worth knowing (4)
- Denominator
- The total underneath a fraction; for a percentage, the group it is a share of.
- Percentage point
- The difference between two percentages, distinct from their relative percentage change.
- P-value
- Under a statistical model’s assumptions, the probability of a test result at least as extreme as the observed one.
- Confidence interval
- An interval constructed by a method with a stated rate of covering the population value across repeated samples, when its assumptions hold.
Sources & further reading
(6) · Checked Oct 2, 2026
- Percentages and percentage points ↗Office for National StatisticsDistinguishes relative percentage changes from differences in percentage points.
- Data visualisation: charts ↗UK Government Analysis FunctionBar-chart zero baselines, line-chart scales, aspect ratios and clear labels.
- Uncertainty and how we measure it for our surveys ↗Office for National StatisticsSampling, non-sampling errors and repeated-sampling interpretation of confidence intervals.
- American Statistical Association Releases Statement on Statistical Significance and P-Values ↗American Statistical Association (2016)Official release reproduces the six principles: significance, effect size, thresholds and transparent reporting.
- Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations ↗Greenland and colleagues, European Journal of Epidemiology (2016)Conditional p-values, model assumptions, selective analyses and confidence-interval limits.
- Writing about data ↗UK Government Analysis Function (2025)Current guidance on context, methodological limits and uncertainty; replaces archived writing guidance.
See a mistake? Report a problem. Corrections are made openly.
CONTINUE THE THREAD
Where this question leads.
How We KnowAnswer
Mutation by Natural Dominance: One Mutation or a Hundred?
If 100 bacteria carry the same DNA change, did it arise once or many times? The answer matters when testing Ricardo Maldonado’s hypothesis.
How We KnowAnswer
Are the colors in space photos real?
A space image can be faithful to measured light without matching human eyesight. Here is how to read the translation.