How We KnowAnswer
Why Can We Read Mammoth DNA but Not Dinosaur DNA?
A frozen mammoth tooth can hold readable DNA. A dinosaur fossil can preserve the shape of a bone for far longer. Here’s why those are different feats.
THE QUESTION LIBRARY HOW WE KNOW
What are the most common thinking traps, and how do we avoid them?
Every human mind takes shortcuts, including the minds of scientists and of the people who wrote this page, so here are the best-known traps and the habits that help.
THE SHORT ANSWER
Everyone falls into thinking traps, including scientists. The best known include confirmation bias (favoring evidence that fits what we already believe), motivated reasoning (reaching the answer we want), judging by what comes to mind easily, ignoring how common something is, seeing patterns in random noise, and survivorship bias (studying only the winners). You can’t switch these off. But habits help: look for what would prove you wrong, ask for the numbers, and let other people check your work.
- In a 1960 test, only 6 of 29 young adults found a simple number rule without first announcing a wrong one, and many tested only cases that fit their guess.
- In 1974, Amos Tversky and Daniel Kahneman showed that mental shortcuts usually work but lead to errors we can predict.
- In a 1949 class, every student got the same personality sketch, and only 5 of 39 rated it below 4 on a 0 to 5 scale.
- In World War II, the statistician Abraham Wald worked out how vulnerable planes were using damage on the ones that made it home.
- Not every famous finding holds up: two large team tests found little or no sign of the claimed “ego depletion” effect.
About this picture
A made-up damage pattern on a World War II bomber. The red dots show hits on planes that made it home. Planes hit in the empty spots, such as the engines, were more likely to be the ones that never came back to be counted.
WHERE THE EVIDENCE STANDS
How sure are we?
- People tend to seek and favor evidence that fits what they already believe.Wason’s 1960 number-rule experiment; Raymond Nickerson’s 1998 review in Review of General Psychology calls confirmation bias ubiquitous and documents it in many settings.
- How easily examples come to mind shapes how common we think something is.Tversky and Kahneman (Science, 1974) describe the availability heuristic with famous-name lists and words with the letter r. Repeats of the famous-names test found moderate (1995) and smaller (2001) effects, and a 1998 study found letter-position judgments generally followed real proportions.
- People often give base rates, meaning how common something is to begin with, too little weight.Tversky and Kahneman’s 1974 engineer-and-lawyer experiment showed it clearly. But Jonathan Koehler’s 1996 review in Behavioral and Brain Sciences finds that the literature does not support the idea that people routinely ignore base rates: they are almost always used, to a degree that depends on how the problem is presented.
- Vague, general statements about personality feel as if they were written just for us.Forer’s 1949 classroom demonstration; a 1985 review by Dickson and Kelly found acceptance of such profiles depends on how relevant and favorable they seem.
- Self-control reliably runs down after a short effort, like fuel in a tank.A 23-lab preregistered replication (2,141 people, 2016) found an effect near zero, and a 36-lab test in 2021 found no evidence for it in its planned analyses.
WHAT WE DON’T KNOW YET
Researchers still debate how big some biases are outside the lab, and how much training can reduce them for good. Some effects that once looked solid have shrunk under larger tests, and more may follow.
WHAT WOULD CHANGE THIS ANSWER
Large, careful replications that failed to find confirmation bias or the availability effect would change this answer. So would strong evidence that simple training removes these biases for good.
Everyone has thinking traps. They are not signs of low intelligence. They are shortcuts the human mind uses all the time, and most of the time they work well. But they also lead to mistakes we can predict. Scientists fall into them. So do experts in many fields, and so do the writers of this page.
People ask about thinking traps for many reasons. Some want to judge a video or a viral post. Some are rethinking what they were raised to believe, or wondering why a relative sees the world so differently. Some want to check their own reasoning. All of these are good reasons to learn how our minds slip.
Here are the traps that show up most often. Confirmation bias is favoring evidence that fits what we already believe. Motivated reasoning is reaching the conclusion we want and then finding reasons for it. The availability heuristic means judging how common something is by how easily examples come to mind. Base-rate neglect means forgetting how common something is to begin with. Patternicity is seeing meaningful patterns in random noise. The Barnum effect is feeling that a vague description was written just for you. And survivorship bias is drawing lessons only from the winners, because the losers are not around to be counted.
These are well studied. In a classic 1960 experiment, most of the bright young adults tested announced at least one wrong rule, and people typically tested only cases that fit their guess. In 1974, two psychologists showed that mental shortcuts make people misjudge how often things happen.
You can’t switch these traps off. Even knowing about them does not make you immune. But habits help. Ask what would prove you wrong. Ask for the numbers. Ask who is missing from the story. And let other people check your work, because that is how science catches its own mistakes.
THE LONG ANSWER
THE LONG ANSWER · ABOUT 8 MINUTES
You’ve read your 3 free long answers this month.
The short answer, how sure we are, what we don’t know, what would change our mind and every source stay free on every page, for everyone.
All Access members read the long answer: the evidence piece by piece, the strongest objections, and how to check it yourself.
Join All Access · $1 a week Or $6 a month
Billed $52 once a year. Cancel anytime in your account.
Until December 4, 11:59 pm Pacific: the Founding rate is $39 a year, and it stays $39 while you stay a member.
Create a free account: 6 long answers a month, email only.
Paying never buys a different answer. Our Paywall Promise
WHERE THE AUTHOR’S RESEARCH TOUCHES THIS
Author’s hypothesis. In the book’s Introduction, Ricardo Maldonado, author of GOD PLAYS DICE, describes a falsifiable hypothesis that our Big Bang may be the aftershock of a higher-dimensional event. On this site it goes by the name HD-Blast. It is an Author’s hypothesis, not an established result. Two passages in the book deal with traps on this page.
In Chapter 5, the author sets aside a number pattern he has explored in particle masses. He calls it “a line of thought rather than a claim” and writes that with enough freedom, you can find a pattern almost anywhere. A real pattern, he argues, has to predict new measurements, not only explain old ones. In Chapter 11, under the heading “The Hardest Result to Correct Is Your Own,” he writes that correction is hardest when the idea is yours, and that a corrected claim is not a corrected human being. Whether HD-Blast survives is up to the data and to other people checking it.
Keep asking
Why do extraordinary claims need extraordinary evidence?
Because a claim that clashes with a huge amount of well-tested evidence starts out unlikely, and it takes strong evidence to outweigh all of that.
See this path: Astrology, psychics, awakening: what holds up?
New here?
God Plays Dice answers big questions in plain words, with sources and one of six honest labels.
Words worth knowing (5)
- Heuristic
- A mental shortcut or rule of thumb that gives quick answers. Heuristics usually work but can lead to predictable errors.
- Base rate
- How common something is to begin with, before you learn any details about a particular case.
- Replication
- Repeating a study, often with new people and new labs, to see whether the result holds up.
- Preregistered
- A study whose plan and analysis were written down and made public before the data came in, so the results can’t be shaped after the fact.
- Apophenia
- A tendency to see meaningful connections in random events, a leaning toward false-positive errors.
Sources & further reading
(10) · Checked Sep 26, 2026
- Judgment under Uncertainty: Heuristics and Biases (Amos Tversky and Daniel Kahneman) ↗Science 185 (1974): 1124–1131Representativeness, availability, anchoring; “highly economical and usually effective” but “systematic and predictable errors”; 70/30 engineers and lawyers, Dick description judged .5 either way; famous-names lists; words with r first vs third; “Experienced researchers are also prone to the same biases” when thinking intuitively. Full text read from the JSTOR copy hosted at https://sites.socsci.uci.edu/~bskyrms/bio/readings/tversky_k_heuristics_biases.pdf. Replication record: McKelvie, Perceptual and Motor Skills (1995), doi:10.2466/pms.1995.81.3f.1331 (famous-names effect d = 0.53); McKelvie and Drumheller (2001), doi:10.2466/pms.2001.92.2.507 (smaller effect, d = 0.34); Sedlmeier, Hertwig and Gigerenzer, JEP: Learning, Memory, and Cognition 24 (1998), doi:10.1037/0278-7393.24.3.754 (no single published replication of the letter result except a one-page article; across 3 studies letter-position judgments generally followed actual proportions; full text from the Max Planck PuRe repository).
- On the Failure to Eliminate Hypotheses in a Conceptual Task (P. C. Wason) ↗Quarterly Journal of Experimental Psychology 12 (1960): 129–1402, 4, 6 rule; the concept was “three numbers in increasing order of magnitude”; 6 of 29 reached the correct rule without previous incorrect ones. Full text read from the copy at web.mit.edu (curhan/www/docs/Articles/biases).
- Confirmation Bias: A Ubiquitous Phenomenon in Many Guises (Raymond S. Nickerson) ↗Review of General Psychology 2 (1998): 175–220Definition; Wason task and positive-test strategy (Klayman and Ha 1987); two people with conflicting views both strengthened by the same evidence (Lord, Ross and Lepper 1979); asking people to consider alternatives reduces the positive-test tendency (Baron, Beattie and Hershey 1988); science succeeds less through self-criticism than through scientists testing each other’s hypotheses. Full text from https://pages.ucsd.edu/~mckenzie/nickersonConfirmationBias.pdf.
- The Case for Motivated Reasoning (Ziva Kunda) ↗Psychological Bulletin 108 (1990): 480–498Abstract (PubMed 2270237): considerable evidence people arrive at conclusions they want, constrained by their ability to construct seemingly reasonable justifications. Also: Pronin, Lin and Ross, “The Bias Blind Spot,” Personality and Social Psychology Bulletin 28 (2002), doi:10.1177/0146167202286008.
- The base rate fallacy reconsidered (Jonathan J. Koehler) ↗Behavioral and Brain Sciences 19 (1996): 1–17Abstract: the literature (including the lawyer–engineer problem) does not support the conventional wisdom that people routinely ignore base rates; base rates are almost always used; degree depends on task structure and representation; used more when frequentist or implicitly learned.
- Patternicity: Finding Meaningful Patterns in Meaningless Noise (Michael Shermer) ↗Scientific American 299, no. 6 (December 2008)Shermer names the tendency patternicity (“I call it”); type I and type II errors; rustle-in-the-grass example; Foster and Kokko’s model. Also: Foster and Kokko, Proc. R. Soc. B 276 (2009; published online Sept. 2008, which Shermer calls “a September paper”), doi:10.1098/rspb.2008.0981; Blain et al., “Apophenia as the disposition to false positives,” Journal of Abnormal Psychology (2020), doi:10.1037/abn0000504.
- The fallacy of personal validation (Bertram R. Forer) ↗Journal of Abnormal and Social Psychology 44 (1949): 118–12339 students; identical 13-item sketch a week after the test; items largely from a newsstand astrology book; only five ratings of the sketch below 4 on a 0 to 5 scale. Read by OCR of the scan hosted by Astronomy magazine (https://www.astronomy.com/wp-content/uploads/2024/01/Forer-fallacy-of-personal-validation-1949.pdf). Also: Dickson and Kelly, Psychological Reports 57 (1985), doi:10.2466/pr0.1985.57.2.367 (acceptance depends on relevance and favorability).
- Abraham Wald’s Work on Aircraft Survivability (Marc Mangel and Francisco J. Samaniego) ↗Journal of the American Statistical Association 79 (1984): 259–267Wald at the Statistical Research Group estimated aircraft vulnerability from survivors’ data; data on downed aircraft unobservable; methods used in World War II, Korea and Vietnam; engine area most vulnerable in the worked example. Read by OCR of the scan at https://people.ucsc.edu/~msmangel/Wald.pdf. MacTutor biography (https://mathshistory.st-andrews.ac.uk/Biographies/Wald/): Statistics Research Group at Columbia.
- A Multilab Preregistered Replication of the Ego-Depletion Effect (Hagger and colleagues) ↗Perspectives on Psychological Science 11 (2016): 546–57323 labs, 2,141 participants, d = 0.04 with a confidence interval including zero. Also: Vohs et al., Psychological Science (2021), doi:10.1177/0956797621989733 (36 labs, 3,531 people, preregistered analyses found no evidence); Klein et al., “Investigating Variation in Replicability,” Social Psychology 45 (2014), doi:10.1027/1864-9335/a000178 (13 effects, 36 samples, 10 replicated consistently, 1 weak, 2 did not replicate).
- GOD PLAYS DICE, Volume OneRicardo MaldonadoIntroduction: “a specific, falsifiable hypothesis about the Big Bang — that it may be the aftershock of a higher-dimensional event.” Chapter 5: the lepton-mass pattern called “a line of thought rather than a claim”; with enough freedom “you can find something almost anywhere”; a real pattern needs a framework that predicts new measurements. Chapter 11, section “The Hardest Result to Correct Is Your Own”: a corrected claim is not a corrected human being.
See a mistake? Report a problem. Corrections are made openly.
CONTINUE THE THREAD
Where this question leads.
How We KnowAnswer
Mutation by Natural Dominance: One Mutation or a Hundred?
If 100 bacteria carry the same DNA change, did it arise once or many times? The answer matters when testing Ricardo Maldonado’s hypothesis.
How We KnowAnswer
Are the colors in space photos real?
A space image can be faithful to measured light without matching human eyesight. Here is how to read the translation.