13.2. Base rate fallacy#

Another way humans can be led astray by data is with the base rate fallacy. This is a well-documented and frequent, if not universal, cognitive bias in humans, which is that when we are presented with individuating information along with base rate information, we tend to over-weight the individuating information and fail to incorporate the base rate information (or outright ignore it).

Individuating information is information about an individual unit or observation. Base rate information is information about the relevant underlying context. A simple example of a base rate fallacy in action is any time we meet a tall person and ask if they are a professional basketball player. While our tall readers already know this is a fallacy (and an annoying one, at that), to spell it out:

  • Individuating information: Jiayu is tall!

  • Base rate information: The probability of someone being a professional basketball player in the US is \(529/329,000,000 = 0.0000016\)

  • Yet: we are tempted to conclude Jiayu must be a professional basketball player!

Faulty inferences from the base rate fallacy are everywhere. They come up any time too little weight is placed on the underlying relevant context and too much weight is put on a specific data point. For example, despite one of the three broad underlying contexts:

  • a company is doing well

  • vaccines are effective

  • sea levels are rising

We might be tempted to overreact to a specific piece of information, such as:

  • the most recent quarterly earnings are down

  • our friend’s uncle had a breakthrough case

  • it’s snowing

and conclude (incorrectly) that:

  • we better sell our stocks NOW

  • vaccines are useless

  • climate change is a hoax

What to do about this?#

Awareness and communication of the base rate fallacy is one of the biggest things we can do as scientists to improve all our collective inferences about data we see in the world. The fallacy is a really tricky one, though, because even though we might be aware we are committing it, or are at risk of committing it, it can be difficult to believe it because we are looking at a particular piece of data that might be perfectly reliable on its own: earnings ARE down, he DID have a breakthrough case, it IS snowing – and, my goodness, Jiayu is tall!

Much has been written about this elsewhere, but to briefly summarize an example of this challenge in the real world: As more and more people got vaccinated against Covid-19 in 2021, we also observed more breakthrough covid cases, and more people who are hospitalized for covid who had already been vaccinated. It’s easy to see how we could conclude that perhaps the vaccines don’t work as well as they do if the percentage of vaccinated people who are hospitalized with covid is going up. But this ignores the base rate: the underlying proportion of all people who have been vaccinated against Covid-19 has also gone up.