Bayes' theorem

A test for a rare disease detects 99% of all people who have it. In healthy people it wrongly triggers in only 1% of cases. Your test is positive. How likely is it that you are ill? Most people answer 99%. The correct answer is often below 10%, and this difference is not a quibble, it has real consequences in medicine and in the courts.

The direction is what matters

The test promises: if you are ill, then the test is probably positive. That is $P(\text{positive} \mid \text{ill})$. But what you want to know is: if the test is positive, am I ill? That is $P(\text{ill} \mid \text{positive})$. These two numbers look almost the same and are completely different, because they count inside different groups.

Bayes' theorem is the rule for getting from one direction to the other. You can write it as a formula, but you can also manage without: simply calculate with a large number of people and count. That route is less error-prone and easier to explain.

Calculating with 10,000 people

Suppose 1% of the population is ill. Among 10,000 people that is 100 ill and 9900 healthy. Of the 100 ill people the test detects 99. Of the 9900 healthy ones it wrongly triggers in 1% of cases, which is 99 people. So in total there are $99 + 99 = 198$ positive tests, and only half of them belong to people who are really ill.

The reason for this surprising result is the size of the two groups. There are almost a hundred times as many healthy people as ill ones. So even a tiny error rate in a huge group produces just as many cases as a large hit rate in a small one. This starting frequency is called the base rate, and ignoring it is the most common error of reasoning in probability.

When the disease is more common

Do the same calculation again with a disease affecting 10% of people. Out of 10,000 that means 1000 ill and 9000 healthy. The test finds 990 ill people and produces 90 false alarms. Now $\frac{990}{1080} \approx 92\,\%$ of the positive tests are correct. The same test, the same accuracy, an entirely different result.

That is exactly why the whole population is not tested for rare diseases, only people with symptoms or in a risk group. In that smaller group the base rate is much higher, and only then does a positive result carry weight. And that is why in practice a positive test is confirmed with a second, different method.