Evaluating models

Our model reaches 99 percent accuracy. You read sentences like this all the time, and they sound finished and great. But a single number can be splendidly misleading. Whether a model is actually useful depends on entirely different questions: which errors does it make, how costly are those errors, and was it checked on data it has never seen? This topic gives you the tools to put success numbers in their place.

The accuracy trap

Accuracy is the share of cases in which the model is right. Imagine a test for a rare disease that affects 1 in 100 people. Now I build a model that always says healthy, without looking at anything. Out of 100 people it is right about the 99 healthy ones and wrong only about the one sick person: 99 percent accuracy, and yet it finds not a single case of the disease. It is completely useless. This is exactly what ails accuracy as a lone number: when one class is rare, a model can shine by simply ignoring the rare class. The first question about any success number is therefore: what would the dumbest imaginable strategy achieve? Only what sits above that is an achievement.

False positives and false negatives

A model can be wrong in two ways, and the two are rarely equally bad. False positive means: an alarm without cause, the spam filter throws an important job application into the spam folder, the test declares a healthy person sick. False negative means: no alarm despite cause, the spam mail slips through, the disease stays undetected. Which kind is worse depends on the use case. For a disease test the false negative is dangerous, for a spam filter the false positive is the nuisance. So you look at both error types separately and ask two questions. First: of everything the model flagged, how much is actually correct? That number is called precision. Second: of everything it should have found, how much did it find? That is the recall. Two numbers instead of one, but they tell the whole story.

Spotting overfitting

The second big trap: a model is measured on the same data it was trained on. Then you are only measuring its memory. The honest yardstick is the gap between training and test. If a model reaches 100 percent on the training data but only 70 percent on held back test data, it has memorised the training examples instead of finding patterns, that is overfitting. A model with 85 percent in training and 84 percent in test is almost always the better one, even though its training number looks worse. The gap grows especially when the model has many parameters and little data, because then its capacity is enough to simply store every special case.

What an honest evaluation needs

A good evaluation answers three questions. First: was the model tested on data it has never seen and that resembles real world use? A traffic sign recogniser tested only on summer photos says nothing about winter. Second: how does the model compare against a simple baseline, such as always giving the most common answer? Third: are the error types broken down instead of hidden inside a single accuracy number? If any of these answers is missing, the success number is advertising, not measurement. That holds for school projects just as much as for the press releases of big companies.