16.7: Regression to the Mean

The grade school in Belleville, KS administers an achievement test to all the children who enter fifth grade. At the end of the school year, they give the same test again. The average score both times is 100, but something odd seems to have happened. Children who scored below average on the test the first time tend to improve (by about five points), and children who scored above average tend to do worse (by about five points).

What’s going on? Might it be that when the two groups of children interact extensively, as they did over the school year, the higher group pulls the other group up, while the lower group pulls the higher group down?

Here is another example. Instructors in an Israeli flight school arrived at the conclusion that praising students for doing unusually well often led to a decline in their performance, while expressing unhappiness when they did poorly often lead to improvement. This group just happened to be studied, but many other instructors and teachers come to similar conclusions. Are they right?

The more likely explanation in both cases is that they involve regression to the mean. Regression to the mean is a phenomenon where more extreme scores or performances tend to be followed by more average ones. It’s also called mean reversion.

The basic idea is that extreme performances tend to revert, regress, or move back toward the mean (i.e., toward the average). So unusually low scores tend to be followed by higher ones (since low scores are below the mean). And unusually high scores tend to be followed by lower ones (since high scores are above the mean). Regression to the mean can occur with anything that involves chance, it occurs frequently, and it is very easy to overlook it.

Suppose that you did unusually well (or unusually badly) when you took the ACT. There is a reasonable probability that if you took them again, your second score would be closer to the average. In sports, someone who has an unusually good or unusually bad game is likely to turn in a more average performance the next time around. Indeed, people often remark on the sophomore slump, in which athletes who did exceptionally well as freshmen fall off a bit as sophomores, and the Sports Illustrated jinx, in which people who do very well and make it to the cover of Sports Illustrated play worse in subsequent weeks. Many cases of this sort simply involve regression to the mean.

Since regression to the mean can occur with anything that involves chance, it affects more than just performances on the basketball court or in the concert hall. For example, there is a lot of randomness in which genes parents pass on to their offspring, and parents who are extreme along some dimension (unusually tall or short, unusually susceptible to disease, etc.) will tend to produce children whose height or susceptibility to disease are nearer the average. Two very tall parents are likely to have tall offspring, but the children are not likely to be as tall as the parents.

Why do things tend to “regress” to the average value? Why not to some other point? Performances involve a “true level of ability” plus chance variation (“error”). The chance variation can involve many different things that lead to better or worse scores than we would otherwise have.

Suppose that you take the ACT several times. Some days you may be very tired, other days well rested; some days nervous, other days more focused and confident; some days you may make a lot of lucky guesses, other days mostly unlucky ones. Often the good and bad conditions will pretty much cancel out, but sometimes you will have mostly the good conditions (in which case you will score very well), and sometimes mostly the bad ones (in which case you will score poorly). If you score extremely well, the chances are that this is a combination of a high ability plus auspicious background conditions, and so your score is likely to be lower the next time around.

Because of the chance error, the distribution of your performances fits a pattern that resembles the standard bell curve. In this distribution of scores, the average value is the value closest to the largest number of cases (we will return to this point when we consider descriptive statistics). So unusually good or unusually bad performances are likely to be followed by the more probable performances, which are just those nearer the average. The idea may be clearer if we consider a concrete example. Suppose that you shoot thirty free throws each day. Over the course of a month the percentage of shots that you hit will vary. There is some statistical variation, “good days” and “bad days”.

Many things may improve or weaken your performance: how sore your muscles are, how much sleep you got, how focused you are. Sometimes all the things come together in the right way and you do unusually well; other times everything seems to go wrong. But on most days these factors tend to cancel each other out, and your performance is nearer your average. Since your performance is more often near the mean, extreme performances are likely to be followed by more average ones.

Regression and Reasoning

Regression to the mean is very common but frequently overlooked, and failure to appreciate the phenomenon leads to a lot of bad reasoning.

Regression and Prediction

Suppose that Wilbur has an exceptionally good or an exceptionally poor performance shooting free throws in a game. It is natural to base our prediction about how he will do next time on his free throw percentage in the game we saw; we just project the same percentage. But if his shooting was way above (or way below) the average for players in general, his percentage in subsequent games is likely to regress toward this average.

Again, a company may detect falling profits over the previous three months. The manager gets worried, thinks about a way to change marketing tactics, and predicts that this will turn things around. The new tactics are adopted and profits go back up to their previous level. But this may result simply from regression to the mean. If so, the new marketing techniques will (incorrectly) be given credit for the turnaround.

There are various cases where our failure to take regression into account leads to bad predictions; for example, if someone does unusually well (or unusually poorly) in a job interview, we are likely to have a skewed impression of how well they will do on the job.

Explanation and Superstition

Suppose that you’ve had a couple of poor performances of late. Things went badly on some exams or in the last two recitals you gave. Then you have an unusually good performance the day you wear your green sweater, the ugly one your aunt gave you, and it may become your lucky sweater. Superstitions are often based on something (e.g., wearing the lucky sweater) just happened to coincide with a shift toward a better performance that is simply due to regression to the mean.

Of course, most of us don’t really believe in lucky sweaters (though we might still wear one, on the theory that “it can’t really hurt”). But lack of awareness of regression to the mean is responsible for a lot of bad reasoning. Whenever an element of chance is involved, regression to the mean comes into play. Nisbett and Ross note that if there is a sudden increase in something bad (e.g., an increase in crime, divorce rates, bankruptcies) or sudden decrease in something good (e.g., a decline in high school graduation rates, or in the amount given to charity) some measure is likely to be taken. For example, if there is a sudden increase in crime, the police chief may increase the number of police officers walking the beat.

If the implementation of a new policy is followed by a decrease in something undesirable or an increase in something desirable, we are likely to conclude that the measure is responsible for the shift. But in many cases, such a shift would have occurred without the measure, simply due to regression to the mean. In such cases, we are likely to explain the reduction in crime by the increased number of police on the beat. The measure will be given too much credit.

As a final example, let’s return to the question of rewards and punishments. Parents and teachers often must decide whether rewards or punishments are more likely to be effective. Unusually good behavior is likely to be followed by less good behavior simply because of regression, and unusually bad behavior is likely to be followed by better behavior for the same reason.

Hence, when we reward someone for doing extremely well, they are likely to do less well next time (simply because of regression to the mean). Similarly, if we punish them for doing badly, they are likely to do better next time (for the same reason). In each case the change in performance may simply be due to regression to the mean, and the reward and punishment may have little to do with it. It will be natural to assume, though, that punishments are more effective than rewards.

When something like punishment or an increase in police on the beat accompanies regression to the mean, we can easily conclude that society in general, or that we specifically, have found a method to solve certain sorts of problems when in fact, we have little power to solve them. Obviously, this doesn’t lead to good decision making at either the public or the personal level.