Understanding Regression to the Mean

Several weeks ago I wrote a post on my blog concerning the seemingly inexplicable fluctuations in patient volume and business that occur from time to time. The post commented that no matter when business is slow, people could come up with a reason that explains it; be it hot weather, cold weather, rain, sun, school, vacation, flooding, drought, what-ever. But here is the actual reason. Are you ready? The answer is. . . regression toward the mean and standard deviation. Yes, that is it. Nothing sexy. Nothing that will end up on the OPRAH Show. Not THE SECRET. No, nothing metaphysical. Just basic statistics. The variations in patient flow, or most any set of numbers, can all be explained by regression towards the mean and standard deviation.

Mean means average. What’s average? Go back to school. Standard deviation is a measure of scatter, or variance, around the average number. Actually, standard deviation is the square root of the variance. Regression to the mean is a concept that states that within a set of numbers, like sales figures, or weekly patient encounters, or golf scores, values will vary, or scatter, around an average, but ultimately they will tend towards the average number. For example, an amateur golfer may have an average score of 92, but may shoot anywhere from 86 to 105, depending on the day. Therefore, if he has 3-4 rounds in the 80’s, rather than feeling as if he “gets” the game of golf, he should realize that based on regression to the mean, he will ultimately be brought back to reality and have 3-4 rounds in the hundreds. Regression to the mean will teach him that he never “got” golf and he is likely not improving, but that he hit a hot streak that will unfortunately reverse itself. Enjoy it while you can. It’s fleeting.

In medical practice, and I suspect in sales and other similar businesses, regression towards the mean has real significance. I have friends in commercial sales that constantly have ups and downs in their numbers. Regression to the mean can explain these hot streaks and cold streaks. Medical practice is no different. For example, let’s say you have a new practice and you see 30 patients per week, on average. Some weeks you may see 35, and other weeks you may see 25. Then after 4 weeks in a row of seeing 33 patients, your office manager tells you that she believes it is time to hire additional staff to accommodate the increased volume. Maybe you should and maybe you shouldn’t, but you may just be experiencing normal variability. You may want to consider standard deviation and regression to the mean as an explanation, especially before you hire additional staff and take on additional expenses. Conversely, when things are slow, and you are in the throws of a downturn, you may wish to determine if this is within your typical variability. Weekly patient volume may simply tend toward the mean. You don’t need to panic or lay off staff. Or maybe you do, but you can calculate it pretty easily.

Let’s go a bit deeper into the terms average, variance, and standard deviation. Mean is the average number, variance is the scatter around that number, and standard deviation is the square root of the variance. In English, if an entire population takes a test, 95% of people should have results that fall within 2 standard deviations from the mean. For our amateur golfer, we’ll take his previous 20 scores, average them and compute the standard deviation. Sparing you the math, his average score is 92 and his standard deviation is 3. This means that 95% chance, on any given day he’ll score between 86 and 98. If he tells you he broke 80, he’s full of it. Statistically speaking, it’s not possible. From your practice’s perspective, you can go back and calculate the average number of visits per week x 6 months, and then compute the standard deviation (use Excel, its easy), on that number. If you saw, on average, 27.1 visits and had a standard deviation of 3.7, then you can determine that any variation in weekly traffic between ~19 and 34 patients per week (2 Standard deviations from the mean) is normal, 95% chance, and will even out over time.

You may find my argument/explanation overly analytical. It certainly is not the stuff that winds up in best selling self-help books. It is way too boring! Yet I find it very comforting to know the wild fluctuations in busy-ness that I have seen in my own practice have such mundane explanations, rather than magical ones. Now I can turn my positive energy, you know, the-send-a-positive-vibe type, toward increasing my average number of encounters per week and to decreasing my standard deviation by eliminating no shows, etc. Or I can pray that my average goes up, while I market myself.

Hope you stayed awake.