This is the first in a two-part series...
Normally this blog doesn’t tackle controversies. Hence, as Alert Reader Owen Muir put it, this topic could be too hot for Quizzify. Nonetheless, here we go...
Last Friday a study called NORDiCC was published in the New England Journal of Medicine refuting everything we thought we knew about colonoscopies (except for the parts we wish we didn’t know).
This was a well-designed study (called a "Gold Standard" by STATNews, a publication which whose conclusions are generally quite accurate) tracking cohorts over many years. They did two separate analyses. First, they compared the to-be-invited group to the not-to-be-invited group. In true Randomized Control Trial (RCT) fashion, those two groups were similar. The outcomes were compared.
The, uh, bottom line in that comparison was: there was no noticeable reduction in all-cause death rate among those who received colonoscopies. There was a noticeable but not even close to dramatic reduction in cases of colon cancer.
That reduction is best described in the context of “Number Needed to Screen.” Were these results to apply to your organization, you would prevent 1 case of colon cancer by screening 455 employees. (You'd have to screen a very large multiple of that, to prevent one death.)
The rebuttal
In addition to comparing the control and to-be-invited groups, the study also compared the subset of the to-be-invited group that actually received the colonoscopies to the control group.
That comparison yielded a much more colonoscopy-friendly result, with a 30% reduction in colon cancer and a 50% reduction in colon cancer death. Therefore, the rebuttal goes, we should all still be getting colonoscopies. Further, the number of rebuttals overwhelmed the number of posts and articles in support.
So are the rebuttals right?
As is often the case when a healthcare revenue stream is threatened, and also when people argue with anecdotes ("my life was saved by a colonoscopy") rather than data, their arguments are themselves arguable. Today's post will show the underpinning of why rebuttalists wanting to compare active participants to a passive control may be overstating their case, just as the proponents of the RCT are overstating their own conclusion. Then tomorrow, we will propose an elegant solution for employers who are justifiably baffled by this debate.
Participants vs. Non-Participants (a)
The rebuttalists, who are comparing actively motivated participants to a reference group, are ignoring participation bias. If there's one thing the wellness industry has taught, it's that non-participants (or an uninvited or " passive" reference/control group which would contain many would-be non-participants) is not a valid control for participants. It may not be as much of a bias for colonoscopies as for wellness, but it's a bias nonetheless.
Two wellness vendors proved this brilliantly. First is a participants-vs-univited comparison on a total population at risk for chronic disease (but not diagnosed) and also an RCT using the exact same groups and came up with two dramatically different conclusions. The RCT revealed virtually no detectable change in the two invited groups invited into the program vs. the uninvited control group. Here is how the two analyses differed.
First, here is the RCT, comparing the two groups at the top left to the control on the top right:
That comparison yielded the following result:
The changes were trivial and the last three actually got worse.
Now compare that result to the one similar to what the rebuttalists of last week's study want us to believe. In both that one and NORDiCC, active participants are compared to a passive control, with non-participants in the invited group ignored. Here is that wellness vendor study design, same people but note the different comparison:
In this case, the results were completely the opposite of the RCT: massive savings in almost all categories, after only one year, on a population that had no diagnosed disease to begin with.
Which is more valid? See what one of the editorial advisors of the journal that published this study had to say about it. (Scroll down that article to the comments.)
Participants vs. non-participants (b)
A famous wellness industry study was able to prove this participation bias beyond a doubt. How did they do it? In a brilliant piece of analysis, the researchers split a population into participants and non-participants. Then, they delayed the start of the wellness program by two years in order to tease out the bias caused by the differing mindsets of participants vs. non-participants. So, in the graph below, the first two years of the comparison (2004-2005) involved no program. The program didn't start until 2006, by which point the would-be participants' annual costs had already tracked very favorably vs. the control:
The obvious take-away: participants will outperform non-participants even if there isn't a program to participate in.
Clearly, then, the true impact of colonoscopies is not what the rebuttalists say, as their participants-only result is highly suspect. But "suspect" is not the same as "wrong," as it is in wellness when it's participants-only.
And is the answer really to discourage employees from colon-screening, as the NORDiCC authors strongly imply?
Five observations that seem to have eluded everyone commenting on NORDiCC
Three recommendations for employers based on the data and the rebuttals, and
Three misinformed objections to this NORDiCC study.
Or, to seriously cut to the chase, just contact us now and we will fix the problem for you, saving you money while making your employees very happy. I mean, at least relative to how miserable they otherwise would have been...
