One of my favorite podcasts is the Nature podcast, produced by the same folks who produce the famous science journal. It's nice to have something to listen to that is fascinating and not (always) economics.
The first story of the August 11, 2011 Nature podcast reports on treating Parkinson's with an invasive technique where the skull is drilled open and new brain cells are implanted into the brain. Yet to keep such experiments "double-blind", people in the control group also experience surgery---they have a hole also drilled into their head, although unlike for the treatment group, the hole doesn't go all the way through the skull, and no new brain cells are implanted. The control group still ends up spending the same time in the surgery theater and the same gnarly bandage on their head. Both the patients and the researchers don't know who is in the treatment and who is in the control group--hence a "double-blind" experiment.
The podcasters discuss the pros and cons of this approach. However, one thing that they don't discuss is the fact that the control group does not capture the counterfactual--what would happen to a person who does not receive surgery. Rather people in the control group are receiving a somewhat invasive surgery and experiencing anesthesia, which possibly has serious health impacts, given that these are people with a severe stage of a degenerative nervous system disorder.
The whole reason for having a double blind experiment is of course that it helps researchers avoid attributing a placebo effect to a treatment effect. But when surgery for the control group is this invasive, it doesn't seem worth it, especially when the counterfactual of interest (e.g., no surgery) is not observed.
One simple way to get around this problem is to simply have 3 groups: one treatment, one control with the half-way drill surgery, and one control with no surgery. This seems like an easy extension to do, especially because the costs of the 3rd group are going to be limited to check-ups.
I wonder if the medical community may have become so focused on the gold standard of double-blind randomized experiments that they have lost focus on meaningful experiments that have easily interpretable results and that minimize harm to patients.
I wonder if the 3rd control is meant to be implicit. There's got to be data on normal populations--even controls from other experiments who's fake treatments should be orthogonal to the relevant sample. You could just compare your two groups to somebody else's. That assumes, I suppose, that it's relatively costly to build a third control group for every new experiment. But you're right--some kind of comparison to a "normal" sample that didn't undergo an invasive pseudo-treatment should be de rigueur.
ReplyDeleteGood point Chris. As long as there isn't significant selection into these experiments a 3rd group doesn't seem necessary if you can compare them to non-experimental populations. To really say that much more on this topic I guess I'd actually have to read their paper . . .
ReplyDelete