The Plural of Anecdote Isn't Data

The Plural of Anecdote Isn't Data

By Maia Szalavitz 04/09/13

When addiction treatment is evaluated, what clients say is too often given the same weight as what science has proven. Even if you think confrontational tactics worked for you, you're probably exaggerating.

Image: 
fishermans-tale_0.jpg
Hard evidence? Photo via

After a video that showed him verbally and physically attacking his players during practice went viral, the now-former Rutgers coach, Mike Rice, may have a bleak future in basketball. But he might be welcomed as a counselor in the addiction field, where harsh verbal confrontation, deliberate humiliation and even physical assaults have an unfortunately long history of being accepted as “therapy.”

Even now, with overwhelming evidence proving these tactics to be ineffective and sometimes harmful, not only do people in the addiction field continue to defend confrontation, but the media continues to “balance” coverage of its failure with anecdotes suggesting success.

Would we cover a scientifically debunked treatment for cancer, like coffee enemas, as credulously as we do "tough love"?

The Fix’s ongoing resolve to hold rehabs accountable is admirable and much needed. I use the following example not because it's particularly egregious—it's not—but because it's close to hand, and illustrates neatly how such approaches can slip under the radar. It's the investigation that ran on this site last week about an allegation of abuse at the Texas-based rehab, Burning Tree.

The executive director of Burning Tree acknowledged that the program was confrontational and seemed to see nothing wrong with calling the its patient population “classic manipulators.” He admitted that its techniques include making patients wear clothing inside out—supposedly in order to show them that they are putting on a front—refusing to allow patients to speak for days at a time, and only permitting them timed phone calls of five minutes, once a week, with family.

A former therapist who worked with Burning Tree said that the program attempts to “break [clients] down mentally”—even those who have a mental illness diagnosis. 

But for anyone looking for effective treatment, that should immediately foreclose Burning Tree as a viable option. Why? Because the scientific research on addiction treatment is clear. As The Fix reported, a major 2007 review of the data by William Miller, PhD, and William White, MA, concluded, “Four decades of research have failed to yield a single clinical trial showing efficacy of confrontational counseling, whereas a number have documented harmful effects, particularly for more vulnerable populations." 

Not a single positive study in 40 years of research. If confrontation were a medication, the FDA would have pulled it from the shelves years ago.

Nevertheless, the Fix article ends with these words:

Yet despite conclusive evidence that confrontational approaches like Burning Tree’s do more harm than good, numerous alumni testified that the place is a “welcome” alternative for some addicts who have been through the rehab mill.

Burning Tree’s hard-core practices are controversial—and are likely to remain so for as long as the debate between confrontational and self-empowerment approaches continues. 

This ending could be read to suggest that this "debate" is a legitimate one: Scientific consensus is on one side, individual anecdotes are on the other, and both have equally valid claims. Any such assumption would be flat-out wrong.

The “debate” is the relic of a time before science was applied to addiction treatment. And it would have been resolved years ago if addiction treatment were actually viewed as medical care. The widespread promotion of confrontation as critical to addiction care comes out of the Synanon cult, on which most publicly funded addiction treatment was long based. Any sane area of medicine would have revisited this treatment when Synanon started stockpiling weapons and forcing sterilization and partner swapping on its members back in the '70s, but the addictions field is unfortunately, um, different. 

Phoenix House, Daytop, Delancey Street, Gateway and Straight Inc., are just a few of the many rehabs that were originally based on Synanon, or founded by Synanon members or by people trained in Synanon methods. Most have either significantly changed their ways or been shut down, typically due to abuse. Data showing that these methods didn’t improve on rates of recovery without treatment and that they could actively harm people has been available for decades.

So how can it still be acceptable for any of us in the media to “balance," even implicitly, four decades of negative research with a few anecdotes of success from alumni? Would we cover a scientifically debunked treatment for cancer, like coffee enemas, so credulously? 

The investigation of Burning Tree on this site was a mild offender in this sense—the addiction field has been plagued with many worse examples for as long as I’ve covered it. Of course, there is always room in journalism for telling individuals' stories and putting faces to statistics to make them more immediate and accessible. But there is a problem if we forget—or risk allowing readers to forget—that data is not the plural of anecdote. And until we give emphatic prominence to data in terms of how we not only choose but report on treatment, addiction care will be stuck in the prescientific era.

The reason we can’t simply take people's (even our own) stories of treatment success at face value is simple: These stories are inevitably only a part of the picture; they are subjective, by definition, and they can't help but have biases. The history of scientific medicine is the history of understanding sources of bias in data. 

Imagine a doctor who believes that bleeding is the best way to treat fever. He has been trained to do this, and he encounters a man who is extremely ill with fever. He leeches off a bit of blood and lo and behold, in a week, the patient recovers! The treatment works, he declares. 

Of course, what probably happened is that the person would have recovered no matter what the doctor did—but now both the patient and the doctor have a wonderful anecdote of how bleeding saved a life. And if the next patient dies, well, that was because the doctor didn’t take enough blood or took too much or started too early or too late—not because there’s anything wrong with the treatment. When people are attached to a particular method of treatment, they tend to rationalize away its failures and focus on its success, research shows.

Consequently, this method of anecdote can’t prove anything. You can use it to see whatever type of result you want to see. In the case of confrontational rehabs, the failures are written off as “not ready” or “didn’t try hard enough” rather than as people who were harmed by treatment; the successes, of course, are all theirs.

In addition, the tough love approach sees the client as infantile, someone with little moral capacity who therefore responds only to discipline; as a result, the counselor is qualified to “break people down" and brooks no argument. By contrast, empathic care sees clients as people who have a medical problem (addiction and, often, a mental illness) rather than a moral one, and who also have strengths and weaknesses—just as counselors have; treatment is a collaboration and communication between one person who needs help and another who is trained to help patients (and help them help themselves).

A large body of psychological research shows that the more difficult and grueling you make an experience—or the more expensive—the more people will believe that it was worthwhile. For example, people find that expensive food tastes better—and harsher initiations make people value groups more. Similarly, claims of “I needed a tough rehab” may simply be a way of making meaning out of what happened. People tend to prefer almost anything to seeing themselves as suckers or seeing their suffering as meaningless: It feels better to say, “I needed being humiliated in order to get better,” than to say, “I paid to be maltreated.”

So, if anecdotes can’t be trusted, what can? 

Medicine worked its way out of routinely using harmful treatments through the introduction of what are called double blind randomized controlled trials. Unfortunately, however, the addiction field has yet to recognize why such trials are critical before treatments can be accepted. These trials are designed very carefully to eliminate as many extraneous variables as possible in order to determine whether a cause and effect are linked.

Consequently, comparing claims based on scientific research, which makes every effort to get rid of bias, to claims based on anecdotes, which make no effort to do that, makes little sense. In the case of confrontational addiction care, all the studies have even reached consensus, so giving equal time to "both sides" makes about as much sense as giving climate change deniers equal time with the 99.9% of the data showing that we have altered the earth’s climate.

If we want addiction to be treated as a medical problem, it’s time for the addiction field—and the media—to take science seriously. The research doesn’t support tough confrontation, regardless of individual claims that "being broken down" worked. Giving anecdote and data equal weight is not "balance." Real balance means putting the science first.

This is not to say that we shouldn’t listen to and value people’s stories. But it does mean that we can never tell from one person’s story whether our own treatment was “the only thing that could have helped." We only got the treatment we got.

Of course, science has its own flaws. It can’t always eliminate biases, and there are many ways to lie with statistics. But in the case of treatment techniques intended to shame and break people, the data is unequivocal. So far, every group tested has benefited more from empathy than from confrontation—whether it be sociopathic murderers, drunk drivers, teenagers or women. 

There is no excuse for programs to use shaming and humiliating techniques in the 21st century. If true believers want to demonstrate that tough love works, they should do carefully designed trials directly comparing their technique to that of a more empathic one on the group of addicts that they believe requires what they offer. Until then, the only place treatments suspected of potentially doing harm should be allowed is in such trials. Programs that won’t stop using these tactics should be shut down. 

Maia Szalavitz is a columnist at The Fix. She is also a health reporter at Time magazine online, and co-author, with Bruce Perry, of Born for Love: Why Empathy Is Essential—and Endangered (Morrow, 2010), and author of Help at Any Cost: How the Troubled-Teen Industry Cons Parents and Hurts Kids (Riverhead, 2006).

Please read our comment policy. - The Fix
Disqus comments
Maia Szalavitz.jpg

Maia Szalavitz is an author and journalist working at the intersection of brain, culture and behavior.  She has reported for Time magazine online, and is the co-author, with Bruce Perry, of Born for Love: Why Empathy Is Essential—and Endangered, and author of Help at Any Cost: How the Troubled-Teen Industry Cons Parents and Hurts Kids. You can find her on Linkedin and  Twitter.

Disqus comments