:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/61561975/GettyImages_595263392.0.jpg)
A major meta-analysis published earlier this year that questioned the empirical evidence for supervised drug consumption sites has been retracted by the International Journal of Drug Policy.
Supervised consumption sites (also known as safe injection sites) have gained traction across several US cities, including New York City, Philadelphia, and San Francisco, in response to the opioid epidemic. These sites, which already exist in Canada, Europe, and Australia, create a space where people can use drugs with sterile injection equipment and the supervision of trained staff, who are ready with the opioid overdose antidote naloxone if anything goes wrong. The sites may also link people to addiction treatment on request.
The idea: While in an ideal world, no one would use dangerous and potentially deadly drugs, many people do. So it’s better to give these people a space where they can use drugs with some sort of supervision. It’s a harm reduction approach.
But the meta-analysis, which I reported on shortly after it published, concluded that supervised consumption sites have a small favorable relation to drug-related crimes, but no significant effect on several other outcomes, like overdose mortality and syringe sharing. In short, the meta-analysis didn’t conclude that supervised consumption sites were bad, but they didn’t appear to do much on key outcomes like overdose death.
But the meta-analysis apparently had serious methodological flaws in how it evaluated outcomes. Here’s the technical explanation from the International Journal of Drug Policy:
This [retraction] is supported by the authors’ acknowledgement of methodological weaknesses linked to the pooling of diverse outcomes into a single composite measure. … Accordingly, the authors acknowledge that the combined effect size reported in the original paper should be discounted. Given that the composite measure was a key finding reported by the original paper, the decision to retract the paper from publication had been made, including with the consent of the authors.
In other words, the researchers combined a bunch of different outcome variables into one measurement, potentially obfuscating important differences between the outcomes.
Ahmed Bayoumi and Pamela Leece, who wrote one of two critical reviews that led to the meta-analysis’s retraction, noted that the meta-analysis combined outcomes for mortality, drug-related crimes, sharing of drug equipment, and problematic drug use — even though each of these outcomes could be affected differently by a supervised consumption site.
“Accordingly, that estimate of effect has no practical public health or policy implications,” Bayoumi and Leece concluded.
The meta-analysis’s authors, Tom May, Trevor Bennett, and Katy Holloway, owned up to the mistake, calling it an “honest human error in the use of methods.” The International Journal of Drug Policy, meanwhile, acknowledged “that the peer review process did not pick up on the specific methodological weaknesses identified post publication.”
The meta-analysis contradicted some of the previous research on the topic. For example: Drawing on more than a decade of studies, the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) in 2018 concluded that safe injection sites led to “safer use for clients” and “wider health and public order benefits.” Among those benefits: reductions in risky behavior that can lead to HIV or hepatitis C transmission, drops in drug-related deaths and emergency service calls related to overdoses, and greater uptake in drug addiction treatment, including highly effective medications for opioid addiction.
But the meta-analysis fashioned itself as more rigorous because, unlike reviews like the EMCDDA’s, it evaluated not only what different studies found but also the strength of the studies, allowing it to exclude research that didn’t clear a certain bar. That, in theory, was meant to let the meta-analysis rule out methodological weaknesses in different studies, giving — again, at least in theory — more credibility to the review.
It turns out, however, that the meta-analysis had its own fatal methodological flaw.
Supervised consumption sites have often drawn not-in-my-backyard (NIMBY) criticisms. Essentially, people are worried that if a supervised consumption site opens in their area, it will attract individuals who use drugs to their neighborhood, and that could cause a rise in general crime and social disorder. But those concerns are not supported by the research.
The US Department of Justice has, under President Donald Trump and Attorney General Jeff Sessions, also argued that the facilities “would violate federal law.” The Justice Department claimed, “It is a crime, not only to use illicit narcotics, but to manage and maintain sites on which such drugs are used and distributed.”
The retraction probably won’t do much to quiet critics of supervised consumption sites, particularly their calls for more research into the sites before they’re adopted in the US. But the retraction does mean that supporters for the sites can now discount at least one piece of evidence against them.
For more on supervised consumption sites, read Vox’s explainer.
