It seems as if the past few years have seen a rise in the number of “spree killings” committed by people who had/have mental health problems, and a rise in news coverage (including more time and more articles devoted to individual incidents). Naturally, among the many questions this raises is whether or not current medications for relevant mental health disorders are effective at reducing violent criminal behavior. A recent paper in a premier medical journal (The Lancet) “Antipsychotics, mood stabilisers, and risk of violent crime” purported to have looked into this very question. According to the study and countless media reports, the answer is “yes”.
Why do I say purported? Well, imagine you wanted to answer this question. How might you do so (imagining that you had access to national records)? A central issue is the determining your target population from which to sample. We’re interested in the efficacy of medications in reducing violent crimes, but most people who take them don’t convict crimes, many people who are diagnosed with mood or psychotic disorders aren’t prescribed medications, and many violent criminals don’t have either mood or psychotic disorders.
But the researchers are experts at this kind of problem, right? If they weren’t, surely their research would never have passed through the kind of rigorous peer-review a journal as preeminent as The Lancet conducts, correct? Well, we’d like to hope so. But no.
The god-awful methods these morons employed are so terrible the only thing more amazing than their sheer idiocy is that a journal like The Lancet published it. The researchers had the kind of access to data most researchers for most research can only dream of. They looked at everybody born in Sweden between 1961 and 1990 and then used the Swedish Prescribed Drug Register (an extremely accurate record of all drugs prescribed in Sweden) to identify individuals treated with mood stabilizers or antipsychotics between the beginning of 2006 to the end of 2009: 40,937 men and 41,710 women (82,647 total).
They then went to the national crime register. To determine the efficacy of antipsychotics and mood stabilizers on violent crime they used rates of convictions of these 80,000+ people from 2006/1/1-2009/12/31. They defined individuals who went without prescriptions for at least 4 months during the entire period as “off treatment”. They compared the rates of those “off treatment” to those in “medication groups”.
Let me make sure this is clear: the actual sample was composed of almost everybody in Sweden who received antipsychotics or mood stabilizers from 2006/1/1-2009/12/31. Despite the fact that the researchers compared “medication groups” to non-medication groups (“off treatment”), there was no actual non-medication group and we have no idea if and when anybody in the sample received meds outside of the 4 years in question.
It gets much worse. Out of the 82,647 people in their sample, guess how many had any violent crime convictions at all? 3,261, or less than 4%. How do you determine the reduction in violent crime convictions for the 79,386 people with 0 convictions? You can’t (at least, without getting more data, such as rates of violent crime convictions overall). The basic research question involves likelihood: among those with certain mental health disorders, how likely is it that one will commit a violent crime on medication compared to one off medication? The researchers found that the probability for violent crime was almost 0, and thus relied on fancy but ill-founded statistical tricks (somewhat hard to evaluate because their actual procedures are buried in various citations rather than explicitly described).
Can we determine that the researchers’ conclusions were unfounded without a detailed explanation of the statistical methods in survival analysis? Well, there’s this: “differences in rates of violent crime were negligible between patients with previous violent convictions (9382 patients) and those without such convictions (73 261 patients)”. REALLY? This nugget alone is more significant than anything obtained by the researchers’ primary method. Why? First, because over HALF of the total convictions were from the group with previous convictions. 2,869 of the 4,948 total no. of violent crimes were committed by 9,382 individuals from that group. That’s about 11% of the total people convicted for ~58% of the total crimes. So, what is by far the biggest predictor of violent crime conviction? Previous violent crime conviction.
Second, if the medication were effective as claimed, we SHOULD see a difference in rates. We can see this from two related findings:
1) A tiny minority of the total sample was convicted of any violent crimes
2) An overwhelming number of convictions were of a tiny minority: those with previous violent crime convictions
In other words, the two groups (those with previous convictions and those without) differed in the extreme along one dimension (probability of violent crime conviction). The researchers suggest that medication is related to the reduction of this probability. Therefore, if the effects of medication don’t differ between the two groups, they don’t differ in any reduction of probability. But, in order to BE effective, they’d have to because one group is drastically more likely to be convicted of violent crimes.
The researchers did not look at people convicted of violent crimes in general or at the general population. Instead, they looked at the rates of convictions for violent crimes among a population composed almost entirely of people without any convictions of violent crimes. The probability for being convicted of any violent crimes, given their population, is so low that they can’t actually test the efficacy of any treatment. It’s like looking at the violent criminal population to find whether pacifists are more likely to be paroled. You can’t look at how anything reduces the rate of something doesn’t happen.