According to a UK’s Sunday Express report (“Scientists cure phobia of dentists – by sending tiny electrical currents into the brain”), “MILLIONS of people could be cured of their fear of dental surgeries with the help of electrotherapy”. Amazing! Or at least it would be, were it true. In yet another stunning example of bad reporting on worse research, we find that both the claims in the report and the study are underwhelming.
First, the actual study (Koleoso et al.’s “The Role of Relaxation Therapy and Cranial Electrotherapy Stimulation in the Management of Dental” published in IOSR Journal of Dental and Medical Sciences) didn’t just examine the efficacy of “electrical currents” on anxiety. True, two of their three treatment groups did receive cranial electrotherapy stimulation (CES), but one group received only “relaxation therapy” (the third received both, and a fourth “control” group received no treatment; each of the four groups consisted of ten participants). However, the researchers found no differences between the three treatment groups, meaning that the reduction in anxiety for those who simply had a therapy session was basically the same as those who received CES as well as those who received BOTH treatments. So, this amazing treatment with the potential to cure millions is as effective as talk therapy-but much more expensive.
Second, the report states that “electrical currents stimulate nerve cells in the brain which produce the chemicals serotonin and acetylcholine. These reduce stress and agitation and increase calmness and concentration.” This isn’t true. These neurotransmitters are implicated in various attention, anxiety, similar emotional/psychological states, in part because e.g., serotonin is “implicated” in just about every cognitive, psychological, and behavioral process that humans experience. However, ignoring for now our ignorance of causal relations between e.g., serotonin production and anxiety, what might it mean for acetycholine or serotonin to “reduce stress and agitation”? Does that mean more of it reduces stress? Well, some of the earliest experiments which involved inducing anxiety/panic in the lab found that injecting patients with acetylcholine caused panic. Yet, increases in acetylcholine are also implicated in anxiety/panic reduction. How can this be? Easily: the effects of the most abundant neurotransmitters vary widely for many reasons. Acetylcholine, for example, is found all throughout the body because it allows you to use your muscles for rare actions like moving, standing, sitting, etc. Also, states of arousal, from sexual arousal to panic, are not only related but are partly interpreted. The increased heart-rate, quickened breath, etc., that one feels in a scary situation can be experienced as sexual arousal if someone of whatever gender one is attracted to is nearby (this was actually shown experimentally in a fascinating study years ago). Finally, even differentiated states aren’t necessarily unrelated. A small amount of test anxiety improves performance partly because this entails increased focus and attention to detail. Basically, any neurotransmitter that is involved in increased anxiety is going to be involved in increased attention. So one could just as easily say that acetycholine increases anxiety and decreases attention.
But what about the actual study? The researchers used a sample of 40 participants out of a larger group of individuals with “chronic oral pain” who all “scored high on [a] dental anxiety scale”. Already the astute reader may see a potential confounding factor. The existence of a chronic dental problem is likely to create an abnormally high level of anxiety associated with going to the dentist. Such people have to go more often and for procedures and/or treatments that differ from your average check-up. But ok, let’s ignore that. Seeing as though the researchers didn’t identify differences among any of the three treatments (the third, again, being both of the other treatments), how did they determine whether they treatments had any real effects? They had a control group. In other words, 40 people took a test that supposedly showed how much anxiety participants experienced related to dental appointments, 30 of them subsequently came in for three days for treatment, and then all 40 came back to take the test again. Is there any good reason for thinking the results were not a placebo effect? No.
Actually, there isn’t really any good reason for thinking that the results were at all meaningful. The scale used consisted of 5 questions each with a 5 point scale (i.e., a question might be “how would you feel if you had to go to the dentist’s tomorrow?” and the possible answers might be “probably very anxious”, “probably anxious”, “possibly anxious”, “possibly fine”, “probably fine”). The important point is that answers are assigned numerical values 1-5, but the answers aren’t actually numbers. There are a number of statistical methods designed to allow researchers to treat ordered responses like this as numbers (some bad, some better), but the researchers didn’t use these. They used statistical tests which assume (among other things) that the possible responses to each question are uncountably infinite. Put more simply, an answer could correspond exactly to the number π, the square root of 2, .5897204954728054578452095355624, or any other number in some interval of the real numbers. Between any two real numbers, say .55555 and .55556, there are infinitely many other numbers. In reality, though, every question had only 5 responses and none of them were actually numbers. The small number of questions has a similar effect: it’s harder to identify “significant” differences because there aren’t enough ways for participants to differ. The fact that there were only 40 participants makes this even worse.
At best, then, the researchers found that an expensive shock treatment was no better than talk therapy, and at worst their results are worthless (true, results can always be worthless no matter how well an experiment is designed and carried out, but in this case it’s highly likely to be worthless). So why is this newsworthy? Because anything is when you lie about it to make it seem important. When it comes to science, such lies are called “science reports” or “scientific reporting”.