Why Colin Firth should stick to acting, not neuroscience

Some tips for other BBC employees interested in authoring peer-reviewed journal articles without having a clue, taken from
Kanai, R., Feilden, T., Firth, C., & Rees, G. (2011). Political orientations are correlated with brain structure in young adults. Current biology, 21(8), 677-680.
How to hide behind technical and mathematical jargon by flooding your “methods” section with it, even if this means lies

Lie 1

In their “Results & Discussion” section, the authors state they used “political attitudes [self-reported] confidentially on a five-point scale from very liberal to very conservative. We then…investigate[d] the relationship between these attitudes, expressed as a numeric score between one and five.” (emphasis added)

Only, they didn’t express anything as a “numeric score” of 5 (or as between 1-5 exclusively, because they included 1). In their methods section they state: “Because none of the participants reported the scale corresponding to very conservative, the analyses were conducted using the scales of 1, 2, 3, and 4.” (emphasis added)

Lie 2

So how do we determine what they actually did to get their results? We go to the methods (experimental procedures) section and look. As soon as we get beyond the simple ratings and into the analysis, almost everybody would skip back to the conclusions: “”T1-weighted MR images were first segmented for grey matter and white matter using the segmentation tools in Statistical Parametric Mapping 8 (SPM8, SPM – Statistical Parametric Mapping). Subsequently, we performed diffeomorphic anatomical registration through exponentiated lie algebra in SPM8 for intersubject registration of the grey matter images.”

Wow. Sounds amazingly complicated. In fact, it’s so complicated the researchers don’t know what it means. The first part, in which they “segmented” the images into white and grey matter using SPM8 means that they used a program and developed by others so that people like them can press a button and hey-presto segmentation complete. They don’t have to know anything about what’s involved. In fact, it is set-up so that they won’t (most of the manual, the help files, etc., available on the site they linked to do not explain what’s going on or what the processes mean, only how to do them).

Next, they get really fancy (it seems). They “performed diffeomorphic anatomical registration through exponentiated lie algebra”. But fear not! You don’t have to learn what any of that means, because not only did the researchers not learn it, they didn’t use it.

To the normal human, “Lie algebra” sounds likes a program you might use to cheat on a high school math test. In reality, it has to do with a set of related mathematical notions (fields, vector spaces, rings, groups, etc.) which might be grouped under the name “abstract algebras”.

And it sounds like the researchers did something really fancy. They didn’t. “Diffeomorphic anatomical registration through exponentiated lie algebra” comes from the acronym DARTEL, which is part of the SPM8 software package. What does the SPM8 manual say about this acronynm? “DARTEL stands for “Diffeomorphic Anatomical Registration Through Exponentiated Lie algebra”. It may not use a true Lie Algebra, but the acronym is a nice one.”

The people who designed DARTEL decided they liked the acronym even though it isn’t accurate. But it doesn’t sound impressive to say “we then used SPM8’s DARTEL” when you can say instead “we performed diffeomorphic anatomical registration through exponentiated lie algebra”, even if this isn’t true.

Lie 3

A useful way to hide your conclusions is to cite a bunch of sources for one point (or a few related points). The authors did: “One of the functions of the anterior cingulate cortex is to monitor uncertainty [16,17] and conflicts [18]”

Good to know! Now we have a rational, sound, and perfectly reasonable basis for thinking that the conclusions show anything about this function. Or not.

Source number 17 there is the study “Optimal decision making and the anterior cingulate cortex” from Nature Neuroscience. It concludes that the primary role of the ACC is not to monitor uncertainty, but “integrating reinforcement information over time rather than in monitoring”. This study is basically contradicting the view that the ACC “monitor[‘s] uncertainty.”

Even better, study 17 cites study 18 in order to say that it is incorrect. The authors cited two studies, one after another, when the first one is devoted to demonstrating that the second is wrong.

Lie 4

If you can’t hide behind your sources, manipulate citations. The authors “characterized the extent to which these correlations between gray matter volume and political attitudes might permit us to determine the political attitudes of a single individual based on their structural MRI scan” using a mathematical tool called a “support vector machine algorithm”. Usually, when researchers use some statistical technique that is more complicated than the most basic bivariate correlation measures or linear regression models (and sometimes even then) they cite a source or sources to indicate that this technique has been used like this before. Here, the authors cited a 1998 book Statistical Learning Theory which contains several chapters on various types of support vectors and their uses, and so it is unclear what algorithm the authors used.

Which brings up the researchers’ knowledge of statistics and math. Apart from the lying about using a lie algebra, the fact that their knowledge is deficient is also indicated by the citation of that textbook. Usually, citations like this are to a specific study, often to the study in which the mathematical tool was developed. Such citations are important because they

1) show what exactly the researchers did

2) show why it is an appropriate method.

But we don’t know the method. What they said is equivalent to “we used statistics” or “we used machine learning”. They cited a textbook. No page number. No indication of what algorithm they used or why it is appropriate. This defeats the entire point of such citations, but it does indicate that the researchers didn’t really understand what it was they did. They used MATLAB, in which such methods can be done by anybody, as all you need is numbers and you can enter in some commands and WHAM out pops your value. So, they went to a textbook, found some classification method they figured probably was ok (or maybe tried a bunch and settled on the one that gave the right result, which is something I’ve seen done), and used it. But as they didn’t really know what they were doing, and had to indicate why it was they used this method, they cited the entire textbook. They hid what it was they did.

Advertisements
This entry was posted in Uncategorized and tagged , , , , , , . Bookmark the permalink.

2 Responses to Why Colin Firth should stick to acting, not neuroscience

  1. Jeff says:

    Mathematical Propaganda… sounds like propaganda anyways.

    • “There are lies, damned lies, and”…any and all use of mathematics in most of the sciences most of the time. But we have RULES about when, where, how, and why one can rely on such propaganda, and violations of such rules are hideous sins because they risk exposing the rest of us as frauds.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s