Suppose you were a drug company, and you've invented a new drug. It's OK, but it's no better than the competition. How do you convince people to buy it?

You need a selling point - something that sets your product apart. Fortunately, with drugs, you have plenty of options. You could look into the pharmacology - the chemistry of how your drug works in the body - and find something unique there. Then, all you need to do is to spin a nice story to explain how the pharmacological properties of your drug make it brilliant.On an entirely unrelated note, aripiprazole (Abilify) is an antipsychotic marketed in the US by Bristol Meyers-Squibb. A Cochrane meta-analysis finds that it's about as good as any other antipsychotic in terms of efficacy and side effects. As good, but no better. However, uniquely, aripiprazole is a D2 receptor partial agonist. Other antipsychotics work by blocking D2 receptors in the brain, switching them off (full antagonism). Aripiprazole also blocks D2 receptors, but it activates them slightly in the process (partial agonism).

Is that a good thing? A paper just published says yes - The relationship between subjective well-being and dopamine D2 receptors in patients treated with a dopamine partial agonist and full antagonist antipsychotics. The research in question was funded, by the way, by Bristol Meyers-Squibb. Let's see if it holds up.

The authors got 22 patients with schizophrenia who were taking an established antipsychotic, either olanzapine or risperidone. These are both D2 antagonists. Incidentally, neither of them is made by Bristol Meyers-Squibb. 11 of the patients were switched to aripiprazole, while 11 stayed on their original drug. There was no blinding, and no randomization. (The dose of aripiprazole was randomized, although still unblinded, but the assignment to aripiprazole itself wasn't).
Lo and behold, the patients who switched reported improved "well-being". Because there was no randomization and no blinding, and because the outcome was entirely subjective, this could be entirely explained as a placebo effect (or an experimental demand effect.) Especially when you consider that the patients were most likely convinced to take part in the study by being told that aripiprazole would make them feel better than their original drug.

That's not all, though. They also did some brain scanning, using PET to measure D2 receptor occupancy. On average aripiprazole blocked more D2 receptors than the other antipsychotics, which is what you'd expect, as it has a very high affinity for that receptor. But it's a partial agonist, remember - it binds to D2 receptors without switching them "off" entirely.

The paper suggests that this is a good thing because it doesn't make people feel horrible, which is what normally happens when you block almost all of someone's D2 receptors (they're rather important). By switching on the receptors as well as blocking them, it makes you feel OK.
Nice story, and scientifically it's not unreasonable. And as you can see on this plot, in the non-aripiprazole patients (triangles), D2 occupancy in the ventral striatum was negatively correlated with well-being, but in the aripiprazole patients (circles), it wasn't.

Great - except that the range of D2 occupancies in the aripiprazole group is so narrow that no correlation would be apparent even if there was one. The occupancies in the aripiprazole group are all extremely high, 80-95%. There's just no room for a correlation to appear. (Think about it this way - in children, age is strongly correlated with height, but if you only looked at a bunch of 7 year olds, you wouldn't know that.) This is high-school statistics.

This scatter-plot is in fact exactly what you'd expect assuming that a) aripiprazole strongly blocks D2 receptors, and makes people feel awful, just like any other strong D2 blocker and b) the placebo effect made some of the aripiprazole group feel (or at least say that they feel) a bit better than they otherwise would.

You'll note also that the aripiprazole group reported feeling no better than the other antipsychotic group, and that the single most miserable patient was on aripiprazole. The paper concludes on an optimistic note -

Children with autism often shown repetitive behaviours, ranging from repeated movements to compulsively collecting or arranging objects and desiring that daily routines are always done in the exact same way. Repetitive behaviour is often considered one of the three core features of autistic disorders (alongside difficulties in social interaction, and difficulties in communication).

SSRI antidepressants are often used to try to treat repetitive behaviours. Unfortunately, they don't work, at least according to a new study - Lack of Efficacy of Citalopram in Children With Autism Spectrum Disorders and High Levels of Repetitive Behavior.

The trial included 149 American children with autism aged from 5 to 17 years old, all of whom had moderate or severe repetitive behaviours. They were randomly assigned to get either citalopram, an SSRI, or placebo, and were followed up for 12 weeks to see if it had any effect on their repetitive behaviours. The dose of citalopram started at 2.5 mg and gradually increased to, in most cases, 20 mg, which is the dose that an adult person with depression would most commonly take - for a kid, this is a high dose.

The results were unequivocal - citalopram had absolutely no benefit over placebo. Zilch. On the other hand, it did cause side effects in some children - gastrointenstinal problems like diarrehea, skin rashes, and, most worryingly, hyperactivity - "increased energy levels", insomnia, "Attention and concentration decreased", and so forth. (Two children in the citalopram group also experienced seizures, but it's not clear that this was related to the drug, as citalopram is not known for causing seizures in adults.)

So, citalopram was not just useless, but actually harmful, in these children. This is the largest trial of an SSRI for repetitive behaviours in autism so far; there have been a few others, including one double-blind study of Prozac finding some benefit, but this is by far the most compelling.

But there's a big question here - why would anyone think that citalopram would work? Citalopram was designed to treat adults with... depression. Hence why it's called an antidepressant. Depression in adults is no more like compulsive behaviour in autistic children than is having a broken leg or heart disease. They're completely different conditions.

The main reason why SSRIs are used to try to treat repetitive behaviour is that they also work quite well against obsessive-compulsive disorder (OCD). People with OCD have repetitive behaviours, "compulsions". They might wash their hands ten times after going to the toilet. Or check that the fridge door is closed and the oven is switched off every time they leave the kitchen. Or count up to one hundred in their head whenever they see the number 13. And so forth.

SSRIs do work against OCD. Does this mean that they ought to also work against the repetitive behaviors in autism? Only if you think all repetitive behaviours are the same, with the same causes.

People with OCD feel compelled to perform their ritualistic behaviours as a way of coping with their "obsessions" - intrusive, unpleasant thoughts that they can't otherwise get out of their heads. Someone might be obsessed with the thought of germs and disease whenever they go to the toilet, and the only way to feel clean is to wash their hands 10 times. They might be obsessed with the idea that their family will die whenever they see the unlucky number 13, unless they "cancel it out" by counting to 100. The repetitive behaviours, in other words, are a consequence of the obsessions, which are unwanted, anxiety-provoking thoughts. SSRIs probably work by making the obsessions seem less troubling, so there is less need for the compulsions.

People with autism are often described as having "obsessions" too, but in the sense of "Things they are very interested in", not "Thoughts they cannot get rid of". Likewise, autistics may show "compulsive behaviours", but not as a way of dealing with obsessions. The words are the same, but the reality is different.

Maybe autistic people just like sameness and routine. That's part of who they are, and it's not something that can be treated with drugs. People with OCD hate having it - they don't like their obsessions or compulsions, they feel stuck with them. The compulsions are a coping mechanism. But in autism, at least most of the time, that's not how it works. An autistic child "compulsively" playing with the same toy over and over, or reading yet another book about their "obsession", dinosaurs, may be perfectly happy. In which case, why give them happy pills? And this is what the authors of the paper eventually suggest -

It may be that the repetitive behavior in children with ASDs is fundamentally different from what is observed among children with obsessive-compulsive disorder in its behavioral picture and in its biologic underpinnings.

Much of science, and especially neuroscience, consists of the search for "positive results". A positive result is simply a correlation or a causal relationship between one thing and another. It could be an association between a genetic variant and some personality trait. It could be a brain area which gets activated when you think about something.


It's only natural that "positive results" are especially interesting. But "negative" results are still results. If you find that one thing is not correlated with another, you've found a correlation. It just happens to have a value of zero.

For every gene which causes bipolar disorder, say, there will be a hundred which have nothing to do with it. So, if you find a gene that doesn't cause bipolar, that's a finding. It deserves to be treated just as seriously as finding that a gene does cause it. In particular, it deserves to be published.

Sadly, negative results tend not to get published. There are lots of reasons for this and much has been written about it, both on this blog and in the literature, most notably by John Ionnidis (see this and this, for starters). A paper just published in Science offers a perfect example of the problem: Neural Mechanisms of a Genome-Wide Supported Psychosis Variant.

The authors, a German group, report on a genetic variant, rs1344706, which was recently found to be associated with a slightly raised risk of psychotic illness in a genome-wide association study. (Genome-wide studies can and do throw up false positives so rs1344706 might have nothing to do with psychosis - but let's assume that it does.)

They decided to see whether the variant had an effect on the brains of people who have never suffered from psychosis. That's an extremely reasonable idea, because if a certain gene causes an illness, it could well also cause subtle effects in people who don't have the full-blown disease.

So, they took 115 healthy people and used fMRI to measure neural activity while they were doing some simple cognitive tasks, such as the n-back task, a fairly tricky memory test. People with schizophrenia and other psychotic disorders often have difficulties on this test. They also used a test which involves recognizing people's emotions from pictures of their faces.
They found that -

Regional brain activation was not significantly related to genotype...Rs1344706 genotype had no impact on performance.

In my last-post-but-one I slammed the claim that the British are suffering from an epidemic of anxiety disorders. I declared it a myth pushed by the Mental Health Foundation and echoed uncritically by British newspapers (although The Economist has since run a kind-of skeptical piece on it.) But I also promised that there are important lessons to be learned here. So, here we go:

The Mental Health Foundation produced a report, In The Face of Fear, which contains various interesting thoughts about the role of fear in public debates. Here's just one:

Individually we experience both rational and irrational fears that drive our behaviour and fear also drives communities and social policies... Excessive fear poses an enormous burden on our society directly through anxiety related illness, which can be physical as well as mental, and indirectly through inappropriate behaviours such as excessive supervision of children or failure to invest. It also paralyses long term rational planning to deal with key future threats such as global warming by diverting attention to more immediate but less important fears.

Ed Vul et al recently created a splash with their paper, Puzzlingly high correlations in fMRI studies of emotion, personality and social cognition (better known by its previous title, Voodoo Correlations in Social Neuroscience.) Vul et al accused a large proportion of the published studies in a certain field of neuroimaging of committing a statistical mistake. The problem, which they call the "non-independence error", may well have made the results of these experiments seem much more impressive than they should have been. Although there was no suggestion that the error was anything other than an honest mistake, the accusations still sparked a heated and ongoing debate. I did my best to explain the issue in layman's terms in a previous post.

Now, like the aftershock following an earthquake, a second paper has appeared, from a different set of authors, making essentially the same accusations. But this time, they've cast their net even more widely. Vul et al focused on only a small sub-set of experiments using fMRI to examine correlations between brain activity and personality traits. But they implied that the problem went far beyond this niche field. The new paper extends the argument to encompass papers from across much of modern neuroscience.

The article, Circular analysis in systems neuroscience: the dangers of double dipping, appears in the extremely prestigious Nature Neuroscience journal. The lead author, Dr. Nikolaus Kriegeskorte, is a postdoc in the Section on Functional Imaging Methods at the National Institutes of Health (NIH).

Kriegeskorte et al's essential point is the same as Vul et al's. They call the error in question "circular analysis" or "double-dipping", but it is the same thing as Vul et al's "non-independent analysis". As they put it, the error could occur whenever

data are first analyzed to select a subset and then the subset is reanalyzed to obtain the results.

All British journalists are psychotic. Pathologically obsessed with "mental health issues", and suffering from grandiose delusions of their competence to discuss them, these demented maniacs...

Sorry. I got a bit carried away there. But you'll forgive me, because I was just following the example of seemingly everyone in the British media these past couple of weeks. If you believe the headlines, we're in the grip of an epidemic of anxiety:

The Telegraph: Britons 'living in fear' as record numbers suffer from anxiety

The Independent: Britain is becoming a more fearful place – and the economy is paying the price. The Indie also ran a comment by Janet Street-Porter - "The main reason people feel anxious is loneliness.", thanks Janet, qualifications: none, career path: fashion journalist - and a piece by a clinically anxious person - "I reckon a root cause of my anxiety is the modern notion that we can do away with risk by anticipating every imaginable danger."

more than twice as many of us agree that people in general and the world itself are becoming more frightened and frightening as agree that they themselves are more frightened and anxious

Just when you thought it was safe to compute a correlatation between a behavioural measure and a cluster mean BOLD change...

The fMRI voodoo correlations controversy isn't over. Ed Vul and collegues have just responded to their critics in a new article (pdf). The critics appear to have scored at least one victory, however, since the original paper has now been renamed. So it's goodbye to "Voodoo Correlations in Social Neuroscience" - now it's "Puzzlingly high correlations in fMRI studies of emotion, personality and social cognition" by Vul et. al. 2009. Not quite as catchy, but then, that's the point...

Just in case you need reminding of the story so far: A couple of months ago, MIT grad student Ed Vul and co-authors released a pre-publication manuscript, then titled Voodoo Correlations in Social Neuroscience. This paper reviewed the findings of a number of fMRI studies which reported linear correlations between regional brain activity and some kind of measure of personality. Vul et. al. argued that many (but by no means all) of these correlations were in fact erroneous, with the reported correlations being much higher than the true ones. Vul et. al. alleged that the problem arose due to a flaw in the statistical analysis used, the "non-independence error". For my non-technical explanation of the issue, see my previous post, or go read the original paper (it really doesn't require much knowledge of statistics).

Vul's paper attracted a lot of praise and also a lot of criticism, both in the blogosphere and in the academic literature. Many complained that it was sensationalistic and anti-fMRI. Others embraced it for the same reasons. My view was that while the paper's style was certainly journalistic, and while many of those who praised the paper did so for the wrong reasons, the core argument was both valid and important. While not representing a radical challenge to social neuroscience or fMRI in general, Vul et. al. draws attention to a widespread and potentially serious technical issue with the analysis of fMRI data, one which all neuroscientists should be aware of.

That's still my opinion. Vul et. al.'s response to their critics is a clearly worded and convincing defense. Interestingly, their defense is in many ways just a clarificiation of the argument. This is appropriate, because I think the argument is pretty much just common sense once it is correctly understood. As far as I can see the only valid defence against it is to say that a particular paper did not in fact commit the error - while not disputing that the error itself is a problem. Vul et. al. say that to their knowledge no accused papers have turned out to be innocent - although I'm sure we haven't heard the last of that.

Vul et. al. also now make explicit something which wasn't very clear in their original paper, namely that the original paper made accusations of two completely seperate errors. One, the non-independence error, is common but probably less serious than the second, the "Forman error", which is pretty much fatal. Fortunately, so far, only two papers are known to have fallen prey to the Forman error - although there could be more. Go read the article for more details on what could be Vul's next bombshell...

EDWARD VUL, CHRISTINE HARRIS, PIOTR WINKIELMAN, AND, & HAROLD PASHLER (2009). Reply to comments on “Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition” Perspectives in Psychological Science