Functional MRI may be less than meets the eye
Functional MRI (fMRI) imaging has been a powerful tool for visualizing processing of information in the brain. This technique is based on the observation that the MRI signal changes with changes in the amount of blood flowing to a particular region of the brain, which correlates with the activity of that brain region. This is a very sophisticated technique that relies on complicated computer algorithms and this is where the problem lies.
A review of the three most popular software processing packages suggested that false-positive results are present in up to 70% of studies, which means up to 40,000 published trials may provide erroneous results. This review was published in the Proceedings of the National Academy of Sciences.
fMRI reports often provide tantalizing details about the effect of emotions, thoughts, drugs, etc on the brain. I searched through my posts and found three “Expect relief and you will get it“, “Botox helps headaches, makes you happier“, and “Science of acupuncture“.
This is not to say that all of this research is worthless. However, I would be skeptical of studies that involve a small number of subjects and from centers not known for rigorous scientific research.
Dear Dr. Mauskop,
Thank you for keeping this interesting blog and sharing academic research with a broad audience.
I just wanted to point out in response to your post that it is important to separate task-fMRI from resting-state fMRI studies.
The paper you mention in this post investigated the effect of cluster-wise inference testing in task-fMRI. They found that indeed parametric cluster-wise methods have an inflated false-positive rate, and they rightfully point out that non-parametric permutation inference testing is to be preferred. Luckily the tools to do this are now available and fairly easy to use.
However, the conclusions of this paper do not extend to resting-state fMRI studies, which also are become increasingly more popular in the field of (clinical) fMRI.
One of the blog posts you refer to, “Science of acupuncture“, indeed performed a resting-state analysis, which does not suffer from the statistical analysis issue raised.
The authors of the “Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates” might have overstated their conclusions, as they felt the need to publish a correction which tones down the strongest claims http://www.pnas.org/content/113/33/E4929.full
However, as always, one should be careful when interpreting studies with small sample sizes and barely significant effect-sizes.
Best wishes,
Dinant Riks
DPhil student in Clinical Neuroscience
FMRIB Centre, University of Oxford