fMRI Pioneer Casts Doubt on Technology, 15 Years of His Own Work

  • <<
  • >>

565001.jpg

 

Have you ever had a hard timing admitting you were wrong? If so, you’re only human and one of many—which makes what Duke University professor Ahmad Hariri just did all the more impressive.

For the better part of 15 years, Hariri, a professor of psychology and neuroscience, has been using functional MRI (fMRI) technology to predict a person’s patterns of thoughts and feelings during specific mental tasks. Unfortunately, Hariri started to see unreliable results across individuals when they were tested more than once. He and his colleagues then reexamined 56 published papers based on fMRI data to gauge their reliability across 90 experiments. The results were not favorable.

“The correlation between one scan and a second is not even fair, it's poor,” Hariri said of his new analysis, published in Psychological Science. “[These results are] more relevant to my work than just about anyone else! This is my fault. I'm going to throw myself under the bus. This whole sub-branch of fMRI could go extinct if we can't address this critical limitation.”

In addition to the 56 papers, Hariri and his colleagues examined data from the Human Connectome Project and their own generated as part of the Dunedin Multidisciplinary Health and Development Study.

Launched in 2009, the goal of the Human Connectome Project was to build a network map, or connectome, that sheds light on the anatomical and function connectivity within the healthy human brain, as well as to produce a body of data that can help further research on brain disorders like schizophrenia and Alzheimer’s Disease. Completed in 2018, the Human Connectome Project is to neuroscience what the Human Genome Project is to biologists.

According to Hariri’s analysis of Human Connectome Project data, reproducibility of fMRI results for the same person was “weak.” The team concluded that, for six out of seven measures of brain function, the correlation between tests taken about four months apart was poor. The seventh measure, language processing, showed a “fair” correlation.

The researchers found a similarly poor correlation when they looked at their own data collected through the Dunedin Multidisciplinary Health and Development Study in New Zealand. In this project, 20 people were put through task-based fMRI twice, two or three months apart. Upon reanalysis, Hariri observed weak correlation from one test to the next per individual.

“It's not as if we haven't known these issues of reliability, but this paper brings them together more sharply,” said Russell Poldrack, professor of psychology at Stanford University, who authored a 15-year-old fMRI paper that was among those Hariri reanalyzed. “This is a good wakeup call, and it's a marker of [Hariri’s] integrity that he's taking this on.”

Still, fMRIs are effective for specific research applications, and Hariri is committed to exploring how the technology can be furthered in the future. Currently, fMRIs are the standard for finding and understanding general brain structures involved in a given task across a group of people, such as 50 people trying to remember a name. For the future, Hariri believes developing new tasks from the ground up with the explicit purpose of reliably measuring individual differences in brain activity could be a solution to enhance reproducibility. Another possible solution, he says, exploits existing technology to scan and collect data from individuals over a full hour or longer, rather than just five minutes.

In the meantime, further fallout from these findings is expected within the resonance imaging field.

“There are three things you can do,” Poldrack said. “You can just up and quit, you can stick your head in the sand and act as if nothing has changed, or you can dig in and try to solve the problems.”

The third option is the hardest—and to Hariri’s credit, that’s the road he has decided to travel.

Photo: Brain scans showing functional MRI mapping for three tasks across two different days. Warm colors show the high consistency of activation levels across a group of people. Cool colors represent how poorly unique patterns of activity can be reliably measured in individuals. Credit: Annchen Knodt, Duke University


Related Product Categories