Advertisement

Modern studies published in prominent journals are much more difficult to read and comprehend compared to papers published a century ago, according to an analysis conducted by Karolinska Institute researchers.

The team, from the Institute’s Department of Clinical Neuroscience, analyzed 707,452 article abstracts from 122 high-impact journals from 1881 to 2015. The journals included Nature, Science, the Lancet, JAMA, and others, and the papers were downloaded from PubMed.

They used two established measures of readability—Flesch Reading Ease (FRE) and the New Dale-Chall Readability Formula. These formulas are used to assess how easy or challenging a specific text is to read.

For example, an FRE score of 100 reflects the reading level of a 10- to 11-year old, while a score between 0 and 30 represents a college graduate level.

The formulas count the number of words within a sentence, syllables per word, and how many words are not included in a predefined list of “common” words.

The analysis found that in 1960, 16.3 percent of abstract texts had a score below zero. But by 2015, the rate rose to 26.5 percent of abstracts.

“In other words, more than a quarter of scientific abstracts now have a readability considered beyond college graduate level English,” wrote the authors.

The team validated abstract readability against full text readability, showing that it functions as a suitable approximation for comparing main texts.

“These results are concerning for scientists and for the wider public, as they impact both the reproducibility and accessibility of research findings,” they added.

The study shows that dense, complex vocabulary littered within scientific literature may also be contributing to issues with research reproducibility.

An independent team of scientists should be able to replicate an experiment and observe the same results, but a recent survey by Nature found that 52 percent of researchers believe there’s a “reproducibility crisis” while another 38 percent said there was a “slight crisis.”

Interestingly, the Karolinska Institute study focused only on biomedicine, so there lies an opportunity to see if similar results are replicated in papers related to other scientific fields.

The team was unable to pinpoint direct causes of the steady decrease in levels of readability, but did offer some theories, as well as rule out others.

One could argue that science has become more complex and specialized over the last century, which would create new fields of vocabulary that may not be fully understood across all disciplines. But the researchers noted that the jargon used in the abstracts wasn’t as diverse as they would have expected if this was the case.

However, according to the authors, there has been an increase in “general scientific jargon” or multi-syllable words that don’t actually have any technical meaning, but have become common to use in scientific papers. Words like “robust,” “underlying,” “furthermore” and “significant” were given as examples.

The team refers to this language as “science-ese”—and they argue that while these types of words are relatively well known in our day-to-day lives, the high prevalence of these words in scientific literature increases the mental effort it takes to read through the text.

The study highlights that less than 30 percent of American adults are now scientifically literate. Additionally, recent global multi-year measures show either plateauing or decreasing trends in scientific literacy in children.

The researchers suggest scientists strive to estimate their own readability, and journals could implement more stringent criteria of article readability during the review process.

The study can be found on the pre-print website bioRxiv.

Advertisement
Advertisement