The newly developed metric incorporates several factors, including the diversity of the paper authors (in terms of gender and geographic location), diversity of the patients studied, and how interdisciplinary the research team is. Credit: Jose-Luis Olivares/MIT
Key points:
- Researchers have proposed an alternative to journals’ impact factor—the diversity factor.
- MIT researchers used three criteria to establish the new diversity factor.
- The team says the new system better captures the impact of medical papers on health outcomes for all patients, including those in low- or middle-income countries, and eliminates blind spots.
Scientific journals and research papers are evaluated by their “impact factor,” which is based on how many times a given paper is cited by other papers. However, a new study from MIT and other institutions suggests that this measure does not accurately capture the impact of medical papers on health outcomes for all patients, particularly those in low- or middle-income countries.
To more fully capture a paper’s impact on health, metrics should take into account the demographics of the researchers who performed the studies and the patients who participated in them, the research team says. To that end, they have developed a metric they call the “diversity factor.” In a new study published in PLOS Global Public Health, researchers evaluated more than 100,000 medical papers published in the last 20 years and found that most did not do well on this proposed diversity factor.
“What happens when all of the authors involved in a project are alike is that they’re going to have the same blind spots. They’re all going to see the problem from the same angle. What we need is cognitive diversity, which is predicated on lived experiences,” said Leo Anthony Celi, a senior research scientist at MIT’s Institute for Medical Engineering and Science and a physician at Beth Israel Deaconess Medical Center.
The proposed metric incorporates several factors. One is the diversity of the authors of a study, including whether the authors are located in a high-income country or a low- and middle-income country. The second is the diversity of departmental affiliations of the authors of a study. Under this metric, papers are given a higher score if they include authors from a wider range of disciplines. The final metric is based on the diversity of traits of patients included in a given study, including sex, race-ethnicity, language, age, and geography.
Celi emphasizes that the metrics chosen for this study should be considered only a starting point for measuring impact on health outcomes in a more equitable way.
“What we wanted is to start a public dialogue on this topic, and we also want the community to contribute and recommend what the metrics should be,” he says.