Last February, some media outlets reported on a study by two Chinese universities according to which the coronavirus SARS-CoV-2, which causes COVID-19, “probably originated in a laboratory in Wuhan”. The news made headlines around the world, bolstering the case for conspiracy theorists about the origin of the virus. However, such a study did not actually exist; it was a little more than a one-page document posted on the Internet, whose two authors —with affiliations to a pair of Chinese universities— limited themselves to speculations based on the existence of two laboratories in Wuhan that study coronaviruses. Under normal circumstances, such a document, closer to a school paper than a scientific study, would likely have passed unnoticed. However, in the pandemic era, we are also subject to an infodemic, which is partly caused by the flood of data and studies —the one cited is an extreme but illustrative case— that in many cases are disseminated without due safeguards.
Even in these times of “publish or perish”, when scientists are under increasing pressure to publish more papers and of a higher quality if they want to survive, good science still requires time. A scientific study, like wine, is a product that requires repose and maturation. When researchers finish writing their work, there is still a long way to go. From the time a study is first submitted to a scientific journal until it is finally published on its pages, months or even years can pass; if not initially rejected, the work must still undergo a lengthy peer review process that turns a paper into science accepted by the community. Even in these cases, a number of studies end up being retracted some time later, if fraud or simply bad science is discovered.
More speed
But the emergence of the pandemic does not sit well with the leisurely pace of scientific publication. In response to the urgent need to release research that may reveal useful clues about the coronavirus and its disease, many journals have accelerated their peer-review processes so that the growing flood of studies on COVID-19 can see the light of day as soon as possible. According to an analysis by science and society expert Serge Horbach at Radboud University in the Netherlands, 14 medical journals have reduced the publication time of pandemic-related studies by an average of 49%, especially the peer review process. “Clearly, some of the changes are very laudable,” Horbach tells OpenMind.
Researchers are grateful for this increased tempo: “The speed of publication is evident when you look at the submission and approval dates of articles, even in large journals. I wish it were always like this!” microbiologist and immunologist José Villadangos, from the Peter Doherty Institute at the University of Melbourne, Australia, told OpenMind. However, as molecular biologist and science communicator José Miguel Mulet, from the Institute of Molecular and Cellular Biology of Plants (Spain), points out to OpenMind, “this has two sides. The good one is that information flows very fast. But the downside is that there is little filtering and, along with good studies, bad or false papers can slip in that only contribute to creating noise.”
This possibility that a more expeditious process may lessen the quality of published science is of concern to a number of scientists. In the journal Science, science ethics experts Alex John London and Jonathan Kimmelman warn that a crisis should not be an excuse to reduce scientific standards, as poor clinical studies in particular, without proper protocols and controls, can lead to harmful consequences.
There has been no shortage of examples of this during the current pandemic. One of the most well-known is that of hydroxychloroquine and chloroquine, anti-malarial drugs that in preliminary data appeared to provide some benefit to patients with COVID-19, leading prominent figures such as Donald Trump to endorse and defend their use. Now, one of the studies that proposed this treatment has been withdrawn, another is being investigated by the journal that published it, and recent research on more than 96,000 patients has determined that not only are its benefits non-existent, but that it can actually increase mortality. A review of the available trials has found that the evidence for the effects of chloroquine and hydroxychloroquine is “weak and conflicting“. For this reason, the World Health Organization has stopped its trials of these drugs.
One part of the problem: preprints
Part of the problem with the quality of certain studies can be summarized in one word: preprints. These are works that have not yet passed a peer review process in a journal, but are made available to the community through freely accessible repositories on the Internet, such as bioRxiv, medRxiv or chemRxiv. This system, traditionally used as a channel for scientific discussion in certain fields such as physics or mathematics, used to play a marginal role in the biomedical sciences, but the urgency of the pandemic has made it a fast track for the dissemination of studies. This has its risks: the retracted chloroquine study was a preprint, as was another one that claimed to find an “uncanny similarity” between the coronavirus and HIV, sparking much confusion.
But it must be said that of the studies retracted so far on COVID-19, there are more formally published ones than preprints, according to the Retraction Watch website. In the journal Nature, those in charge of the main repositories defend themselves by claiming that they have intensified their screening controls to ensure that the studies meet a minimum quality standard. What’s more, in contrast to the traditional publication system, which is slow, opaque and economically costly for researchers who wish to access published science, there is no lack of defenders of the virtues of preprints: open science, free and with the opportunity for transparent online peer review, a system that has been applied in some digital journals.
“Open peer review on preprints offers opportunities that closed review does not,” policy and science communication expert Danny Kingsley from the Australian National University told OpenMind. “Having open feedback allows for discussion and refinement of the work in a collaborative manner. Some papers on arXiv have gone through multiple iterations so are very much ‘peer reviewed’, and the reader can see every earlier version of that process. Why would you bother then sending it to a journal?”
Open science
Kingsley also points to a curious trend, which is that the publishers of science journal are trying to take advantage of the rise of the preprint phenomenon by setting up their own repositories as part of the publication process for their studies. “Publishers never had control of preprints —they were always an academy-led initiative. What publishers are doing is trying to wrest control from the academy,” the expert explains. This is now one of the new fronts of a major struggle that has been going on for years, that of the traditional system of publication versus so-called open science. Numerous publishers and journals have joined an initiative promoted by Wellcome to share relevant data and findings related to COVID-19, including the publication of studies in open format. But for Kingsley, it is not enough to share the studies, but “also the data that underpins the claims made in the articles. Sharing the data as well as the article starts to move us towards true open research.”
In short, the COVID-19 pandemic has sparked an earthquake in the science system. According to Kingsley, “most decidedly the ecosystem of scientific publication is changing.” But we are living in exceptional times, and the question is: will these changes continue beyond the pandemic? Will they serve to improve the system, taking full advantage of the benefits of the preprints route —open science and faster publication— without a deterioration in quality? For Horbach, it is too early to say for sure, but he does believe we will see long-term effects. “The usage of preprint servers as a fast outlet for knowledge dissemination is a more general shift that I think might have a lasting effect,” he says. “It seems likely that this will continue well into the future.”
Javier Yanes
@yanes68
Comments on this publication