At the onset of the COVID-19 pandemic, when new daily cases were averaging in the thousands in hotspots throughout the United States, a study conducted by a group at Stanford University was published on medRxiv, a popular preprint server for health sciences.
Researchers tested 3,330 adults and children in Santa Clara County, California for antibodies that detect SARS-CoV-2, the virus that causes COVID-19. The presence of antibodies should indicate that a person has previously been infected by the virus. The researchers’ findings suggested that COVID-19 infections in Santa Clara might have been 50-85 times higher than previous estimates. If that’s the case, the COVID-19 fatality rate was only 0.12 percent — no more deadly than the seasonal flu.
The preprint was shared rapidly across social media and news outlets, with many commenters questioning the lethality of the virus. Some conservative groups used the study to minimize the threat of the pandemic and spread misinformation about the virus. The scientific community was quick to voice their objections and point out glaring flaws in the study’s recruitment efforts, testing methods, and conclusions.
Errors in the study could be traced back to faulty antibody tests that too often indicated that the virus was present when it wasn’t, which the authors failed to take into account. So, how did a study with blatant errors attract national attention?
The Rise of the Preprint
The study was a preprint, a scientific manuscript that has not yet undergone peer review or been published in an academic journal. One of the main goals of preprints is to help researchers disseminate scientific work quickly. Preprints have been around for decades but have recently gotten more attention as researchers feel a new urgency to share COVID-19 research as soon as possible.
The outcome of the Stanford preprint highlights the emerging challenges of conducting and sharing scientific research during a global health pandemic.
This was the focus of the most recent meeting of the Berkeley Ethics and Regulation Group for Innovative Technologies (BERGIT). Hosted by the IGI, BERGIT is a meeting ground for collaborations across disciplines to integrate ethics, regulation, and policy with science. Their goal is to provide space for discussion, facilitate opportunities for collaboration, and instigate a proactive cultural shift in responsible innovation.
This semester’s meetings examine science in the time of COVID-19. In the October 2020 BERGIT meeting, IGI Science Communications Specialist Kevin Doxzen highlighted the challenges the scientific community has faced this year: “We decided to focus on how this pandemic may have long-lasting consequences on how science is performed, disseminated, and trusted. These conversations help us latch on to what is currently working and what could be improved.”
Doxzen emphasized how science has moved to the forefront of everyday conversations: dinnertime chatter regularly involves discussing recent mandates from the CDC, the World Health Organization, and other regulatory agencies. Health and disease experts find themselves trending on Twitter. Scientific research has never been more in the public eye.
The October session aimed to critically examine how research makes its way from the lab bench to the evening news and eventually to policymakers. In a pandemic where scientific research is critical for medical innovation and public health policies, how do we balance speed and quality control?
Let’s break it down.
A Case for Preprints
Traditionally, the peer review process is used to vet scientific studies before publication. If a journal is interested in publishing a paper, they send it to independent researchers to review individually. In a process that is typically anonymous to all parties, the researchers assess whether the paper meets specific criteria. In theory, reviewers should act as a filter, making sure that only original, high-quality research gets published, which often includes making recommendations on how to improve the study before publication. But a traditional peer-reviewed article can take months or even a year to be published — time that we can’t afford to lose in a pandemic.
In contrast, a preprint of a research paper can be published in less than five days. The most well-known preprint servers are non-profit databases such as arXiv, SSRN, bioRxiv, and medRxiv.
Rapid data sharing has been a vital component of successful collaboration and innovation during the global health crisis. In our recent memory, when have so many researchers united to study the same subject collectively?
Anneliese Taylor, Head of Scholarly Communication at the UCSF Library, gave the session’s first presentation. As an expert in scholarly publishing, she has witnessed considerable changes in scientific publishing during the pandemic, including a rapid increase in preprints.
Between January and April of this year, 16,000 journal articles and preprints relating to COVID-19 were published across databases. Of those articles, approximately 6,700 were preprints. According to ASAPbio, a non-profit organization promoting the productive use of preprints in the life sciences, from April 2019–April 2020, the publishing of preprints in all areas of biomedical research tripled across all databases.
When researchers can share their data, it allows others to capitalize on their work. During the COVID-19 pandemic, this sharing of data has led to rapid progress in understanding the basic biology of the virus, vaccine development, and creating treatment options. IGI teams have used preprints during the COVID-19 pandemic to help share information that could be of use to other academic institutions, including a blueprint for establishing a pop-up testing lab, and how to optimize surveillance testing in a university environment.
Preprint servers take steps to prevent the misuse of data from studies that haven’t been peer-reviewed. For example, medRxiv prefaces each article with a direct statement: “This article is a preprint and has not been peer-reviewed. It reports new medical research that has yet to be evaluated and so should not be used to guide clinical practice.”
Researchers understand the limits of preprints, and have the skills to read preprints critically. But those outside of academia — including journalists — often lack the background to assess the work themselves.
“The thing that’s so needed when the media reports on [preprints] is context,” says Spencer Hey, a faculty member at the Harvard Center of Bioethics. “It’s important to remember that this study is one out of 1,000 that are going on. The question is not, ‘What does one study show?’ but rather, ‘What does all of the evidence show us?’”
In addition to the complexities of the vetting process, the pandemic has exposed larger, systemic issues in publishing.
Most research lies behind a paywall. Accessing a single article can cost hundreds of dollars. A large portion of scientific research conducted at universities in the U.S. and abroad is funded by taxpayer dollars, leading many to advocate for open access to research. Many publishers have agreed to remove their paywalls and make any research related to the coronavirus, COVID-19 disease, or research on previous pandemics freely available. But how long will this last?
“This is an important effort to help research that needs to be done. However, it does remind us that so much of this content is still owned by publishers and sits behind paywalls,” says Taylor. “We’re really at their mercy and goodwill to make and keep content openly accessible.”
When the pandemic ends, publishers will reclaim research behind their paywalls, inhibiting global collaboration. Researchers will continue to use preprints as a way to reach audiences by circumventing journal paywalls, but maybe there are a few lessons the pandemic has taught us on how to reform science publishing.
Mapping the Way Forward
The increased publication of preprints has encouraged the scientific community to imagine a scientific enterprise that doesn’t rely entirely on the publishing of peer-reviewed papers.
Many of the participants in this discussion brought up the important role of peer review as a gatekeeping function to ensure the validity and quality of scientific research. Taking time to thoughtfully review and revise manuscripts not only ensures accuracy but also gives scientists an opportunity to invite different voices to weigh in.
During the discussion, Hey asked participants, “Can we get to a place where we have the benefits of fast and slow science?” Taking into account the need for serious and thorough review, Hey proposed a scenario to the group: What if protocols and data from experiments could be made available quickly to the scientific community, but researchers could take ample time for analysis?
At first glance, preprints are a great tool to disseminate research quickly, but there may be a better way to accomplish this goal.
Registered reports are a method in which researchers submit their proposed methodology, protocols, and preliminary data for review prior to conducting the actual research. The reports are designed to increase the transparency of published research without calling for a complete overhaul of the current system.
The idea to publish protocols early as a way to increase rigor has been recognized for decades. Recently, The Center for Open Science has helped over 250 journals adopt the practice. Researchers are incentivized to submit high-quality protocols with a provisional promise of publication at a later date.
With the protocols and methods pre-approved, it essentially cuts the peer-review process in half. According to Chris Chambers, Professor at Cardiff University and Chair of the Registered Reports Committee supported by the Center for Open Science, who was not part of the meeting, “[Registered reports] are not a panacea — the format needs constant refinement. It currently sits rather awkwardly between the old world of scientific publishing and the new. Innovations over the next few years should make this format even more powerful and stimulate wider reforms.”
There is no replacement for time and thoughtful oversight in science, but publication methods such as registered reports could offer a bridge allowing for rapid publishing of data with a measured approach to its analysis. It provides authors with the opportunity to share their protocols and preliminary data with other researchers, fostering a culture of rapid collaboration. With only a rationale and proposed methodology required to publish, registered reports may also provide less fuel for those who are likely to misunderstand or misrepresent non-peer-reviewed findings.
We don’t know how the pandemic will change publishing moving forward, but it has brought clear issues to the forefront. We will always be balancing speed and safety in an equation that cannot be solved perfectly. But the urgency brought about by the pandemic is leading to global collaborations and new patterns of disseminating research that favor speed. Preprints, registered reports, and increased access to scientific research are providing new avenues for communities to vet research faster, which could have lasting positive effects beyond the pandemic.
To receive news about the next BERGIT meeting and take part in these crucial conversations, join the BERGIT email list.
