Skip to main content
SearchLoginLogin or Signup

Disconnecting the evaluation of scientific results from their diffusion

By working with specific scientific communities, PCI is decoupling peer review from publication to disseminate research faster and more equitably

Published onNov 15, 2022
Disconnecting the evaluation of scientific results from their diffusion
·

Abstract

The traditional way to communicate research results by publishing scientific journal articles addresses two main purposes: the evaluation of scientific information and its diffusion to the scientific community. These two key features can now be addressed separately. Indeed, preprint servers now allow rapid and free diffusion in open access. The recent development of portable peer-review services for preprints addresses the need for the evaluation of scientific results. The Peer Community In (PCI) initiative was created in 2017 as an alternative to the costly and somewhat opaque traditional publishing system. Its aim is to create communities of researchers evaluating and recommending preprints for free. Following the launch of Peer Community in Ecology, Peer Community in Evolutionary Biology, 15 thematic PCIs have already been created. Once they have deposited their preprint in a preprint server or in an open repository, authors can submit it to one of these thematic PCIs. One of the many recommenders (the equivalent of associate editors in traditional journals) can decide to organize its peer-reviewed evaluation. If the preprint is accepted, the corresponding PCI publishes a recommendation text and the peer-review history (reviews, decisions, and author responses). The PCI-recommended preprint has the same value as any journal article, but it can still be further published in a journal including Peer Community Journal if the authors wish to do so. Deeply rooted in the open science context, PCI advocates for transparency of the peer-review process and decisions, and for the independence and reproducibility of scientific evaluation. As a non-profit organisation entirely funded by universities, institutions and libraries, PCI offers authors the possibility to publish their articles free of charge in open access. Because the different communities are created and managed only by scientists for scientists, PCI gives researchers back control over the scientific publication process.

Peer Community In. What is it?

Introduction

The science publication system has been evolving continuously, to adapt to the evolution of communication technologies, but also to the way scientists build and access science. Researchers currently access articles individually through web search engines, and no longer by the table of contents of a printed journal[1]. Because articles are now also read on computers and phones, and because researchers have more articles to read[2], they can directly access special sections (methods, results, etc…) to search for very specific information. They can even access the raw data when they are made available by the authors on an open repository. The digital format of scientific articles also progressively changed the way they are processed. When articles were only published in a printed journal, the authors would send their article to the journal and the only form of the article that could be shared was the paper version published in the journal.

With the transition from printed to digital publication, the different stages of an article are now accessible[3] (see figure 1).

The different stages of an article are: 
- Preprint: first version of an article
- Postprint: article accepted by a journal
- Version of record: article published in a journal
All these versions are available as e-prints and might be posted on a public server, when authorised

Figure 1: Different stages of a research article

The open science movement encourages sharing, commenting, and diffusion by the scientific community of all these stages. Preprints in particular are a focus of attention by the scientific community because they allow to share the results of research at an early stage[4]. Now widely used and encouraged in many fields of research, preprints are however not peer-reviewed when deposited by authors on open archives.

The fast development of preprint use, especially in life sciences[5] and the need for preprint evaluation led to many initiatives for preprint peer-review, many of which were recently listed by ASAPbio on the Reimagine Review platform[6]. The Peer Community In initiative is one of them.

The Peer Community In initiative

Overview

Peer Community In (PCI) was created in 2017 by 3 researchers of the French National Research Institute for Agriculture, Food and Environment, as an answer to the 3 main flaws of the publishing system:

  • too opaque: data, reviews and decisions not visible

  • too slow: several months to years until the publication in a journal, because of the cascade of submissions-rejections

  • too costly: several thousand euros to publish an article in open access

PCI aims at creating communities of researchers who validate and recommend preprints, based on open peer-review, at no cost for authors and readers. Structured as a non-profit organization, it is funded by public research organizations, universities and libraries worldwide[7].

First developed in the field of Evolutionary Biology and Ecology, the 15 currently existing thematic PCIs now cover various disciplines (including Genomics, Microbiology, Zoology, and others)[8]. A specific PCI was created in 2021 for registered reports[9].

The principle

Once they have deposited their data, scripts and code in an open repository and their preprint in a preprint server, authors can submit it to one of these thematic PCIs. PCIs are large communities of volunteer scientists named “recommenders”, who have a similar role to associate editors in traditional journals. One of these recommenders can decide to organize the peer-reviewed evaluation of the submitted preprint if they find it interesting. After peer-review, the recommender in charge of the preprint evaluation reaches an editorial decision (modification requirements, acceptance, or rejection). This is a major difference between PCI and other initiatives of preprint peer-review, in which no editorial decisions are made. If, following this peer-review process, the preprint is accepted, the corresponding PCI publishes a recommendation text, written by the recommender of the preprint, and the complete peer-review history (reviews, decisions, and author responses), with a DOI (see figure 2).

The editorial workflow for Peer Community In includes the following steps:
1/ the authors deposit their data and code on an open repository, and their preprint on a preprint server;
2/ they submit the DOI/URL of their preprint on the website of the chosen thematic PCI; 
3/ if one of the recommenders picks up the preprint, the peer-review process (as in a journal) starts. The recommender submits an editorial decision on the basis of at least 2 reviewers' reports, and the authors revise their preprint. New revised versions of the preprint are posted on the preprint server;
4/ if after several rounds of reviews the recommender accepts the preprint, they write a recommendation that is published on the PCI website with all the decisions, reviews and answers from the authors; 
5/ the recommended article is final, valid, citable, and accessible for free on the preprint server.

Figure 2: Editorial workflow for the Peer Community In initiative

The editorial decision of acceptance (the recommendation) and the editorial correspondence are therefore visible and citable, which are the main differences between PCI and publications in classic journals[10].

Management by the scientific community

Managed only by volunteer scientists, PCI is an example of community-led editorial management. Thematic PCIs are created by a handful (1 to 4) of founders following a few steps[11]. These founders are recognised scientists (usually with a strong background in their field) who bring together a wide network of volunteer scientists to play the role of recommenders. The project of creation of a new PCI must be validated by the PCI organisation. Once set, the founders must set a managing board. This managing board is renewed every 4 years and is made up of several recommenders who, in addition to controlling submissions and editorial decisions, vote to approve the admission of new recommenders.

The recommenders (any scientist in the field with enough experience and expertise to evaluate articles) can apply directly to a thematic PCI or they can be suggested by the existing recommenders. Diversity and inclusiveness are important criteria for the admission of new recommenders.

Each thematic PCI designates a delegate who represents the PCI in the organisation, contributes collectively to the development of the PCI Initiative, and votes to approve the creation of new PCIs.

Decoupling peer-review from publication

PCI manages scientific validations of preprints through a rigorous peer-review process, but also organizes the fate of these validated and recommended preprints.

Indeed, although the PCI-recommended preprint has the same value as any journal article, it can still be published in a journal notably in Peer Community Journal if the authors wish to do so (see figure 3).

After the recommendation of their preprint, the authors have several options:
1/ they can consider the PCI recommended preprint as a final and valid article;
2/ they can publish it directly in diamond open access in Peer Community Journal;
3/ they can submit it to one of the PCI-friendly journals. These journals can either accept the article with no further peer-review, give a fast response to the authors or use the PCI evaluation if appropriate;
4/ they can submit it to any journal;
If they are not satisfied by the journal decision at step 3 or 4, they can try another PCI-friendly journal or any other journal or, at any time, they can choose step 1 or 2.

Figure 3: Fate of recommended preprints

Peer Community Journal

The PCI organization launched Peer Community Journal in late 2021, to provide a diamond open-access publication solution to authors who wanted to publish their recommended preprint in a journal. This journal is free for authors and free for readers.

Peer Community Journal breaks the codes of the traditional publication system because the peer-review process is managed by the thematic PCIs prior to publication in the journal. The journal only accepts submissions of preprints recommended by a thematic PCI, and hence already peer-reviewed and accepted as scientifically valid articles. The publication step is therefore very fast, nearly immediate after its “transfer” to the journal.

Peer Community Journal

Peer Community Journal is a generalist journal publishing articles under a CC-BY license, with multiple sections: each section corresponding to a thematic PCI. It is funded by public research institutions through its parent organisation PCI. It is indexed in the DOAJ[12], in Google Scholar, in Dimension and currently under evaluation for indexation by major international scientific databases (Scopus, WOS, PubMed...).

PCI-friendly journals

Authors who get a recommendation of their preprint by a PCI can also submit it to a PCI-friendly journal that has chosen between 3 types of commitments[13]:

  • Accept, without further peer review, any preprint recommended by a PCI providing they meet any additional procedural requirements not requiring further scientific evaluation by the journal. Such additional requirements may include compatibility with a defined disciplinary scope;

OR

  • Inform the authors within a short-specified reply time (e.g. 5 days), about one of the following decisions on their article:

  • Acceptance with minor modifications, with no further peer review;

  • Need for further peer review before decision;

  • Rejection.

OR

  • Consider submissions of articles recommended by a PCI and to optionally use the PCI reviews and recommendations for their own evaluation process, if appropriate.

The way in which scientific journals become PCI-friendly journals varies. For part of these journals, the founders of the thematic PCI in connection with the field of the journal contacted the editors-in-chief (EiC) who agreed to declare their journal PCI-friendly. For another part, the EiCs themselves contacted PCI to declare their journal PCI-friendly. In all cases, to add their journal to this list, the EiCs must fill in a form describing the economic model, the link with scientific societies, etc…

The list of PCI-friendly journals is available on the PCI website, with the specification, for each journal, of the submission procedure for proper consideration of the PCI reviews and the recommendation.

A response to the flaws of the traditional publication system

Free early access to research results

One of the many flaws of the traditional publication system is the delay of several months to several years between the submission of an article to a journal and its publication after several cycles of submission-rejection by different journals[14]. Because the PCI evaluation process relies on preprints, and because all the revised versions after several peer-review rounds are deposited in open archives, readers can have early access to the evaluated article for free. By giving value to preprints, this initiative also displaces the status of “article validated by the peers” from the version of record stage to the recommended preprint stage.

Additionally, all PCI-recommended preprints have open data, scripts and code, hence fulfilling the open science commitments demanded by a growing number of institutions, countries and funding agencies.

Increased visibility and energy saving

Finding reviewers and robust review reports is quite challenging for journals, given the increasing number of journals, the pressure to publish, and the lack of recognition of reviewing work in researchers’ careers. This lack of recognition is partly due to the fact that review reports are usually anonymous and not publicly accessible.

Reviewers do not belong to the PCIs, although recommenders belonging to the PCI can act as reviewers. As in journals, they are volunteer scientists identified and invited by the recommender in charge of the evaluation of a preprint, in the light of their expertise in the field of the study.

At PCI, reviewers can choose to sign their reports if they want to, and — anonymous or not — these reports are published with the recommendation text on the PCI websites. Institutions, therefore, have direct access to the reviews signed by their researchers through dedicated pages on the PCI websites.

The researcher who accepts to handle the evaluation process of a preprint submitted to a PCI, gets, if the article is recommended, a DOI for their recommendation text that can be cited in their list of publications.

Finally, because the peer-review process is clearly separated from the publication in a journal, and fully transparent, reviewers can avoid unnecessary new peer-review rounds each time the recommended preprint is submitted to a journal.

Transparency and ethics

Unlike other preprint peer-review initiatives, PCI makes editorial decisions based on the peer-review reports. If the article is recommended, all the decisions of the recommenders throughout the process are published and therefore freely accessible along the reviews. In the traditional publication system, editorial decisions leading to the publication of an article are invisible to the readers.

Conflicts of interest are managed throughout the editorial process and a statement of conflict of interest is mandatory in the recommended preprints. Contrary to most journals, authors with financial conflicts of interest of are simply prohibited.

Workload distribution

Each thematic PCI gathers many recommenders. For example, there are more than 400 recommenders of the PCI Evolutionary Biology and PCI Ecology communities. Recommenders have no commitment to take charge of the evaluation of a minimum number of preprints, but are simply encouraged to follow up on one or two preprints per year. Due to the large number of recommenders, PCI can ensure the evaluation of a large number of preprints. Unlike most journals, recommenders choose to handle preprints or not, depending on their interest and the time they can devote to this editorial task. The editorial workload is placed on a larger number of researchers, thus reducing individual burden.

Publication control and financial issues

Because PCI is an initiative led by the scientific community, it gives control on the publication process back to researchers and institutions. The absence of a commercial third party in the whole process also allows focusing on the scientific aim of publication, at a low cost for society and without any financial issues[15].

Conclusion

Deeply rooted in the open science context, PCI advocates for transparency of the peer-review process and decisions, and for the independence and reproducibility of scientific evaluation. As a non-profit organisation entirely funded by universities, institutions and libraries, PCI offers authors the possibility to publish their articles free of charge in open access. Because the different thematic PCIs are created and managed only by scientists for scientists, PCI gives researchers back control over the scientific publication process.

Comments
0
comment

No comments here

Why not start the discussion?