nature materials | VOL 10 | FEBRUARY 2011 | www.nature.com/naturematerials 81
editorial
Transparency is one of the fundamental
guiding principles in science: each
experiment, each theoretical derivation
needs to be documented and verifiable.
Transparency promotes and drives
scientific progress and ensures that
scientific results are accountable to the
highest standards.
At the same time, transparency in
science is not always possible. For example,
most corporate research is confidential, and
in that sense doesn’t have an active part in
the scientific discussion. Another example
is the peer review process, whether it is the
review of funding proposals or scientific
publishing. In the review of funding
proposals the original ideas of scientists
are protected until these researchers have
the opportunity to implement their ideas.
In scientific publishing, similar protection
applies until a paper is published.
Once published, research is of course
subject to scrutiny and verification by
the scientific community. And often the
issues debated then mirror those raised
during the peer review process. Indeed,
discussions during peer review can be
intense. Referee reports can be several
pages long, and author rebuttal letters
even longer. We have had cases of rebuttal
letters exceeding 20 pages — much longer
than the paper in question. Unfortunately,
only a few are privy to these discussions —
authors, referees and editors. The question
therefore is whether the scientific
community would benefit from knowing
retrospectively what has been discussed
during peer review, at least for those papers
that do get published.
This is the aim of a trial at The EMBO
Journal, which is published by Nature
Publishing Group1. Since 2009, the journal
has published a Peer Review File as part
of the supplementary online information
of papers taking part in the trial. The file
provides, amongst other data, the decision
letters from editors to authors, which
include the anonymized referee reports, as
well as the rebuttal letters from the authors.
A Commentary in Nature by the head
of scientific publication at the European
Molecular Biology Organization (EMBO),
Bernd Pulverer, describes the journal’s
experience with the trial2. The trial seems
to be very successful. Almost 95% of
authors have agreed to take part, and
only a few reviewers declined to review
papers under the condition of seeing their
reports published. As for the interest in the
published files, download rates are about
10% of that of the corresponding papers.
Interestingly, it is not the review files of
controversial papers that get downloaded
most, but those of what the editors consider
the best papers.
The successful experience at EMBO is
in contrast to a number of other attempts
to tweak peer review. An example is open
peer review. There, a paper is peer reviewed
publicly and anyone can leave comments on
submitted papers. Editors then take those
comments into account when deciding on
a paper. A few years ago, Nature conducted
a trial of open peer review. However, given
the low uptake by other researchers as well
as the generally low technical quality of the
comments made, this trial was eventually
abandoned3. Another example is double-
blind peer review, where the identity of
authors is withheld. However, experiments,
for example at the American Physical
Society, have concluded that it is difficult in
practice to really conceal authors’ identity
from their colleagues.
In contrast to that, the reason for
the more successful experience in the
publication of the peer review files could
be the minimal influence it has on the
conventional process, while delivering
the benefit of enhanced transparency. For
example, in the EMBO model the editor’s
contributions would be more visible. This
increased scrutiny of our efforts certainly
could benefit science. After all, we editors
have to take responsibility for the decisions
that we take.
A further benefit would be that the
extent to which a paper has been improved
during the peer review process would
become more obvious. This improvement
can be quite substantial and the input
provided by reviewers will be more
identifiable and recognizable. Similarly,
the relationship between reviewers and
editors may also be influenced as reviewers
will be aware that their comments could
get published. On the other hand, there
might still be parts of the editorial process
that would not necessarily be apparent. In
many instances informal communication
between authors and editors as well as
reviewers takes place. Nevertheless, all
information relevant to the discussion
would of course be contained in the
official correspondence.
Scientific results are scrutinized at every
stage, from the design of experiments or
the derivation of theoretical findings to the
discussions after publication. The issues
raised during the peer review process
constitute an important contribution to
this debate. Therefore, the knowledge
about these discussions has a value beyond
their immediate purpose, which is a
journal’s decision whether to publish a
paper or not.
Indeed, the benefits of the transparent
review process seem to have convinced
EMBO to extend this scheme to their other
journals. Beyond these we are not aware of
other trials of this scheme, particularly in
the physical sciences. Of course, researchers
from other fields may feel differently about
this proposed transparency, but the benefits
of this approach by far seem to outweigh
the drawbacks. What is your opinion?
We’d love to hear it. ❐
References
1. http://www.nature.com/emboj
2. Pulverer, B. Nature 468, 29–31 (2010).
3. http://go.nature.com/n67mfk
Would the publication of anonymous referee reports and editorial decision letters of published papers
benefit the scientific debate? Results from a trial seem to suggest this.
Transparency in peer review
Correction
In the Editorial ‘It’s still all about graphene’
(Nature Mater. 10, 1; 2011), the first name of
the person the image is credited to was spelt
incorrectly as Yannik; it should have read Jannik.
Corrected in the HTML and PDF versions after
print: 22 December 2010.
©
GETTY IM
A
GES
nmat_2952_FEB11.indd 81 11/1/11 15:33:30
© 2011 Macmillan Publishers Limited. All rights reserved