Reproducibility: A Primer on Semantics and Implications for Research

Abstract

Reproducibility is a core tenet for conducting and validating scientific investigations and thus producing high-quality research. The reproducibility of a result or finding is a fundamental goal in science. Oſten, investigators, publishers of journals, and others give low priority to results or findings that are merely confirmatory of previous findings. Increasingly, however, this narrow focus on “new” research results is seen as a challenge to scientific progress and to the acceptance of research (or science and the scientific method) by the public at large. This growing acknowledgment of the importance of reproducibility has been accompanied by rising confusion about this and related concepts, such as repeatability, replicability, transparency, and quality. Globally, experts are pointing to the muddled state of affairs in terms of understanding the literal and technical meanings of these constructs—and using them correctly. Conflating the terms undermines communication across scientific disciplines and endeavors and makes the practical processes of ensuring rigorous multidisciplinary research more difficult. We consider the issues about reproducibility to be core elements of high-quality research. Most research organizations have formal expectations and policies or programs setting standards for high-quality research. For example, RTI’s commitment to robust research is articulated in its own quality manual. Its goal is to ensure excellence for all RTI’s products and services, from project conception through completion and delivery of research data and final reports, as well as for publications in peer- reviewed journals, the RTI Press, and other venues. The challenges, and the motivation for this publication, lie more in understanding concepts around reproducibility—their similarities and differences, their nuances, and their applications in everyday scientific activities.