Nullius in Calculo: on the explicitness and reproducibility of cladistic analyses

Abstract

The result of a Maximum Parsimony cladistic analysis should be repeatable. Yet I present here numerous examples of recent papers in which the results contained therein cannot be replicated, given only the content of the paper, supplementary materials and links. Barriers to study replication include (1) absence of requisite information, (2) typesetting errors, and even (3) author error. I argue that these problems, many of which are easily-spotted, should not be appearing in peer-review published papers with such regularity. Inspired by the Royal Society's motto, I humbly suggest that reviewers and editors not only examine the words of papers, but also the underlying data and calculations: Nullius in Verba, Nullius in Calculo. Furthermore, I believe the reporting of phylogenetic analyses would greatly benefit from increased Standardization following community-agreed criteria c.f. MIAPA (Leebens-Mack et al, 2006), and data deposition in appropriate data archives specifically designed to accommodate phylogenetic data e.g. TreeBASE or MorphoBank. In addition to problems with reproducibility, I also detail problems with explicitness of method reporting. A manual examination of over 300 recently published incongruence length difference (ILD) tests provides evidence to suggest that method sections are rarely sufficiently explicit in their detail to exactly replicate the methods used to generate reported results. In particular I suggest that authors should be encouraged to explicitly state how gaps are coded and treated, and which branch collapsing rules are followed in analyses. Different settings can and do generate different results, therefore all such important settings should be explicitly stated. Improvements in computing and the Internet offer us an excellent opportunity to digitally enrich the quality and legacy of the science we publish we just need to embrace some small beneficial changes to the way we publish.

Cite this document ^(BETA)