In statistics, samples are drawn from a population in a data- generating process (DGP). Standard errors measure the uncer- tainty in sample estimates of population parameters. In sci- ence, evidence is generated to test hypotheses in an evidence- generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sam- ple. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.
CITATION STYLE
Menkveld, A. J., Dreber, A., Holzmeister, F., Huber, J., Johanneson, M., Kirchler, M., … Bao, L. (2021). Non-Standard Errors. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3961574
Mendeley helps you to discover research relevant for your work.