HARKing and P-Hacking: A Call for More Transparent Reporting of Studies in the Information Systems Field

Abstract

While researchers are expected to look for significant results to confirm their hypotheses, some engage in intentional or unintentional HARKing (Hypothesizing After Results are Known) and p-hacking (repeated tinkering with data and retesting). If these practices are widespread, one possible result is field-wide exaggerated (inflated) results reported in Information Systems (IS) publications. In this paper, we summarize the literature on HARKing and p-hacking across different disciplines. We offer an illustrative example of how an IS study could involve HARKing and p-hacking in various stages of the project to generate a more “publishable” result. We also report on a survey targeted at IS researchers to explore their experiences and awareness of this issue. Finally, we provide recommendations and suggestions based on the review of practices in other fields and advocate for more transparency in reporting research projects, so that study results can be interpreted properly, and reproducibility and replicability can be increased.