Exploring social practices of peer-review in an agent-based simulation: The cost action peere

Abstract

The peer-review process was designed to assure the validity and quality of science that seeks publication. As such, peer review is a cornerstone of science, confirming the “worth” of scientific studies and results. However, the peer-review process is increasingly seen as biased, opaque, slow and inconsistent. Its quality and efficiency depends on a complex, large-scale collaboration process, which is sensitive to motivations, incentives and institutional contexts. PEERE (New Frontiers of Peer Review), a recent EU-funded action, aims to improve efficiency, transparency and accountability of peer review through a trans-disciplinary, cross-sectorial collaboration. In this paper, we report on the results of one of PEERE's activities, in which we explore, in a simulated environment, different approaches to peer reviewing. The objectives of this Action are: (i) to analyse peer review in different scientific areas by integrating quantitative and qualitative research and incorporating recent experimental and computational findings, (ii) to evaluate the implications of different models of peer review and to explore new incentive structures, rules and measures to improve collaboration in all stages of the peer review process; (iii) to involve science stakeholders in data sharing and testing initiatives, and, (iv) to collaboratively define a joint research agenda that points to an evidence-based peer review reform. In order to understand the effect of different peer-review models on the behaviour of reviewers, an understanding of the context and culture of peer review is required. In this paper, we present a framework to analyse different peer-review formats, based on models of social aspects of human motivation. In order to analyse the effect of different peer-review models on the behaviour of reviewers, a deep understanding of the context and culture of peer review is required, which includes a rich model of agent's motives and behaviours. The theory of human motivation by McClelland distinguishes four motives (McClelland (1987)): • Achievement: is about achieving goal states and drives people to try to achieve different things, thus, fostering explorative behaviour that satisfies the need for novelty. • Power: is about trying to have an impact on the world, and includes both the need to control both its own and other's actions. • Affiliation: drives people to seek the company of others and to establish and maintain positive interactions with others. • Avoidance: leads to self preservation, seeking certainty, and emotional regulation, and fosters the categorization and simplification of behaviour so that it becomes more standardized (and thus predictable). Moreover, our simulation framework supports the specification of different peer-review practices. These practices are then compared by letting agents with different social frames enact the different reviewing approaches. The model considers motives and social practices as basis for deliberation. We show initial simulation results on two different peer-review processes.