PEERAssist: Leveraging on Paper-Review Interactions to Predict Peer Review Decisions

Abstract

Peer review is the widely accepted method of research validation. However, with the deluge of research paper submissions accompanied with the rising number venues, the paper vetting system has come under a lot of stress. Problems like dearth of adequate reviewers, finding appropriate expert reviewers, maintaining the quality of the reviews are steadily and strongly surfacing up. To ease the peer review workload to some extent, here we investigate how an Artificial Intelligence (AI)-powered review system would look like. We leverage on the paper-review interaction to predict the decision in the reviewing process. We do not envisage an AI reviewing papers in the near-future, but seek to explore a human-AI collaboration in the decision-making process where the AI would leverage on the human-written reviews and paper full-text to predict the fate of the paper. The idea is to have an assistive decision-making tool for the chairs/editors to help them with an additional layer of confidence, especially with borderline and contrastive reviews. We use cross-attention between the review text and paper full-text to learn the interactions and henceforth generate the decision. We also make use of sentiment information encoded within peer-review texts to guide the outcome. Our initial results show encouraging performance on a dataset of paper+peer reviews curated from the ICLR openreviews. We make our codes and dataset (https://github.com/PrabhatkrBharti/PEERAssist ) public for further explorations. We re-iterate that we are in an early stage of investigation and showcase our initial exciting results to justify our proposition.