Learning to Rank Claim-Evidence Pairs to Assist Scientific-Based Argumentation

Abstract

We consider the novel problem of learning to rank claim-evidence pairs to ease the task of scientific argumentation. Researchers face daily scientific argumentation when writing research papers or project proposals. Once confronted with a sentence that requires a citation, they struggle to find the manuscript that can support it. In this work, we call such sentences claims – a natural language sentence – that needs a citation to be credible. Evidence in our work refers to a paper that provides credibility to its corresponding claim. We tackle the scientific domain where the task of matching claim-evidence pairs is hindered by complex terminology variations to express the same concept and also by the unknown characteristics beyond content that makes a paper worth to be cited. The former calls for a suitable representation capable of dealing with the challenge of content-based matching considering domain knowledge, whereas the latter implies a need to propose semantic features of suitable characteristics to guide the learning task. To this end, we test the scope and limitation of a deep learning model tailored to the task. Our experiments reveal what specific attributes can guide the learning task, the impact of using domain knowledge in the form of concepts and also the assessment of which metadata of a document, e.g., ‘background’, ‘conclusion’, ‘method’, ‘objective’, or ‘results’ should be considered to achieve better results.