Surface-similarity based textual entailment recognition for Japanese text

Abstract

This paper proposes a surface-similarity based method for recognizing textual entailment (RTE) in Japanese. First, we experimentally show that there is a positive correlation between semantic similarity (textual entailment) and surface similarity between sentences. The most effective measure of surface similarity for RTE is the character overlap ratio, which achieves classification accuracy of 78.3%. Based on the result, we design a two-step RTE system for binary classification. The first step classifies a given text pair into positive or negative entailment based on the character overlap ratio. If the pair is classified into the positive class, the second step examines whether the assigned class should be flipped or not by using heuristic rules that detect the mismatch of named entities and numbers. In addition to the RTE system, we also implement the MC system that classifies a given text pair into one of four classes (forward entailment, bidirectional entailment, contradiction, and the others), by combining a contradiction detector and the RTE system. In the RITE-2 formal run, the RTE system was ranked 7th among 42 systems at the RTE task, and the MC system was ranked first among 21 systems at the MC task. These results show that the surface-similarity basedmethod achieves high performance in RTE.