Performance issues and error analysis in an open-domain question answering system

Abstract

This paper presents an in-depth anal-ysis of a state-of-the-art Question An-swering system. Several scenarios are examined: (1) the performance of each module in a serial baseline system, (2) the impact of feedbacks and the insertion of a logic prover, and (3) the impact of various lexical resources. The main conclusion is that the overall performance depends on the depth of natural language processing resources and the tools used for answer finding.