Evaluation of question-answering system about conversational agent’s personality

Abstract

We develop a question-answering system for questions that ask about a conversational agent’s personality based on large-scale question-answer pairs created by hand. In casual dialogues, the speaker sometimes asks his conversation partner questions about favorites or experiences. Since this behavior also appears in conversational dialogues with a dialogue system, systems must be developed to respond to such questions. However, the effectiveness of personality-question-answering for conversational agents has not been investigated. Our user-machine chat experiments show that our question-answering system, which estimates appropriate answers with 60.7% accuracy for the personality questions in our conversation corpus, significantly improves user’s subjective evaluations.