Lost in Conversation: A Conversational Agent Based on the Transformer and Transfer Learning

Abstract

State of the art in dialogue and conversational models is currently being attained with generative approaches. In this work, we present our winning solution for the ConvAI2 Competition at NeurIPS 2018, created by the Lost in Conversation team. Our solution combines an encoder-decoder architecture based on a modified version of OpenAI GPT and a transfer learning approach to training, pretraining our model on a separate large dataset and then fine-tuning for the actual conversational datasets. Interestingly, our solution did not place first in automated evaluation with commonly used metrics, but then proceeded to win quite convincingly in human evaluation, which shows that automatic evaluation of conversational models still leaves a lot of room for further research.