A snapshot of NLG evaluation practices 2005 - 2014

Abstract

In this paper we present a snapshot of end-to-end NLG system evaluations as presented in conference and journal papers1 over the last ten years in order to better understand the nature and type of evaluations that have been undertaken. We find that researchers tend to favour specific evaluation methods, and that their evaluation approaches are also correlated with the publication venue. We further discuss what factors may influence the types of evaluation used for a given NLG system.