We propose a new metric, Voted Appropriateness, which can be used to automatically evaluate dialogue policy decisions, once some wizard data has been collected. We show that this metric outperforms a previously proposed metric Weak agreement. We also present a taxonomy for dialogue model evaluation schemas, and orient our new metric within this taxonomy.
CITATION STYLE
Gandhe, S., & Traum, D. (2016). A Semi-automated Evaluation Metric for Dialogue Model Coherence (pp. 217–225). https://doi.org/10.1007/978-3-319-21834-2_19
Mendeley helps you to discover research relevant for your work.