Integrating verbal and nonverbal input into a dynamic response spoken dialogue system

Abstract

In this work, we present a dynamic response spoken dialogue system (DRSDS). It is capable of understanding the verbal/nonverbal language of users and making instant, situation-aware response. Incorporating with two external systems, MultiSense and email summarization, we built an email reading agent on mobile device to show the functionality of DRSDS.