Multimodal interaction description language based on data modeling

Abstract

A methodology for developing a multimodal interactive system is required to be implemented as the number of input and output devices is increasing. Previous MMI (MultiModal Interaction) description languages used a state transition model for defining interaction, which has low maintainability and extensibility. In this paper, we design and implement a new MMI description language named MrailsScript from the software engineering point of view. MrailsScript consists of the data model definition that can be inherited from existing semantic web class. By expanding the existing web application development framework, our Mrails framework can automatically generate prototype code of MMI applications that are based on the MVC (Model-View-Controller) model referring to the annotations of task type and dialogue initiative. We also develop a helper application for the MMI system, MrailsBuilder, which assists in the coding of MrailsScript and contents management using semantic web standards, such as OWL, RDFS, and RDF.