Abstract
Multimodal systems must process several input streams efficiently and represent the input in a way that allows the establishment of connections between modalities. This paper describes a multimodal system that uses Combinatory Categorial Grammars to parse several input streams and translate them into logical formulas. These logical formulas are expressed in Hybrid Logic, which is very suitable for multimodal integration because it can represent temporal relationships between modes in an abstract way. This level of abstraction makes it possible to define rules for multimodal processing in a straightforward way. © Springer-Verlag Berlin Heidelberg 2007.
Cite
CITATION STYLE
Giuliani, M., & Knoll, A. (2007). Integrating multimodal cues using grammar based models. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 4555 LNCS, pp. 858–867). Springer Verlag. https://doi.org/10.1007/978-3-540-73281-5_94
Register to see more suggestions
Mendeley helps you to discover research relevant for your work.
