Towards the creation of a unified framework for multimodal search and retrieval

Abstract

In this paper, a novel framework for search and retrieval of multimodal content is introduced as part of the EU-funded project I-SEARCH. The main objective of I-SEARCH is to create a unified framework for multimodal content search, i.e. to retrieve content of any media type (text, 2D images, video, audio and 3D) by using as query any of the above media, along with real-world information, expressive and social cues. The outcome will be a highly user-centric search engine, able to deliver to the end-users only the content of interest, satisfying their information needs and preferences, which is expected to significantly improve end-user's experience. The paper will present the concept of I-SEARCH, as well as its major scientific advances. © 2012 ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering.