Unified multimodal search framework for multimedia information retrieval

Abstract

There is a trend towards construction of multimedia digital information resources which may hold diverse data types in the form of image, graphics, audio, video, and text based retrieval artifacts or objects. WWW is a huge multimedia information resource. Existing search mechanisms available on WWW are mostly mono-modal. Multimedia information needs are partially satisfied by using mono-modal search mechanisms. Multiple modalities of information are associated with multimedia retrieval object so multimodal search mechanisms are required for searching multimedia information resources. Existing search mechanisms available on WWW whether they are general purpose, domain specific or media specific partially fulfil multimedia information needs because they are mostly mono-modal and able to perform search with in one media type at a time. We explore with the help of multimedia object analysis that search with in all the subsets of multimedia object types is possible by using only four index types, two models of interactions and fifteen possible modes at most. A framework based on an architectural approach is proposed for multimedia information retrieval. Proposed retrieval framework is implemented. Present implementation gives query formulation and information presentation for all modes and search functionality with in few modes of multimedia search framework. © Springer Science+Business Media B.V. 2010.