Multimedia Resource Discovery

Abstract

This chapter examines the challenges and opportunities of MultimediaInformation Retrieval and corresponding search engine applications.Computer technology has changed our access to information tremendously:We used to search authors or titles (which we had to know) in librarycards in order to locate relevant books; now we can issue keywordsearches within the full text of whole book repositories in order toidentify authors, titles and locations of relevant books. What about thecorresponding challenge of finding multimedia by fragments, examples andexcerpts? Rather than asking for a music piece by artist and title, canwe hum its tune to find it? Can doctors submit scans of a patient toidentify medically similar images of diagnosed cases in a database? Canyour mobile phone take a picture of a statue and tell you about itsartist and significance via a service that it sends this picture to?In an attempt to answer some of these questions we get to know basicconcepts of multimedia resource discovery technologies for a number ofdifferent query and document types: piggy-back text search, i.e.,reducing the multimedia to pseudo text documents; automated annotationof visual components; content-based retrieval where the query is animage; and fingerprinting to match near duplicates.Some of the research challenges are given by the semantic gap betweenthe simple pixel properties computers can readily index and high-levelhuman concepts; related to this is an inherent technological limitationof automated annotation of images from pixels alone. Other challengesare given by polysemy, i.e., the many meanings and interpretations thatare inherent in visual material and the corresponding wide range of auser's information need.