Semantics Extraction From Multimedia Data: An Ontology-Based Machine Learning Approach

Abstract

It is often the case that related pieces of information lie in adjacent but different types of data sources. Besides extracting such information from each par- ticular type of source, an important issue raised is how to put together all the pieces of information extracted by each source, or, more generally, what is the optimal way to collectively extract information, considering all media sources together. This chapter presents a machine learning method for extracting complex semantics stem- ming from multimedia sources. The method is based on transforming the inference problem into a graph expansion problem, expressing graph expansion operators as a combination of elementary ones and optimally seeking elementary graph operators. The latter issue is then reduced to learn a set of soft classifiers, based on features each one corresponding to a unique graph path. The advantages of the method are demonstrated on an athletics web-pages corpus, comprising images and text.