Multimedia content description using semantic web languages

Abstract

During the last decades, digital media have revolutionised media reproduction. The availability of cheap consumer electronic devices that allow the consumption and management of digital multimedia content (e.g. MP3 players, digital cameras, DV camcorders, smart phones) has caused a media availability explosion. The amount of digital media that has been generated and stored, and which continues to be at an exponential rate, has already become unmanageable without fine-grained computerised support. This, in combination with the media distribution break-up carried out by the WorldWideWeb and the emergence of advanced network infrastructures that allow for the fast, efficient, and reliable transmission of multimedia content, has formed an openmultimedia consumption environment. Digitalmultimedia content services are provided in this environment, which offer high content quality, advanced interaction capabilities, media personalisation and adaptation according to the user preferences, and access conditions. Such an open environment will be successful only if it is based on standards that allow the services provided by different vendors to interoperate. The specification of different multimedia content description standards poses interoperability requirements and necessitates guidelines for semantic interoperability. These issues are discussed in detail in Tzouvaras and Pan (2007). The dominant standard in multimedia content description is MPEG-7 (ISO MPEG Group), which provides rich general purpose multimedia content description capabilities, including both low-level features and high-level semantic description constructs. However, the lack of formal semantics in MPEG-7 makes the gap between low- and high-level descriptions difficult to cope with for the existing tools. Consequently, low-level features are common, as they can be easily extracted from the content, but there is a lack of high-level descriptions. Low-level approaches, based on signal analysis, are proving to be extremely limiting in making multimedia database systems accessible and useful to the end-users. These content-based descriptors lie far away from what the users recognise as media description means (Celma, Gomez, Janer, Gouyon, Herrera and Garca 2004). Consequently, recent research has begun to focus on bridging the semantic and conceptual gap that exists between the user and the computer -from content-based to high-level descriptions. One approach to overcome this gap is the use of knowledgebased techniques based on web ontologies. As formal and web-wide shared conceptualisations, ontologies facilitate automated integration and meaningful retrieval of multimedia - both content and metadata - from different sources. Searching in digital libraries has been widely studied for several years, mostly focusing on retrieving textual information using text-based methods. These queries can be complemented and improved with advanced retrievalmethods using contentbased descriptors extracted from the audiovisual information by applying signal processing, even though some knowledge management and representation of the content is necessary. Moreover, from the service and content providers point of view, multimedia metadata represents an added value to audiovisual assets, but then again manual annotation is a labour-intensive and error-prone task. Thus, managing audiovisual essence implies structuring its associated metadata using description schemes, taxonomies, and ontologies in order to organise a meaningful data knowledge representation. In addition to the syntactic interoperation, which is achieved through the standards, semantic interoperation, which is achieved through the integration of domain knowledge expressed in the form of domain ontologies, is also needed for providing efficient retrieval and filtering services. The domain knowledge is subsequently utilised for supporting semantic personalisation, retrieval, and filtering and has been shown to enhance the retrieval precision (Tsinaraki, Polydoros and Christodoulakis 2007). This chapter describes the representation of multimedia content descriptions that are structured according to the MPEG-7 metadata description model and expressed using the Semantic Web languages. The rest of the chapter is structured as follows: Section 2.2 provides an overview of MPEG-7. The general purpose approaches for multimedia content description that are supported by the MPEG-7 standard are presented as well as the limitations of the current MPEG-7 version (mainly a lack of explicit semantics). Section 2.3 presents the existing web ontology languages, while Section 2.4 outlines the efforts made to move the MPEG-7 standard into the Semantic Web. In our case, this is accomplished by interpreting and expressing the informal MPEG-7 semantics using Semantic Web languages. An approach for mapping XML schema (Fallside 2001) constructs to OWL constructs (McGuinness and van Harmelen 2004) is presented in Section 2.5, while Section 2.6 presents two use cases that show the benefits of this approach, including semantic integration and retrieval in the music domain. An integrated ontological infrastructure for the semantic description of multimedia content is presented in Section 2.7. This infrastructure allows for combining the general purpose MPEG-7 constructs with domain and application-specific knowledge through the systematic representation of this knowledge in the form of web ontology language (OWL) domain and application ontologies integrated with the MPEG-7 semantics. The chapter conclusions are presented in Section 2.8.