GUEST EDITORIAL
Introduction to the special issue on “data semantics
for multimedia systems”
Mei-Ling Shyu & Yu Cao & Jun Kong & Ming Li &
Mathias Lux & Jie Bao
Published online: 17 November 2009
# Springer Science + Business Media, LLC 2009
1 Introduction
In the last decade, substantial progress has been made in content-based analysis andmultimedia
streaming to facilitate the development of large-scale multimedia information systems.
Together with the recent progress on semantic web, it is now possible to build a new generation
of multimedia applications that enable large-scale semantic representation, analysis, and
delivery of multimedia data from heterogeneous data sources. However, there is still a long way
to go for mature solutions of multimedia database systems that are capable of processing
semantics-rich, large-volume multimedia data. It could be even more challenging if such
systems are under stringent functional and non-functional (e.g., QoS) requirements.
The goal of this special issue is to bring the semantic web community and multimedia
processing & computing community together and provide a forum for multidisciplinary
research opportunities, with a focus on how to apply the semantic technologies to the
acquisition, generation, transmission, storage, processing, and retrieval of multimedia
information. Discussions on future challenges in multimedia information manipulation, as
well as practical solutions for the design and implementation of multimedia database
Multimed Tools Appl (2010) 46:147–154
DOI 10.1007/s11042-009-0418-1
M.-L. Shyu
Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL 33146,
USA
Y. Cao (*) : M. Li
Department of Computer Science, California State University, Fresno, Fresno, CA 93740, USA
e-mail: yucao@csufresno.edu
J. Kong
Department of Computer Science, North Dakota State University, Fargo, ND 58105, USA
M. Lux
Institute for Information Technology (ITEC), Klagenfurt University, Klagenfurt 9020, Austria
J. Bao
Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY 12180, USA

software systems are also encouraged. Topics of interest include but are not limited to
practical areas that span both semantic technologies and multimedia processing &
computing.
2 The review process
This special issue solicited high quality research papers in the following areas of interests:
& Automatic generation of multimedia presentations
& Semantic multimedia metadata extraction
& Annotation tools and methods for multimedia semantics
& Media ontology generation/learning/reasoning
& Content-based multimedia analysis
& Multimedia indexing, searching, and retrieving
& Multimedia streaming
& Semantic-based QoS control and scheduling
& Semantic-based Internet data streaming and delivery
& Multimedia standards (e.g., MPEG-7 and XMP) and Semantic Web
& Semantics enabled multimedia applications (including annotation, browsing, storage,
retrieval, and visualization)
& Semantics enabled networking and middleware for multimedia applications.
Submissions related to any area of semantic computing for multimedia systems were
welcomed. We received a very diverse pool of thirty nine (39) submissions and the contact
authors were from China (5), United States (5), France (3), Greece (3), South Korean (3),
Australia (2), Germany (2), Spain (2), Tunisia (2), Turkey (2), United Kingdom (2), Taiwan
(1), Belgium (1), Brazil (1), India (1), Ireland (1), Malaysia (1), Portugal (1), and
Netherlands (1). Due to the interdisciplinary nature of the Call for Paper, the papers
represented a wide range of approaches and scenarios.
Most of the papers are reviewed by at least three experts in the topical area. We have
carefully selected reviewers with the track record publications and review experiences in
the particular sub-area. The guest editors have sent out more than 100 review invitations
and most of the invited reviewers finished their reviews on time. We are very grateful to the
reviewers for their excellent job. In addition, the guest editors examined the reviews for
each paper and profound discussions were performed among the organizers and reviewers
to reach a consensus for each paper. In the end, sixteen (16) papers were recommended for
acceptance. The contact authors of the accepted papers are from Greece (3), United States
(3), Australia (1), Belgium (1), France (1), Germany (1), India (1), South Korea (1), Spain
(1), Netherlands (1), Tunisia (1), and United Kingdom (1).
3 Accepted papers
The sixteen (16) papers selected for publication span a variety of aspects of semantic
computing for multimedia systems, which include the areas of semantic multimedia content
analysis (e.g., annotation, classification, and segmentation), semantic multimedia systems
(e.g., multimedia streaming), as well as intriguing applications of semantic multimedia
computing (e.g., multimedia computing in medical applications). The key ideas and
contributions of these papers are summarized as below.
148 Multimed Tools Appl (2010) 46:147–154

3.1 Semantic multimedia content analysis
In the paper entitled “GAT: A Graphical Annotation Tool for Semantic Regions,” the
authors Xavier Giro, Xavier Giro, Neus Camps, and Ferran Marques present a
Graphical Annotation Tool based on a region-based hierarchical representation of
images. The implementation uses MPEG-7/XML input and output data to allow
interoperability with any type of Partition Tree. The annotation tool is public available
as an open source software. The paper “A Robust Framework for Joint Background/
Foreground Segmentation in Complex Video Scenes Filmed with Freely Moving
Camera” by Walid Barhoumi, Slim Amri, and Ezzeddine Zagrouba explores a robust
region-based general framework for discriminating between background and fore-
ground objects within a complex video sequence. The proposed framework could
identify novel objects and non-novel ones mutually while profiting of the semantic
information offered by regions. In the paper entitled “Exploit Camera Metadata for
Enhancing Interesting Region Detection and Photo Retrieval,” Zhong Li and Jianping
Fan investigate a machine learning-based interesting region detection algorithm for
consumer photos. Based on this algorithm, a computer can reversely calculate what
the interestingness of the photographer is and what the core content of a photo is. The
Paper “Content and Task-based View Selection from Multiple Video Streams” by
Fahad Daniyal and Andrea Cavallaro introduces techniques for content-aware multi-
camera selection that generates view-dependent ranking information using a multivar-
iate Gaussian distribution. The best view is selected by a Dynamic Bayesian Network
(DBN), which utilizes camera network information. In “personalTV—A TV Recom-
mendation System based on Content-based Filtering,” Günther Hölbling, Michael
Pleschgatternig, and Harald Kosch introduce a TV recommendation system based on
the detailed program information and the interaction history of the user with the
system. In the paper entitled “MyOwnLife: Incremental and Hierarchical Classification
of a Personal Image Collection on Mobile Devices,” Antoine Pigeau designs and
implements a personal image classification system for mobile devices. In the paper
entitled “A Service for Validating MPEG-7 Descriptions w.r.t. to Formal Profile
Definitions,” the authors Raphael Troncy, Werner Bailer, Michael Hausenblas, and
Martin Höffernig tackle the problem of lacking formal grounding semantics of MPEG-
7 elements. The proposed approach expresses the semantics explicitly by formalizing
the constraints of various profiles using ontologies, logical rules, and ad-hoc
programming. The authors also implemented the proposed approach as a full semantic
validation web service that is public available over the Internet. Stamatia Dasiopoulou,
Vassilis Tzouvaras, Ioannis Kompatsiaris, and Michael G. Strintzis present a
systematic overview of the state of the art in MPEG-7 based ontologies in their
paper entitled “Enquiring MPEG-7 based Ontologies.” They also highlight issues
pertaining to the intended context of usage, obstacles hindering interoperability, as
well as possible directions towards their harmonization.
3.2 Semantic multimedia systems
In the paper entitled “NinSuna: a Fully Integrated Platform for Format-independent
Multimedia Content Adaptation and Delivery using Semantic Web Technologies,” the
authors Davy Van Deursen, Wim Van Lancker, Wesley De Neve, Tom Paridaens, Erik
Mannens, and Rik Van de Walle present the design and functioning of a fully
integrated platform for multimedia adaptation and delivery. Due to the use of format-
Multimed Tools Appl (2010) 46:147–154 149

agnostic adaptation engines (i.e., independent of the underlying coding format) and
formatagnostic packaging engines (i.e., independent of the underlying delivery
format), this platform is able to efficiently deal with the aforementioned heterogeneity
in the present-day multimedia ecosystem. In “A Multimedia Data Streams Model for
Content-Based Information Retrieval”, Shi-Kuo Chang, Shenoda Guirguis, Rohit
Kulkarni, and Lei Zhao provide a multimedia data streams model with the objective
of furnishing a formal framework to efficiently design a Multimedia Data Streams
(MMDS) schema that achieves an efficient performance in regard with content based
retrieval. Nikolaos Konstantinou, Emmanuel Solidakis, Anastasios Zafeiropoulos,
Panagiotis Stathopoulos, and Nikolas Mitrou investigate the problem of the real-time
integration and processing of multimedia metadata collected by a distributed sensor
network in their paper entitled “A Context-aware Middleware for Real-Time Semantic
Enrichment of Distributed Multimedia Metadata.” They propose an approach for the
real-time, rule-based semantic enrichment of lower level context features with higher-
level semantics. In the paper entitled “Incorporating Packet Semantics in Scheduling
of Real-time Multimedia Streaming,” Sungwoo Hong and Youjip Won develop a
packet scheduling algorithm which properly incorporates the semantics of a packet.
This research aims to develop a packet scheduling mechanism which can improve the
user perceivable QoS instead of focusing on improving packet loss, delay, nor
burstiness.
3.3 Intriguing applications of semantic multimedia computing
In the paper entitled “Content based Radiology Image Retrieval using a Fuzzy Rule based
Scalable Composite Descriptor,” the authors Savvas A Chatzichristofis; Yiannis S Boutalis
address the problem of creating an effective method for the indexing and retrieval of
radiology images. One of the most important features of the proposed Fuzzy Rule Based
Compact Composite Descriptor is that its size adapts according to the storage capabilities of
the application that uses it. Mathias Lux, Oge Marques, Klaus Schöffmann, Laszlo
Böszörmenyi, and Georg Lajtai present a video summarization tool and demonstrate how it
can be successfully used in the domain of arthroscopic videos, which are video streams
generated by a small camera used in the arthroscopic surgery procedure. They discuss how
this tool can be used for arthroscopic videos, leveraging some domain-specific aspects,
without losing its ability to work on general-purpose videos. In “A MPEG-7 Authoring and
Retrieval System for Dance Videos,” Rajkumar Kannan introduces a system with the
capacity of semi-automatic authoring and access to dance archives. This system offers an
MPEG-7 based semiautomatic and search engine based on spatial, temporal, and spatio-
temporal features of the dancers using the tree-embedding technique. In the paper entitled
“Ranking Canonical Views for Tourist Attractions,” Lin Yang, John K Johnstone, and
Chengcui Zhang leverage online photo collections to automatically rank canonical views
which are a subset of the photographs that best summarize a photo collection, for a tourist
attraction.
Acknowledgments We are very grateful for all of the contributing authors and the hard work of more than
100 reviewers. We would also like to thank the anonymous reviewers of the proposal and Editor-in-Chief Dr.
Borko Furht for sharing our vision. Special thanks got to the MTAP editorial staff (Mhanilet Leon de, Laura
Lander, and Emma Kalb) for their technical and administrative assistance.
150 Multimed Tools Appl (2010) 46:147–154

Dr. Mei-Ling Shyu has been an Associate Professor at the Department of Electrical and Computer
Engineering (ECE), University of Miami (UM) since June 2005. Prior to that, she was an Assistant Professor
in ECE at UM dating from January 2000. She received her Ph.D. degree from the School of Electrical and
Computer Engineering, Purdue University, West Lafayette, Indiana, U.S.A. in 1999, and her three master
degrees from Computer Science, Electrical Engineering, and Restaurant, Hotel, Institutional, and Tourism
Management from Purdue University in 1992, 1995, and 1997, respectively. Her research interests include
data mining, multimedia information systems, multimedia networking, database systems, and security. She
has authored and co-authored more than 170 technical papers published in various prestigious journals, book
chapters, and refereed conference/workshop/symposium proceedings. She received the “Best Student Paper
Award” from the Third IEEE International Conference on Semantic Computing (ICSC2009) in September
2009 and the “Johnson A. Edosomwan Scholarly Productivity Award” from the College of Engineering at
UM in 2007. She has been a guest editor of several journal special issues, an associate editor/editor board
member for many journals, and a program chair of many conferences/workshops.
Dr. Yu Cao has been an Assistant Professor at the Department of Computer Science, California State
University, Fresno (Fresno State) since August 2007. Prior to that, he was a Visiting Fellow of Biomedical
Engineering at Mayo Clinic, Rochester, Minnesota. He received his M.S. and Ph.D. degrees in Computer
Science from Iowa State University in 2005 and 2007, respectively. He received the B.Eng. degree from
Harbin Engineering University in 1997, the M.Eng. degree from Huazhong University of Science and
Technology in 2000, all in Computer Science. His research interests span a variety of aspects of image
processing, computer vision/visualization, and multimedia database. His research work has appeared in
various prestigious journals, book chapters, and refereed conference/workshop/symposium proceedings. His
research is being supported by U.S National Science Foundation. He has collaborated with researchers from
Mayo Clinic and Iowa State University to develop the first intelligent multimedia system to analyze, retrieve,
and visualize important content in medical images and videos captured during endoscopy. He is a member of
ACM, IEEE, and Upsilon Pi Epsilon (UPE).
Multimed Tools Appl (2010) 46:147–154 151

Jun Kong received the B.S. degree in Computer Science from Huazhong University of Science and
Technology, China in 1998, the M.S. degree in Computer Science from Shanghai Jiao Tong University,
China in 2001, and the Ph.D. degree in Computer Science from the University of Texas at Dallas in 2005. He
is an assistant professor of Computer Science at North Dakota State University. His research and teaching
interests include software modeling and design, pervasive computing, human-computer interaction and visual
languages.
Ming Li has been a faculty in the Department of Computer Science, California State University, Fresno,
since August 2006. He received his M.S. and Ph.D. degrees in Computer Science from The University of
Texas at Dallas in 2001 and 2006, respectively. His research interests include QoS strategies for wireless
networks, robotics communications, and multimedia streaming over wireless networks. He is the recipient of
the Best Student Paper Award in the First IEEE International Workshop on Next Generation Wireless
Networks (WoNGeN’05). He has served as the TPC co-chair of The First International Workshop on
Pervasive Computing Systems and Infrastructures (PCSI 2009), The Second International Workshop on
Sensor Networks (SN’09), Multimedia Networking track in ICCCN’08, the IEEE International Work-shop
on Data Semantics for Multimedia Systems and Applica-tions (DSMSA’08), and the Third IEEE
International Workshop on Next Generation Wireless Networks (WoNGeN’08). He is a guest editor of a
special issue on Recent Advances in Sensor Integration for International Journal of Sensor Networks
(IJSNet), a special issue on Data Semantics for Multimedia Systems in Springer Multimedia Tools and
Applications, and a special issue in Journal of Multimedia. He has served the technical program committees
of several international conferences such as ICC, ICCCN, ISM, ROBOCOMM, VTC, and ChinaCom. Ming
Li is a member of ACM and IEEE.
152 Multimed Tools Appl (2010) 46:147–154

Dr. Mathias Lux is University Assistant at the Institute of Information Technology at Klagenfurt University
in the distributed multimedia systems. He received his master degree in Mathematics in 2004 and his Ph.D.
in Telematics in 2006, both with distinction from Graz University of Technology. His Ph.D. thesis focused on
semantics in multimedia metadata, especially MPEG-7. Mathias Lux has an extensive research record in the
field of multimedia metadata. His experience encompasses research on annotation, metadata based
multimedia retrieval, and social software and user intentions. He published numerous papers at peer
reviewed conferences and several peer reviewed journals. Lately, he published a book on multimedia
semantics as editor and has won the open source competition of the ACM Multimedia with the open source
MPEG-7 tools Caliph & Emir. His current research focus is on intentional metadata and social aspects of
multimedia annotation and retrieval. He is member of the ACM and founding member of the Multimedia
Metadata Community (http://www.multimedia-metadata.info), and has organized several successful scientific
events in this context.
Dr. Jie Bao received his B.S. and M.S. in electrical engineering from Hefei University of Technology in
1998 and 2001, respectively, and his Ph.D. in computer science from the Iowa State University (ISU) in
2007. At ISU, he developed new formalisms and tools for semantic data integration, modeling modular
ontologies, collaborative ontology building and Web privacy protection. From 2008 to present, Dr. Bao is a
postdoctoral research associate at the Tetherless World Constellation, Rensselaer Polytechnic Institute, where
he worked on a variety of topics in Semantic Web including social semantic web, ontology integrity
constraint languages, policy formulation and scalable reasoning with Web ontologies. Dr. Bao has authored
Multimed Tools Appl (2010) 46:147–154 153

more than 40 research papers in journals and peer-reviewed international conferences and workshops. He has
served on Organizing Committees or Programming Committees of more than 20 international conferences
and workshops, including International Joint Conferences on Artificial Intelligence (IJCAI), International
Semantic Web Conference (ISWC), Extended/European Semantic Web Conference (ESWC), Asian Semantic
Web Conference (ASWC) and Association for the Advancement of Artificial Intelligence (AAAI) Symposia,
and is currently serving on the editorial board of Journal of Emerging Technologies in Web Intelligence. He
is a member of the OWLWorking Group at W3C in which he co-authored two W3C specification documents
of OWL 2.
154 Multimed Tools Appl (2010) 46:147–154