AStandard for MultimediaMiddleware
D.J. Duke
Dept. of Computer Science
The Universityof York
Heslington, }~rk, YOI 5DD, U.K.
4 ~904 432 m2
duke@cs.york.ac.uk
1. msTRAcT
Advmees in hardware, software, and coding
stindards for digiti media have now made the
defivery of mdtimedia ifiormation a sWdard
component of many systems. Unfortunately, the
pace of this technologiml development coupled
Itith strong commercial competition between
leadng vendors, has meant that fittle consensus
hx emerged over the dwign of programming
interfaces to Wow the creation, maniptiation,
~d presentation of such dab. PREMO &Re-
sentition Environments for Mtitimedia
Objeck) is a new MO S*dard, to be pub-
~~bed in 1998, that defies a middeware
frmework encompassing synchrotiation, the
mnagement of distributed media resourcw,
and the sdess integration of dab and pro-
cessw from d~parate appti~tion areas. This
paper @ves m ove~ew of the concepts under-
l~tingPREMO and the approach by which the
shndard ut~ies these concepts to addrws key
mpeck of mtitirnedia systems design.
1.1 Keywords
Distributed multimedi% Standards, PREMO
2. ~RODUC~ON
Early in 1998, SC24, tie subcommittee of ISO~C ~C 1
concerned with computer ~waphicsand image processing,
completed work on a new standard for multimtia
presentation, cdld PREMO @PresentationEnvironment for
hjultimedia Objects). me otigind proposal for PREMO was
for a new computer .mphics standar~ to be based expficidy
on an object oriented approach. Such an approach was seen
~<timely, as object~fiented design and proe~g had
rapidly become established, and work on a number of object
oriented APIs for computer graphics had generated interest
Pemission10makedigitalorhardcopiesofall or pan of
~hs work for pemonal or classroomuse is granted
wi~hout fee provided that copies are not made or
distributedfor profit or commercialadvartxage,and lhat
copiesbear this nolieeand the full citationon the first
page.To COPYothem-ke,to republisk topost onsen’ers
or 10 redistribute 10 lists, requires prior specific
perrniss~onand’ora fee.
AChl hfultimedia’98.Brisrol.UK
@1998AChi 1-5611>036S[9S{0008 S.00
1.Herman
Centrum voor Wiskunde en lnformatica (CWI) I
Kruislaan
Amsterdam, The Netherlands
+31 205924163
lvan.Herman@cwi.nl
within the graphics community for this technology ~nventor,
the precursor of OpenInventor, is probably the best known
example [20]). Development of a new stindard was also seen
as an opportunity to address firther technological issues.
First the new standard should encompass other media, such
as video, audio @oth captured and synthetic), and in
principle be extensible to new modrdities such as haptic
output and speech or gesturd inpu4 which have become
increasingly integrated within graphics applications; virtual ● “
environments and systems for visualization being a prime
examples. me smond requirement was that the standard
should allow the construction of distributed systems, where
p- of a system involved in the generation, processing, or
the presentation of media data could be distributed across
geographicrdly remote sites, interacting through a network.
me graphics-related origins of PREMO helps explaining
how it differs from existing media standards. Well known
standards such as MPEG, MHEG, or ~IG, and more recent
work such as S~[22], are concerned with declarative
models of media conten~ andor algorithms by which that
content can be efficiently encoded for transport. Although
PREMO does encompass au abstract model of media content
(see swtion 6.1), its main concern is with the programming
environment through which mdla presentations can be
construct process~ and rendered as part of an overall
application. me PREMO environment allows existing media
devices to inter+perate, and be interfaced to an application.
It dso provides an object+riented programming infra-
s~cture to support the development of such applications. In
achieving this, the design of P~MO recognizes that the
evolution of multimedia systems and technologies from
reswch tools into mature products has lead to a rich set of
programming frameworks for meeting fundamental systems
requirements such as synchronization and quality of service
management (see, for example, [1,13,17], or others).
However, it is not the role of a standard such as P~MO to
arbitrarily choose between these implementations, many of
which address specific and complementary demands.
bstead, PREMO servw two roles.
1) Zt provides a level of middlware which SUDDOti the
implementation of a ‘range of processing m;~els. me
term “middleware” has come to the fore in the past years,
referring to a software layer between the fundamental
services of an operating system and more specific appli-
cation developme~~environments. PREMO, as a rniddle-
ware, does not define services in dl their details; instead,
it provides an environment where various, vendor–spe-
cific components can. cooperate. ~is also means that
3s1
—
-.,.

2)
here is a deficate brdance in the specification of each
P~hfO objecc it should not be too detail~ otherwise
it would restrict the range of possible implementations,
but it should not be too vague, i.e., too meaningless,
either. ~Is stive for balance underpins the specification
of each objects in P~40.
me rniddleware layer provided by P~MO becomes
important when large, distribute multimedia applica-
tions are to be developd a vide~onferencing system
is a good example. Due to the variety of avdable media
formats, resource requirements, means of distribution
control, etc., a sia%ificantportion of these app%cations
have to consider issues fike configurability, adaptability,
access to remote resources, distribution, etc. A stiar
level of adaptiveness is dso required when traditionrd
metla are com~lnd wifll, e.g., synthetic graphics and
computer animation. Without middleware such as
P~hIO, much of this intitructure has to be developed
from scratch, or adapted from a similar application, and
the costs involved in continurdly mo~lng software to
meet new demands are well known. me variety of
graphics formats, available primitives, animation dg~
nthms, etc., dso continues to expand, and portable
app3cations increasingly have to adapt to ~ evolving
environmen~ P~40 has put a great emphasis on these
aspects, and a PNhi&based middlewase layer can
.~eatiy facilitate the development of such applications.
me P~10 middeware is dso closely related to dis-
tributed object+riented architectures, such as OMGS
CO~A[l 8].1 Mthough P~IO itself is form~y not
dependent on the OMG spmfications, P-O should
be viewd as a multimdl~rientd extension of the
basic object services and architecture providti by sys-
tems like OhIG or, to take another example, Java’s W
services. As such, P=hfO fills the gap between tie very
geneti set of facilities offered by OMG and a distrib-
uted multimedia application. me relationship between
PWhIO and a distributed object+rientd architecture is
so close, that it would be ill-advised to attempt an imple-
mentation of P~fO without the use of such services.
PREh~O also sewes as a reference mdel. P~O can
of course sen’e as a ‘Yefl implementation environment
(a prototype implementation is currentiy under prepara-
tion in Java see section 7.), but it rdso contains a whole
range of uni~lng concepts needed in mdtimtia sys-
tems developrnen~ h practice, “portability of program-
mers” is almost as important as the “potiltity of
programs”; although only the latter is usurdly pubficised
by organi~tions ~ie 1S0, the former approach to porta-
bility was dso considerti to be a major gord for
P~10. Having a common, well understood set of
principles and tectilques greafly helps in understanding
both the specificities and the commonrdities of various
] In fac~ a Eaison existed behvcen ONIG and the relevant 1S0
group, during the development of P~IO, which clearly itiu-
enced tie design of thestandard
3S2
multimdia programming environments; to achieve its
goals, P~MO deliberately chose to “spell ou~’ a
number of details which are sometimes hidden in other
systems. ~s reference model aspect is not only impor-
tant in a didactic sens~ a unifying set of concepts may
play an important role in classi~ing, relating and organ-
izing the growing range of software toolkits that are
available to the potential developer of a multimedia sys-
tem. Without such concepts, WIStechnological cornuc-
opiais in danger of becoming an anarchic ensemble of
incompatible antior incomparable artifacts.
h parallel with the development of processing technologies,
the growth of multimedia systems has rdso generated a large
number of mdla formats, some of which have rdready
become international or national standards. me
combination of various media formats with different models
for media processing means that a multimedia systems
development features strong requirements for configuration
and coordination of processing activities. Concurrency,
either within a single device or as a consequence of physical
distribution, provides a further level of complexity in the
development of systems. me facilities provided by PWMO
support the integration and coordination of media
proc~sing elements and devices involved in the
presentation of m~la data in a variety of formats, both from
captured sources an~or as part of a rerd-time application,
for example a virtual environment. P=MO by itself is not a
complete presentation environment (unlike a system such as
P~GS[4]), but rather defines a level of rniddleware that
provides generic, fundamentrd services and infrastructure
for a range of applications. ~ls is the point where various
other multimedia specifications and standards meet with
P~MO: to give an example, while the 1S0 MPEG
specification[15] describes the details of a video format,
P=O concentrates on how an MPEG coder/decoder can
be used together with other media processing entities like a
~-phics renderer.
A number of papers concerning specific aspects of PWMO,
such as its objat model and synchronization facilities,
appeared during the development of the Standard (see,
e.g., [11,12,14], although the rapid development of the
standard was usually faster than the publishing process,
which made some of these overview papers slightly
outdated). With the completion of the Standard, the current
paper is intended to describe how the various components of
P=MO contribute towards the goals and requirements
outind in the preceding paragraphs, and to serve as a final
overview for the standard itself. me 1S0 document
becomes officially available in the summer of 199S. A
reques~ asking the central secretariat of 1S0 to release the
document on the hteme~ is currentiy pending.
Section 2 of this paper provides an overview of the four
Components that comprise the Standard. Sections 3,4 and 5
then expand on three issues that concern significant
requirements in the design of multimedia applications:
synchronization, resource management and coordination,
and presentation. Each of these sections describes the
1
1
:.
>
1
!-
)
+
:-
.,
I
+
,
,
I