D.J. Duke, I. Herman, M.S. Marshall
PREMO: A Framework for Multimedia Middleware
A Java description of the ISO/IEC Standard
Lecture Notes in Computer Science No. 1591
Springer Verlag
1999

PrefaceEarly in 1998, SC24, the subcommittee of ISO/IEC JTC 1 concerned with computer
graphics and image processing, completed work on a new standard for multimedia pres-
entation, called PREMO (PResentation Environment for Multimedia Objects), and pub-
lished under the official reference ISO/IEC 14478. The original proposal for PREMO
was for a new computer graphics standard, to be based explicitly on an object-oriented
approach. Such an approach was seen as timely, given that object-oriented design and
programming had rapidly become established, and work on a number of object-oriented
APIs for computer graphics had generated interest within the graphics community for
this technology (Inventor, the precursor of OpenInventor, is probably the best known ex-
ample). Development of a new standard was also seen as an opportunity to address fur-
ther technological issues. First, the new standard should encompass other media, such
as video, audio (both captured and synthetic), and in principle be extensible to new mo-
dalities such as haptic output and speech or gestural input, which have become increas-
ingly integrated within graphics applications; virtual environments and systems for
visualization being prime examples. The second requirement was that the standard
should allow the construction of distributed systems, where parts of a system involved
in the generation, processing, or the presentation of media data could be distributed
across geographically remote sites, interacting through a network.
Although the original goals for the development of PREMO included the detailed
specification of an API for multimedia programming, including all kinds of rendering
and media-coding facilities, it soon became clear that such goals were unrealistic. The
diversity of requirements for various applications and the wide range of different tech-
niques made the development of a detailed specification problematic. Instead, interop-
erability became the key issue: existing tools, applications, and programming interfaces
should be able to cooperate, even if they come from different implementations and ven-
dors. The term “middleware” came to the fore, denoting a software layer between the
operating system facilities and application programs: in this way PREMO evolved into
a middleware specification for multimedia programming.
Although PREMO defines objects for implementing multimedia middleware, the
emphasis on interoperability means that PREMO also functions as a reference model for
distributed multimedia. Concepts common to a range of approaches in this area have
been described and integrated in the PREMO model, and consequently the standard has
an important role in education, and in promoting cooperation between programmers in-
volved in multimedia development projects across potentially heterogeneous platforms.
The text of an International Standard is usually dry, and notoriously difficult to read.
Although this book does not replace the official text, its goal is to provide a more read-
able version of the concepts, to present some of the more interesting details of the PRE-
MO multimedia objects, to highlight the reasons for specific design decisions, and to
give simple examples which clarify the underlying concepts. If the goal of the reader is
to implement the PREMO standard, this book should aid in understanding the precise
specification of the ISO text. However, the book should also be useful for students and

professionals whose goal is to gain a better understanding of the issues involved in dis-
tributed multimedia, regardless of the intricate details of the PREMO standard; this
group probably represents the majority of our readers.
Obviously, PREMO is the result of team work, which involved experts from four
continents and more than 10 countries. It is impossible to list all the people who, for a
shorter or a longer period, participated in the work. Nevertheless, we would like to men-
tion the contributions of three people who played particularly important roles. Horst
Stenzel (FH Köln, Germany) was the rapporteur of the working group within ISO which
was responsible for the development of PREMO. It was his task to coordinate the de-
velopment of the standard. James Van Loo (Sun Microsystems Inc., USA) was a co–ed-
itor of the document, and was instrumental in integrating the so–called Multimedia
Systems Services definition, which became the core of the final PREMO document. Fi-
nally, David Duce (Rutherford Appleton Laboratory, UK) coordinated the ERCIM
Computer Graphics Network which, between 1993 and 1997, played a seminal role in
the precise specification of large portions of the standard. We express our gratitude to
them, as well as to all experts who participated in the development of PREMO; this
book is the result of their work.
February 1999 David Duke
Ivan Herman
Scott Marshall

Contents
PREMO: A Standard for Distributed Multimedia . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 What PREMO Is . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 What PREMO Isn’t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Formal Description Techniques and PREMO . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Structure of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Typographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 Graphical Conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
An Overview of PREMO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 The Structure of PREMO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 The PREMO Object Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
2.3.2 From Language Bindings to Environment Bindings . . . . . . . . . . . . . . 12
2.3.3 Object References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.4 Active Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.5 Operation Dispatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.6 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.7 Non-object Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 The Foundation Component. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4.1 Structures, Services, and Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4.2 Inter-Object Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.3 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4.4 Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4.5 Property Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4.6 Object Factories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5 The Multimedia Systems Services Component . . . . . . . . . . . . . . . . . . . . . . . . 22
2.5.1 The Paradigm of Media Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.5.2 Virtual Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.5.3 Stream Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5.4 Virtual Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5.5 Virtual Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5.6 Higher-Levels of Organization: Groups and Logical Devices . . . . . . . 27
2.5.7 Working in Unison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.6 The Modelling, Rendering, and Interaction Component . . . . . . . . . . . . . . . . . 28
2.6.1 Object-Oriented Rendering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6.2 Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6.3 Modelling and Rendering Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.6.4 Coordination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.7 Closing Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

6.5 Virtual Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.5.1 Configuring Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.5.1.1 Global Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.5.1.2 Port Configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.5.2 Examples of Virtual Devices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
6.5.2.1 Simple Media Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.5.2.2 Transformer Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
6.6 Virtual Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.6.2 Detailed Specification of Virtual Connections . . . . . . . . . . . . . . . . . . 156
6.6.3 Examples of Virtual Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
6.6.4 Multicast Connections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
6.7 Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
6.8 Logical Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
The Modelling, Rendering, and Interaction Component . . . . . . . . . . . . . . . . . 165
7.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
7.2 Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
7.2.1 The Role of Primitives in PREMO . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.2.2 The Hierarchy in Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.2.3 Captured Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
7.2.4 Form Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.2.5 Tactile Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
7.2.6 Modifier Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
7.2.7 Wrapper Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
7.2.8 Tracer Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
7.2.9 Structured Primitives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
7.2.9.1 Aggregate Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
7.2.9.2 TimeComposite. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
7.2.10 Reference Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
7.3 Coordinate Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
7.3.1 Coordinate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
7.3.2 TimeLocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
7.3.3 Colour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
7.4 Devices for Modelling, Rendering and Interaction . . . . . . . . . . . . . . . . . . . . 187
7.4.1 MRI_Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
7.4.2 Efficiency Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
7.4.3 MRI Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
7.4.4 Modeller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
7.4.5 Renderer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
7.4.6 MediaEngine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
7.5 Input Devices, and Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
7.5.1 InputDevice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
7.5.2 Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
7.6 The Scene Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

7.7 Coordination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.7.1 Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.7.2 Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.7.3 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
Detailed Java Specifications of the PREMO Objects . . . . . . . . . . . . . . . . . . . . 205
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
8.2 Foundation Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
8.2.1 Enumerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
8.2.2 Additional Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
8.2.3 Top Level of PREMO Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
8.2.4 Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
8.2.5 General Utility Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
8.2.5.1 Event Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
8.2.5.2 Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
8.2.5.3 Time Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .211
8.2.6 Sychronization Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .211
8.2.7 Negotiation and Configuration Management . . . . . . . . . . . . . . . . . . . 214
8.2.8 Creation of Service Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
8.3 Multimedia Systems Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
8.3.1 Enumerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
8.3.2 Structures and Additional Data Types . . . . . . . . . . . . . . . . . . . . . . . . 216
8.3.3 Configuration Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
8.3.4 Stream Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
8.3.5 Virtual Resource . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
8.3.6 Virtual Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
8.3.7 Virtual Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
8.3.8 Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
8.3.9 Logical Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
8.4 The Modelling, Rendering, and Interaction Component . . . . . . . . . . . . . . . . 221
8.4.1 Objects for Coordinate Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
8.4.1.1 Coordinate Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
8.4.1.2 Colour Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
8.4.1.3 TimeLocation Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
8.4.2 Name Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
8.4.3 Objects for Media Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
8.4.3.1 Primitive Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
8.4.3.2 Captured Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
8.4.3.3 Primitives with Spatial and/or Temporal Form . . . . . . . . . . 222
8.4.3.4 Form Primitives for Audio Media Data . . . . . . . . . . . . . . . . 222
8.4.3.5 Form Primitives for Geometric Media Data . . . . . . . . . . . . 223
8.4.3.6 Primitives for the Modification of Media Data . . . . . . . . . . 223
8.4.3.7 Modifier Primitives for Audio Media Data . . . . . . . . . . . . . 223
8.4.3.8 Modifier Primitives for Structural Aspects of Media Data . 224
8.4.3.9 Modifier Primitives for Visual Aspects of Media Data . . . . 224

8.4.3.10 Organising Primitives into Structures. . . . . . . . . . . . . . . . . . 225
8.4.3.11 Organising Media Data within Time . . . . . . . . . . . . . . . . . . 225
8.4.4 Objects for Describing Properties of Devices . . . . . . . . . . . . . . . . . . . 227
8.4.4.1 MRI_Format Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
8.4.4.2 EfficiencyMeasure Object . . . . . . . . . . . . . . . . . . . . . . . . . . 227
8.4.5 Processing Devices for Media Data . . . . . . . . . . . . . . . . . . . . . . . . . . 227
8.4.5.1 MRI_Device Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
8.4.5.2 Modeller Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
8.4.5.3 Renderer Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
8.4.5.4 MediaEngine Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
8.4.6 Scene Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
8.4.7 Objects for Supporting Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
8.4.7.1 InputDevice Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
8.4.7.2 Router Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
8.4.8 Coordinator Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Selected Implementation Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
A.1 The PREMO Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
A.1.1 Activity of Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
A.1.2 Top Level of the PREMO Hierarchy . . . . . . . . . . . . . . . . . . . . . . . 232
A.1.3 Operation Request Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
A.1.4 Distribution and the Creation of PREMO Objects . . . . . . . . . . . . . 235
A.2 Specific Part 3 Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
A.2.1 Virtual Connection Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
A.2.1.1 Devices on the Same JVM: Piped Streams . . . . . . . . . . . 238
A.2.1.2 Devices on Different JVM’s: Sockets . . . . . . . . . . . . . . . 238
A.2.1.3 Multicast Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

4potential developer of a multimedia system. Without such concepts, this technological
cornucopia is in danger of becoming an anarchic ensemble of incompatible and incom-
parable artifacts.
1.1.2 What PREMO Isn’t
The characteristics that define what PREMO is – middleware and reference model –
also reflect what PREMO is not. In particular, PREMO is intended to build on and uti-
lise existing media standards, not to replace them. Given that there are standards in
place for media formats and processing, these are two concerns that PREMO does not
address.
PREMO is not a Media Format
The PREMO specification does not describe any new format for the representation and
storage of media data. Instead, the Standard makes it quite clear that the data processed
by PREMO-based applications is expected to be stored in existing formats; ALAW,
JPEG, MHEG, MIDI, MPEG [55], SMIL [95], VRML [54], to name a few. What PRE-
MO does provide are mechanisms by which new PREMO objects can be defined for
new formats, and by which existing objects can coordinate the formats that they use to
exchange and process media data.
PREMO is not a Media Engine
The object types defined in the PREMO standard are not sufficient in themselves to re-
alise a working multimedia application. To do this would have required the Standard to
commit to particular kinds of media processors and renderers, with specific interfaces.
All that this would achieve would be to add yet another type of media engine into the
growing collection of such devices. Instead, PREMO provides a number of object types
that can act as “wrappers” around existing engines, and allow these to be used within a
processing network involving other devices that may be based on quite different media
formats or models. Rather than thinking of PREMO as a media engine, a somewhat bet-
ter analogy is to view PREMO as a software architecture for multimedia applications;
the objects defined by PREMO represent the basic constructs, the building blocks, for
multimedia applications. Even this analogy is not quite the whole story though. Al-
though parts of the PREMO specification provide building blocks that are “shaped” for
supporting a particular architectural model of an application, these in turn rely on a set
of lower-level PREMO objects, and users of PREMO is free to build on these, or modify
the higher-level components, in order to instantiate whatever model of multimedia ar-
chitecture that is most appropriate for their needs.
Just as PREMO is not a media engine, it is not a complete environment, either. It does
not, for example, provide a framework for quality of service management. This may
seem strange, since quality of service is a particularly fundamental problem with mul-
timedia applications. However, quality of service management is currently bound tight-

6(G.J. Reynolds, then at CWI, Amsterdam), and invited him to provide an initial report
on the applicability of formal description techniques to SC24 standards [72]. This report
recommended that the formal description technique “Object–Z” [24,25] be used in the
development of PREMO, as a way of gaining further insight into the design of the ob-
ject types needed in the Standard.
Subsequent work with formal methods and PREMO concentrated on two areas that
were proving a source of difficulties within the committee work, specifically the PRE-
MO object model, and the general facilities for synchronization. Work on the object
model has been published as a paper [19]. The initial work on synchronization has been
published as a technical report [20], while subsequent work exploring behavioural as-
pects of synchronization is described in [26]. We will not make use of the formal de-
scription of PREMO in the book, but the interested reader may consult the references
above, or a recent paper summarising this aspect of the development [23].
1.3 Structure of the Book
The main objective of this book is to present a detailed description of the PREMO
standard. From the background material in section 1.1, it should already be clear that
PREMO encompasses a range of concerns, spanning levels of multimedia presentation
from low level synchronization of media streams through to high level facilities for in-
teroperation and coordination of devices. This complexity presents something of a chal-
lenge when attempting to provide a coherent account of the Standard; by starting with
the low-level facilities, one risks describing a large number of detailed facilities but
with little direction or motivation, while in starting with the higher-level facilities, one
is forced to introduce concepts with the proviso that “these will be explained in detail
later”. We have sought to avoid both of these pitfalls by giving an initial, comparatively
informal description of the Standard as a self contained chapter, and then entering into
the details, matching our explanation with the actual structure of the Standard.
As PREMO is intended to be independent of any particular system or programming
language, the description of the official standard was deliberately written using a non-
programming notation, based on the Object-Z specification language [24][25]. We have
chosen not to reuse this notation here, for two reasons. It is awkward to typeset, even
within a poerful typesetting language like LaTeX (which has not been used for this
book). More importantly, a second objective of this book is to describe how the PREMO
standard could be implemented. This requires a programming notation (along with a
supporting environment for distributed objects). For this we have chosen the Java tech-
nologies of Sun Microsystems Inc.; specifically the language [36] and the RMI library
[39] for remote access. And rather than use two notations for describing PREMO, we
have decided to use Java throughout, both as a means of documenting what the Standard
provides, and as the vehicle for discussing implementation issues. As we will see, the
match between what the PREMO standard describes and what Java provides is not ex-
act, and consequently we have dedicated an early chapter of the book to describing how
we are using Java to describe PREMO. Thus, following this introduction, we have:

7• Chapter 2, An overview of PREMO. The high level organisation of PREMO into
four distinct components is explained, and subsequently the role and principle fea-
tures of each component is described. This is an important chapter as it provides the
underlying rationale for some of the design decisions that are described in more
detail in later chapters.
• Chapter 3, Fundamentals of PREMO. In order to understand PREMO, it is neces-
sary to understand the object model on which the whole PREMO approach is based.
This chapter describes the first part of the Standard, which sets out PREMO’s view
of object orientation, other kinds of entity in a PREMO system, and assumptions
about the environment of a PREMO application.
• Chapter 4, General implementation issues. The description of the PREMO object
model is by necessity independent of any language and implementation environ-
ment. In the remainder of the book, we will be using Java to describe the object
types and services defined in the Standard. The general issues that arise from using
Java as the language to describe the Standard are described in the chapter. It covers
aspects such as naming conventions, and the facilities beyond the core language that
are needed to define PREMO.
The three chapters that follow present the content of the remaining three parts of the
Standard, with each chapter addressing a particular component of PREMO.
• Chapter 5, The foundation component. This chapter explores the low level services
and objects that underpin many approaches to multimedia presentation.
• Chapter 6, Multimedia systems services component. Higher-level abstractions over
media processing devices and other resources are introduced, and their role in sup-
porting a particular paradigm for multimedia systems is discussed.
• Chapter 7, Modelling, Rendering, and interaction component. Specialised devices
for media presentation, in particular support for synthetic graphics, is introduced,
along with a framework for declarative modelling of media presentations.
In conjunction with this book, a number of core object types from the Standard have
been developed and placed in the public domain1). It is not a complete implementation
of the Standard, but the most significant object types described in this text have been
realised, in particular those that present interesting challenges to an implementer. This
activity has been carried out, not so much with the intention of using it for subsequent
systems development (though that is certainly possible), but rather to demonstrate how
the requirements defined for PREMO can be met in practice.
• Chapter 8, Detailed Java Specifications of the PREMO Objects, gives the complete
specification of the PREMO standard as a set of Java interfaces.
• Appendix A, Selected implementation issues. A number of the programming tasks
facing a PREMO implementer are relatively routine; concepts such as state
machines, property lists etc. are well known and should require no design hints.
1) ftp://ftp.cwi.nl/pub/premo

8However, a few requirements defined for PREMO are not trivial, and in the appen-
dix we describe a strategy for dealing with issues such as operation dispatch modes
and the creation of connections.
An index is provided at the end of the book.
1.4 Typographical Conventions
We have endeavoured to keep typographical conventions to a minimum. The text of the
book is written in times font (thus), with italics & bold for occasional emphasis. When
mentioned within the text, the names of PREMO object types (Java classes and inter-
faces) appear capitalised, and in italics. The same convention is used for the names of
PREMO operations (Java methods) in the running text. Definitions of Java classes,
packages, methods, and related code fragments, are presented as indented blocks, using
courier font.
1.5 Graphical Conventions
Although UML [30] is approaching the status of de facto standard for the documenta-
tion of object-oriented systems, the notation goes beyond what is needed in most parts
of this book, and we have instead opted for a simple way of documenting object-orient-
ed structures based on the conventions used in the book “Java in a Nutshell” [28]. These
are:
• class names are written in normal times font;
• interface names are written in times italic;
• if class (or interface) B extends class (or interface) A, B is written below A, and
they are joined by a solid line; and
• if class B implements interface A, B is again drawn below A, and they are joined by
a dashed line.
Shaded ovals have been used to show the grouping of classes and interfaces into pack-
ages.

11
Within a development project which uses an object-oriented target language, the
choice of object model is effectively made once the target language is chosen. Indeed,
the precise details of the available object model may be one criteria by which the lan-
guage is chosen. In the case of PREMO, however, the situation is more complicated.
Like the standards that it follows (GKS [48] and PHIGS [49]), PREMO is intended to
be independent of any particular programming language. Thus, just as one can obtain a
C [58] binding or a FORTRAN [16] binding for GKS, it should be possible to obtain a
C++ [79] or Ada’95 [7] binding for PREMO. The need to provide this flexibility raises
a number of difficult technical questions, not the least being whether it should be pos-
sible to bind PREMO to a language with no explicit support for object-oriented pro-
gramming (e.g. FORTRAN’77 [82]). For now, the main point is that if PREMO is to be
language independent and described in an object-oriented framework, it requires the
definition of some object model that defines the concepts from which the remainder of
the standard can be constructed.
One of the fundamental issues that had to be decided at an early stage in the project
was whether to adopt a “classical” object-oriented approach, in which objects are in-
stances of classes that can be arranged in a hierarchy through inheritance, or opt for a
more radical approach based, for example, on the use of prototypes and delegation. The
former is typical of the models that underlie object-oriented design methods, and has
been in widespread use in the form of languages such as Simula [9], Smalltalk [35], and
C++. Prototype-based approaches have, in contrast, been largely the concern of the re-
search community; there has already been discussion on the value of such approaches
in graphics and multimedia [3]. In particular, the use of delegation, and the notion of
“trait” objects used for example in the SELF system [83] are attractive from the view-
point of building highly adaptable and extensible systems. However, technical issues
aside, the fact that prototype models are strongly bound to experimental systems, and
are not in widespread use, represented a serious barrier to their use within PREMO. The
result is that the PREMO object model is based from the outset on a fundamental dis-
tinction between objects and classes, which in PREMO are called “object types”. A de-
tailed account of this model is given in Chapter 3; following an informal overview of
the model, the remainder of this section describes other high-level design decisions that
affected the content of this component.
2.3.1 Overview
A PREMO system consists of a collection of objects, each with a local (internal) state,
and an interface consisting of a set of operations. Each object is an instance of an object
type, which defines the structure of its instances. An object type can be defined as an
extension to one or more other object types through inheritance; note that this allows
for multiple inheritance. An important property of the model is that objects are never
accessed directly. Instead, a PREMO client requests a facility called an “object factory”
to generate an object satisfying specific criteria, and if it is able to comply, the factory
will return a handle to the new object called an object reference. All subsequent activi-
ties involving the object is then done via the reference, for example invoking an opera-
tion on the object, or passing the object as a parameter to another operation. This

13
no constructs for referring to an object other than through pointers. C++ and Ada’95
have a different model. Objects in these languages are defined as generalised records,
and a pointer to an object is a well defined type that is quite distinct from the type of the
object itself.
As PREMO objects can be distributed, various mechanisms for accessing objects
may be used within even a single system. For example, local objects might be refer-
enced via pointers, while remote objects are referenced by some form of extended URL.
To avoid confusion and implementation bias, the standard introduces the concept of an
object reference as an explicit part of the object model, with the intention that this be
bound to whatever means are used within the target language or environment to access
specific objects. The approach taken in PREMO combines elements of the explicit and
implicit approach. In line with the former, the model defines both the concept of an ob-
ject, and an object reference. However, the distinction is there to simplify the use of
multiple implementation strategies — it is not possible to refer to, or use, an object di-
rectly. Instead all access to an object, for example to invoke an operation, must be via
an object reference.
2.3.4 Active Objects
Concurrency is by definition an integral aspect of multimedia presentation, and will cer-
tainly be a property of the type of distributed application which PREMO is intended to
support. Fundamental to such a model is the idea that several threads of control, or proc-
esses, can be active within a system at one time, and that such processes interact through
communication events. Here again there is a tension between adopting a simple model
based on a particular set of facilities, or a more general model that is harder to use within
the standard but is hopefully easier to implement.
On the one hand, there is a natural and appealing parallel between the idea of a proc-
ess and that of an object. A process is an entity which encapsulates a thread of control
and that interacts with its environment through events; an object is an entity that encap-
sulates state and interacts with its environment through operations. Languages such as
Eiffel [66] and more recently Java have built on this view by treating processes (or
threads) as particular types of object; in Java for example, an object will be active if it
implements the Runnable interface. In contrast, other languages have maintained a sep-
aration between these concepts. In Ada’95 for example, processes are realised through
a sophisticated task model, quite separate from the notion of task, while in C++ there is
as yet, unfortunately, no standard model for dealing with processes.
The PREMO object model assumes that all objects are conceptually active; as we
will discuss in section 2.4.1, the standard does however, for efficiency reasons, define
certain types of objects to have trivial activity. What the standard does not do is to man-
date any particular mechanism through which object activity should be realised. What
is required is that each object has the capability to have an internal thread of activity. In
parallel with this internal activity, an object may receive requests for an operation to be
invoked; these requests arrive at operation receptors. At any time an object can select

14
which requests it is willing to service. The PREMO object model does not place any
requirements on the execution order for operations, for example pending requests may
be serviced sequentially or concurrently.
2.3.5 Operation Dispatching
The delays inherent in remote object access and operation invocation mean that asyn-
chronous operations are a fundamental tool in the development of distributed systems.
Synchronous operation calls, in which the caller is suspended until the called operation
terminates, are also required. To support multimedia applications, the design of PRE-
MO also allows for a third kind of operation, sampled. A sampled operation is similar
to an asynchronous one, because once the operation has been invoked the caller is able
to continue its processing while the request is held in a queue. The difference is that the
queue of requests for a sampled operation is effectively a one-place buffer, with any re-
quest for the operation overwriting any pending request.
Each PREMO operation is defined as using one of these operation request modes.
The existence of these modes is one of the more significant differences between the
PREMO object model, and that found in most programming languages, or indeed the
model defined by the OMG.
2.3.6 Attributes
One of the positive aspects of object orientation is the emphasis on data hiding and en-
capsulation — clients of an object should only use the operations in the interface of an
object, and should not have access to the internal state. Instead, if access to a variable
is required, it should be realised through operations that retrieve or set the value of the
variable. A number of such state variables appear within PREMO object types, and rath-
er than define explicit operations for manipulating these variables, the standard intro-
duces the concept of an attribute. The definition of an attribute looks like that of a
variable, however an attribute of an object type is understood as being a shorthand for
a pair of operations in the interface of that object type which set and get the value of an
(internal) state variable. An attribute can be declared as read–only, or write–only, mean-
ing that the corresponding ‘set’ or ‘get’ operation is not available.
2.3.7 Non-object Data Types
SmallTalk was for some time presented as the prototypical object-oriented program-
ming system, and many of the ideas it pioneered were adopted in subsequent languages
and systems. One of its strengths was its simple ontology; everything in the system is
presented as an object, even “atomic” data such as numbers and characters. While this
view produces a remarkably uniform model, it does have a number of consequences.
First, there are a number of general raised by such an approach, including how one in-
terprets the “identity” of numbers, how one relates binary operations on data “objects”
to the conventional mathematical view of numbers. Second, there is the issue of effi-
ciency: treating data values as objects implies that operations such as addition are han-

15
dled by the same run-time dispatch mechanism as other operation calls. Data processing
in computer graphics and multimedia often involves a considerable amount of numeri-
cal processing with large data sets (geometric structures, digital image formats, etc.) and
here the need to use a general dispatch model is clearly an efficiency concern. Finally,
while PREMO is intended to be language independent, the most likely targets for a lan-
guage binding were seen as the family of object-oriented languages, including C++,
Ada’95 and Java, in which object-oriented structures have been added to a language in
which primitive data are treated as values. For these reasons, PREMO has adopted a
model that distinguishes between non-object data types, such as integers and characters,
and object types.
2.4 The Foundation Component
The implementation of most multimedia systems involves a number of fundamental
concerns: control and management of progression through media content, synchronisa-
tion between activities, time, and coordination. Existing standards provide specific fa-
cilities for some of these tasks, while for others an implementor may need to utilise a
general library (for example, for synchronisation) or develop ad-hoc solutions. Without
mandating any specific approach to these general concerns, the PREMO Foundation
component provides a set of general-purpose object and data types that can be used by
a developer to implement the functionality mentioned above. A developer can either use
these facilities “raw”, to create a customised architecture, or they can be used via the
higher level object types and services provided by Parts 3 and 4 of PREMO which are
described later in this chapter. Because the Foundation component is essentially a
toolkit, the remainder of this section describes its main provisions in terms of the prin-
ciple media system requirements that are supported.
2.4.1 Structures, Services, and Types
The requirement that, conceptually, all PREMO objects are active means that in princi-
ple all access to an object must allow for the possibility that the object will have its own
thread of control. Depending on the implementation platform, this assumption may im-
pose a high overhead on the cost of accessing components of objects; such access will
for example have to pass through the operation receptor and request handling infrastruc-
ture. For some aspects of media processing, these overheads are unavoidable; they are
needed to support the provision of distributed services across a media network. How-
ever, in a typical media application, not all objects will necessarily be used as “active”
entities that provide services. One use of objects is as data encapsulators, similar to the
use of records (structures) in languages such as Ada and C. There is clearly a trade-off
here, between the elegance and simplicity of a homogeneous object model on the one
hand, and the practical problems involved in storing and processing large multimedia
datasets on the other. For example, a visualisation application may need to operate on a
volume data set containing in the order of 107 - 109 voxels [74]. If each voxel is repre-
sented as an object, the overhead in processing this dataset will become significant.

16
PREMOObject general object systemPREMO has adopted an approach that retains a fundamentally simple object model
while allowing implementors to avoid the overhead of the full operation request system
where it is not required. The approach is based on the top-level organisation of the PRE-
MO object type hierarchy shown in Figure 2-1. All object types in PREMO are subtypes
of PREMOObject, in which fundamental object behaviour, such as the ability of each
object to return information about its type, is defined. Below this the hierarchy bifur-
cates. SimplePREMOObject serves as a supertype for those object types that represent
data encapsulators. Such object types are referred to as structures. EnhancedPREMOO-
bject is the abstract supertype for those object types that provide services, and which
therefore incur the overhead of the operation dispatch mechanism. This separation is
further formalised through the profiles that are defined in each component to identify
those object and non-object types that should be made available to clients of the com-
ponent. Each profile consists of lists of object types, either under the category “provides
type”, or “provides service”. Only an object type that inherits from EnhancedPREMOO-
bject is allowed to appear in the “provides service” clause, and it is only objects of
these types that a client can expect to interact with through operation dispatching.
2.4.2 Inter-Object Communication
Although ultimately all interaction between objects within a PREMO system takes
place via operation requests, this is not a particularly useful way of representing com-
munication and cooperation within a distributed system. In the case of multimedia, two
models are now well known:
• Stream-based models, in which information related to processing is sent on commu-
nication channels or media streams between objects. These may be the same
streams that are used to carry media data.
• Event-based models, in which there is conceptually a separate mechanism by which
specific operations in the interface of a collection of objects can be invoked in
response to a specific situation in one object.
SimplePREMOObjectEnhancedPREMOObject
objects used as
passive data stores
objects used to
provide services
facilities
Figure 2-1 — Two kinds of object type in the PREMO hierarchy

18
event handler postpones the notification of objects interested in the event until all ob-
jects that have registered as notifiers have signalled the event to the handler. This object
type has a role in the general synchronization facilities of PREMO, which are discussed
next.
2.4.3 Synchronization
Like event handling, synchronization requirements in PREMO span a range of levels.
At the level of data streams, fine-grained synchronization may be used to implement
quality of service requirements, for example maintaining an adequate alignment be-
tween related audio and visual content. At a higher level, a multimedia presentation will
typically consist of a collection of components, some of which may be presented in par-
allel. In addition to any fine level of synchronization between such strands, synchroni-
zation between key milestones (such as the start/end of component strands) may be
required. Beyond direct control of media presentation, synchronization may also be
needed within the general control structure that manages the overall media system.
Synchronization in PREMO is supported at two levels - in terms of events, and in
terms of time. Event-based synchronization has obvious application in dealing with the
processing of structured presentations composed of more primitive media streams,
however it also has a role in synchronizing the presentation of the data within a stream,
where significant milestones are defined by the content of the stream, rather than its ab-
solute position. An example of this is the synchronization of ultrasound or other medical
scan data, where milestones defined by physiological events need to be aligned. Such
an example is described in more detail in [63]. Time-based synchronization is better
known, and involves ensuring that multiple activities reach particular milestones at
times specified relative to each activity.
The event and time-based approaches are both supported by a common framework,
the Synchronizable object type, which PREMO uses as the basis for representing, mon-
itoring and controlling the transmission and processing of media data. Although the in-
terface to this object type is large, it is based around three main ideas:
1. An internal progression space, which acts as a coordinate system for defining the
concept of location within some media stream or content. Synchronizable objects
do not themselves carry media data, but instead are inherited by object types which
are involved in the transport and processing of such data. Conceptually, the progres-
sion space represents the temporal extent of some media representation, and
progress through the progression space is made during processing of that media.
2. Progression is controlled by a finite state machine. This is actually achieved by hav-
ing Synchronizable inherit from another object type, called a Controller,
which is also defined in this component. Controllers are described in Chapter 5 and
their details are not of concern in this overview. It suffices at present to say that a
Synchronizable object can be in one of four states: stopped, playing, paused, and
waiting. Conceptually, when an object is in the playing state, progress is being made
through its progression space. Transitions between the states occur as a result of
operation invocation, and also through interaction with reference points, which are

19discussed below. A number of attributes define the parameters that affect how
progress is made, for example, the direction of progression.
3. Reference points can be placed along the progression space, either individually, or
repeated with a given period. Each reference point consists of an event, a reference
to an event handler, and a special boolean ‘wait’ flag. When a reference point is
encountered during progression, the event is sent to event handler specified. The
wait flag indicates whether progression should be suspended at this point, and if has
the value true, the Synchronizable object is placed into the ‘waiting’ state, where it
will remain until the resume operation in its interface is invoked.
Reference points and the ‘wait’ flag are intended to be used in conjunction with other
PREMO facilities to implement synchronization schemes. For example, by combining
reference points with the ANDSynchronization object type described in
section 2.4.2, processing of one part of a presentation can be suspended once a particu-
lar milestone has been reached until all other Synchronizable objects that involved
in implementing the presentation have reached related milestones. An example of such
a scheme is shown in Figure 2-3.
2.4.4 Time
Media such as sound, video and animation is fundamentally grounded in time, and to
describe and control the presentation of such media it is necessary to have some means
of representing and measuring time. The question of how time should be represented
(for example, as a continuum, or discretized) has been the subject of much philosophical
debate, and is a non-trivial concern in areas such as real-time systems modelling and
verification. PREMO adopts a pragmatic approach, in which all representations of time
are based on ‘ticks’ produced by some clock. The granularity of a ‘tick’ is not fixed by
the standard, but rather depends on the particular clock used.
waiting
playing
position
position
dispatchEvent
resume
ANDSynchronizationPoint
Synchronizable objects
Reference
points
Figure 2-3 — Example of a Synchronization Scheme

22
In order to hide these issues, and provide a uniform interface for object creation, the
foundation component of PREMO introduces the concept of an object factory. A factory
is itself an instance of the GenericFactory object type that provides a single opera-
tion, createObject. This operation accepts an object type, and a set of constraints in
the form of a sequence of key / permitted value pairs, and (if possible) returns a refer-
ence tp an object that is an instance of the given type or a subtype, and whose properties
satisfy the constraint.
Factories are themselves objects, and a PREMO system provides a factory finder ob-
ject that is able to locate a factory capable of producing an object that will meet given
constraints.
2.5 The Multimedia Systems Services Component
Multimedia systems typically integrate a variety of logical and physical devices. For ex-
ample input and output might involve devices such as video cameras, microphones, and
a sophisticated speaker system. Processing in turn may involve logical devices such as
data encoders/decoders, media synthesizers (e.g. a graphics renderer), and a video mix-
er. The data produced an consumed by these devices takes a variety of forms, for exam-
ple a discretised audio signal, a sequence of video frames, or a discrete graphics model.
In turn, these forms can be encoded in a variety of formats (ALAW and ULAW for au-
dio, for example). Finally, different protocols may be available to communicate such
data, depending on the source and destination hardware, and on the available network
infrastructure.
As explained in the introductory chapter, PREMO does not aim to define new stand-
ards for the encoding or transport of media data. Rather, it seeks to provide a set of fa-
cilities that abstract away from the details of low level system services, instead
providing an application developer with a uniform high level view of media processing.
To this end, the multimedia systems services (MSS) component of PREMO defines the
infrastructure for creating and maintaining a network of heterogeneous processing ele-
ments for media data. This includes object types for describing generic resources, de-
vices, and facilities for organising a collection of such components into higher level
units with a single interface. MSS encompasses mechanisms by which media proces-
sors can advertise their properties for network construction, can be interconnected and
controlled, and can be configured dynamically to match the needs of a network while
in operation.
MSS was originally defined by the Interactive Multimedia Association [45], a large
consortium of industrial vendors and developers. IMA were aware of the work within
SC24 on the development of PREMO, and donated the MSS framework to the Commit-
tee. It was subsequently adopted by SC24 as the basis of a distinct PREMO component.
During the development of the standard, several of the main provisions of MSS were
refined and integrated with facilities from the Foundation component.

232.5.1 The Paradigm of Media Networks
In order to abstract away from the details of specific media types, media processing el-
ements are viewed as “black boxes” that can be interconnected through a high-level in-
terface to construct a network of such elements appropriate for a given application. At
this level, a PREMO application using MSS resembles a dataflow network, where the
nodes correspond to media processors, and the data streams carry media content. The
adoption of a dataflow-oriented view of media system architecture is not peculiar to
PREMO. It has appeared in published approaches to multimedia systems (for example,
[34]), and is also increasingly used in “plug and play” applications environments, for
example the visualization tool AVS [84] uses such a model for interactive construction
of applications from a toolkit of basic modules.
Figure 2-4 contains an example of a small network. It represents a video engine
combining input from a local file (for example, in MPEG) with audio clips stored as me-
dia primitives within a remote database (scene). The audio primitives in the scene are
constructed by a number of audio modellers (MIDI devices, or waveform editors, for
example). The combined audio/video output is presented on a TV device.
The devices in the figure are all instances or subtypes of specialised object types de-
fined in the fourth component of PREMO, and which is discussed in section 2.6. What
makes the construction and operation of such a network possible are that all of the ob-
ject types involved extend the virtual device and resource concepts defined in Part 3 of
PREMO. This allows the devices to be connected together, and subsequently to ex-
change media data along the streams shown. In the remainder of this section we de-
scribe the principle concepts and types that the MSS component provides for the
creation of such networks.
2.5.2 Virtual Resources
A high level view of a media network is of a collection of resources that cooperate in
the task of creating or processing media. These resources encompass physical devices
(such as cameras or mixing suites), software processes such as graphics renderers and
audio filters, as well as supporting infrastructure such as connections and software for
sceneaudio
modeller
TVvideo
enginevideo file
handler
audio
modeller
Figure 2-4 — Simple Multimedia Network

27
If the underlying devices are located on the same hardware, a connection may be re-
alised by directly linking the input and output ports of the associated devices. More gen-
erally, the devices will be on distinct, possibly remote, machines and using different
local facilities for inter–object communication. In such cases a virtual connection may
need to create a virtual connection adapter, that provides appropriate interfaces to the
end-parties while managing any recoding or translation of raw data required. Connec-
tion adapters exist only as concepts within the PREMO standard. They do not corre-
spond to any particular object type, and in fact their implementation will in general
require a collection of objects to manage the transfer between the different protocols.
2.5.6 Higher-Levels of Organization: Groups and Logical Devices
Even the simplest non–trivial media network, involving two devices with a single con-
nection between them, involves a significant number of objects: the devices themselves,
the connection, the connection adapter (if needed), event handlers for the ports and de-
vices, and possibly supporting objects to, for example, monitor quality of service. For
a realistic application, the number of objects is significantly greater, and the problem of
tracking which particular groups of objects are relevant to any given part of the network
becomes significant.
To prevent organizational anarchy, it is often convenient for clients to interact with a
single object that represents each “significant component” of a network. PREMO pro-
vides a Group object type to support management of a collection of devices and con-
nections. Groups are resource objects which control a number of other virtual resources
(in particular devices and connections), and their respective network. By default, the
constituent devices remain hidden to the external client; instead, groups provide a single
entry point for stream control, as well as other services. By using the basic group inter-
face, the client does not have to know about the interfaces of these constituent devices.
Because Group inherits from VirtualResource, each group is itself a resource, and
consequently, the configuration of group components can be validated, and the compo-
nents themselves acquired, via the one group interface, rather than individually. Be-
cause a group is itself a resource, a group can itself be a member of a further group.
Although groups can be organized into hierarchies, it is important to remember that
a group is not a device; it has no ports of its own. Instead, a client using a number of
groups is responsible for ensuring that, where necessary, components of distinct groups
are connected. A specialised form of group, called a logical device, combines the cen-
tral resource management capabilities of a group with the processing model of a virtual
device. Resources are added to and managed by a logical device in the same was as a
group, but the client of a logical device can also dynamically define ports on the inter-
face of the device. When defined, each port on the logical device is associated explicitly
with a port on a device that it manages. A logical device thus acquires input and output
ports, and can be built into a network in the same way as other devices.

30
2. It supports an approach to application development based on interconnecting a
number of processing devices — irrespective of whether those devices are operat-
ing on continuous media such as video, or a series of discrete data sets within a ren-
dering pipeline. Once such a network has been defined, it can be used for a variety
of data sets or models, and can be readily modified. In contrast, in an architecture
where (for example) graphics objects render themselves, the control of processing
and flow of data is encoded within specific operations, making it difficult to
develop an application that can be modified or extended without wholesale repro-
gramming of those operations.
By opting for a model in which media data is essentially passive, while media proces-
sors are active objects that provide services, PREMO aims to provide a uniform, inte-
grated treatment of both digital media and synthetic graphics.
2.6.2 Primitives
The potential domains of application for a system such as PREMO are diverse. When
considering the design of a component for modelling and rendering, this raises the dif-
ficult problem of identifying an appropriate set of ‘media primitives’ — or indeed,
whether to include any model of primitives at all. Two directions initially appear feasi-
ble when considering how primitives for modelling and rendering could be supported
in a system like PREMO. First, it would be possible to take an existing set of primitives
from an established system, for example the node set provided by Open Inventor [89],
and adopt these to the needs of PREMO, possibly through some further extensions. The
problem here is in finding a set of primitives suitable for the range of applications ad-
dressed by PREMO, and then deciding on what, if any, extensions to include. The sec-
ond approach is to derive some minimal framework of elementary primitives from
which those used in practice can be derived by composition.
Although an interesting research problem, both this and the first approach are biased
towards a model in which PREMO devices for modelling and rendering would effec-
tively be implementing a new standard for graphics primitives. It is simply unrealistic
today, given the investment in graphics and media technologies, to expect industries to
adopt a new standard for media data. Instead, the philosophy underlying PREMO is to
view the standard as a framework for supporting the integration of different modelling
and rendering technologies, with their own models of media data, within a heterogene-
ous distributed system. This has already been reflected in the discussion on virtual de-
vices, where we noted that the virtual device specification does not mandate any
specific strategy for implementing the processing element, thus allowing existing media
processors to be accommodated.
In this context, the role of primitives is rather different from their role in a detailed
standard such as PHIGS [44]. PREMO clearly cannot attempt to describe a closed set
of primitives for modelling and rendering. Instead, it defines a general, extensible
framework that provides a common basis for deriving primitive sets appropriate to spe-
cific applications or renderer technologies. Graphics modellers, for example, may use
specific representations such as constructive solid geometry, NURBS surfaces, particle
systems etc. Audio modellers may use primitives that represented captured waveforms,

33
The MRI component defines a subtype of MRI_Device called a Coordinator.
Such a device encapsulates a number of other media devices (derived from Virtual-
Device), each of which provides the coordinator with one input port. The coordinator
itself has one input port, and as it receives primitives in MRI_Format, the coordinator
is responsible for decomposing any structured presentation into components that can be
processed by the devices that it encapsulates. In the example, the coordinator may re-
ceive presentations that involve synthetic graphics, video, and audio components. The
audio component of the presentation is delegated to the logical device responsible for
audio rendering, while the graphics and video are managed by the second logical de-
vice. The coordinator is also responsible for ensuring that its components maintain any
synchronization constraints captured by the overall presentation. It may achieve this by
monitoring the overall end-to-end progression of its encapsulated devices, and placing
synchronization constraints on those progression spaces, or by using more specific
mechanisms available within PREMO or a given implementation.
2.7 Closing Remarks
This chapter has presented an overview of the PREMO standard. In the process, we
have set out some of the design constraints that have determined the shape of the stand-
ard, and have discussed some of the alternatives that were considered. The description
of PREMO object types and their behaviour has been, by necessity, incomplete and in-
formal. In the chapters that follow, each of the four components will be presented in de-
tail, including the specific interfaces defined for the object types mentioned here. By
giving an up–front view of the overall provisions of the standard, it is hoped that the
reader will be better able to relate the detailed account of the object types, as they are
given, back to the overall picture of what PREMO is intended to achieve.

35Chapter 3
The Fundamentals of PREMO
3.1 Introduction
To date, standards for computer graphics APIs have inevitably implemented in a low-
level procedural language such as FORTRAN [82] or C [58]. The principle abstractions
that these languages support are function and procedure headers, and type or constant
definitions. An API standard could be described as a collection of data types and ab-
stract procedures, which would then be mapped through a language binding into func-
tion or procedure definitions in a specific host language. Critically, there were few, if
any, assumptions in the standard itself about how functions or procedures behaved. The
main difficulty, at least in the early language bindings, was deciding how to cope with
differences in the expressive power of the programming languages, e.g. FORTRAN did
not allow enumerated type definitions, so any use of ‘conceptual’ enumeration types in
the standard needed to map onto a set of constants within a FORTRAN binding.
At one level, PREMO represents a straightforward evolutionary step; rather than
binding the standard to the facilities of a procedural language, the standard is intended
to utilise the facilities provided by object oriented programming systems. A language
binding for PREMO is able to map entities in the standard onto mechanisms such as
classes, methods, and inheritance provided by an object-oriented language. However,
this view misses a significant difference between PREMO and previous graphics stand-
ards. In the case of PREMO, the implementation technology, i.e. objects, and the mech-
anisms for their definition and interaction, are an intrinsic part of the standard. Before
we can give a precise definition of the types and services of PREMO, it is first necessary
to set out what we mean by terms such as ‘object’, ‘object type’, and ‘inheritance’. Not
only will these affect how we go about binding PREMO to a programming language, it
will also affect how we construct complex entities in the PREMO specification from
simpler ones.
The collection of definitions that set out the concepts with which PREMO is de-
scribed form the first component of the standard, referred to as the Fundamentals of
PREMO. This chapter describes the fundamental component, following closely the
structure of the Part 1 document, but also explaining the rationale behind decisions, in
particular those that have consequences on subsequent components and the reference
implementation described in this book. Readers with a background in object oriented
systems or modelling may find that they are familiar with some of the material and can
skip sections. However, for at least issues (object references, and operation request
modes) the approach taken in PREMO may be unfamiliar to larger sections of the audi-
ence.

37
Each object in a PREMO application is created as an instance of an object type. Ob-
ject types correspond to the concept of a class in many object-oriented programming
languages. The term ‘object type’ is used in the standard to make explicit the distinction
between the specification of an entity in the PREMO standard, and the implementation
of that entity in a particular programming system. There need not be a one-to-one cor-
respondence between object types described in the PREMO standard and the classes in
an object-oriented implementation of PREMO. In principle at least, it is possible to im-
plement PREMO in a non object-oriented language. For similar reasons, the standard
uses the neutral terms ‘operation’ and ‘invocation’ to refer to the behavioural compo-
nent of objects, in place of the ‘method’ and ‘message passing’ which are biased to-
wards object-oriented implementations. An object type introduces variables that will be
part of the state of each instance of that type, and similarly, operations that will be avail-
able in the interface of each instance1). PREMO, like mainstream object-oriented lan-
guages such as C++, Ada’95 [7], and Java [36] has adopted a static object model, where
the structure of an object is fixed by its type, and cannot be modified at run-time. Mod-
els have been proposed in which the structure of an object can be modified dynamically,
i.e. while the system is running. Approaches such as CLOS [11] and Smalltalk [32], that
include some capability for reflection’, are good examples of this. A restricted dynamic
object model is also found in the Python language [87], where an object can gain new
operations. Although there is some evidence that multimedia systems might benefit
from the use of a more dynamic model, the practical difficulties of realising this kind of
behaviour in the languages currently in widespread use meant that the PREMO com-
mittee opted for a more traditional, static model. However, some support for dynamic
structures was subsequently provided through the properties mechanism, discussed in
Chapter 5.
3.2.2 Attributes
The restriction that all access to object state be via operations is strong, and in a number
of cases imposes a significant overhead. For example, a device for playing media may
have a number of parameters, such as speed of play, where it is intended that clients are
free to inspect and modify these parameters as needed. Although the value of these pa-
rameters will affect the behaviour of the object, the operation of setting or accessing the
parameter does no further work in itself. If the strict encapsulation model is followed in
detail, each such parameter in an object type must be defined implicitly through a pair
of operations for accessing and modifying the parameter. Doing this throughout the
standard would have added considerable overhead to the specification of the object
types. To overcome this, object types in PREMO can introduce attributes, which can
further specialised into read-only, write-only, and read-and-write (the default). A read-
only attribute can be seen as a shorthand for an operation that retrieves and returns the
1) The possibility of subtyping (see section 3.4) means that instances may also contain variables
and operations inherited from supertypes.

38
value of an internal state variable. Dually, a write-only attribute defines an operation
that sets the value of a corresponding state variable. The default, an attribute that is both
readable and writable, has both operations.
For read-and-write attributes, an implementation of a PREMO object type may be
able to realise the attribute simply as a variable that is publicly accessible, e.g. a ‘public’
state variable in C++ or Java. However, it should be noted that setting the value of an
attribute can result in an exception, and to implement this it may be necessary to realise
the attribute by a pair of methods. In any case, few languages support read/write per-
missions for variables, and consequently most attributes in the PREMO standard will
map on to methods in an implementation.
3.2.3 Non-object Types
The ideal of object oriented programming, that every entity in the system is an object
that can be manipulated through its interface, has probably found its purest expression
in systems such as Smalltalk and SELF [83]. In this environment, all entities – even
things such as numbers and characters – are construed to be objects. This brings a great
simplicity to the language, but creates a significant difficulty. All data manipulation
must (at least conceptually) be carried out by invoking operations, even things such as
adding one number to another. This overhead can be reduced with good optimisation
technology, but the resulting performance is still some way from that obtained with
more direct methods of storage and operation. Partly for this reason, object-oriented
languages have either been defined by adding features to existing programming lan-
guages (as in C++ and Ada’95), or by defining new languages that contain traditional
data types in addition to objects (for example, Eiffel [66]and Java). Efficiency concerns
aside, this approach also benefits from easing the migration path for existing program-
mers.
For very much the same pragmatic reasons, PREMO from the outset makes a distinc-
tion between object types, and non-object types. Each non-object value is a member of
some non-object type, but non-object types are not part of the object type hierarchy de-
scribed in section 3.5. Values of non-object types are atomic, there is no concept of a
state or operations, rather, it is assumed that the environment in which PREMO operates
provides a set of operators for manipulating non-objects, for example arithmetic and re-
lational operators for working with integers and other numeric formats. There is how-
ever an important bridge between the world of objects and the world of values, which
is discussed in the next section.
Part 2 of PREMO (see Chapter 5) introduces a collection of non-object types that are
used widely in the remainder of the standard. Other PREMO components can and do
introduce further, specialised types (in the form of subranges or enumerations).
3.2.4 Object Identity and Object References
A basic tenet of the object-oriented metaphor is that each object has an identity that per-
sists, independent of the changing state [13]. Many object-oriented programming lan-
guages treat objects as pointer variables, with the object itself being but a pointer to

41
have at least an x and y coordinate. A video media stream may pass from a device which
reads data from an external source to a device that displays the video; while the devices
may be objects of different object types, both however may have the capability to stop,
pause, and resume their processing tasks. Subtyping is the relationship that holds be-
tween two object types when objects of one type can be used in contexts where objects
of the first type are ‘normally’ expected. In the examples above, a 3Dpoint object type
may be a subtype of 2Dpoint, while both the video stream producer and render may be
subtypes of a generic object type representing video devices. The definition of a PRE-
MO object type should indicate which object type(s) it is a subtype of.
Exactly what should and should not constitute ‘a subtype’ has been the subject of
much debate within the object-oriented systems communities (see for example [88]),
and indeed different approaches can be required at various stages of the software devel-
opment process. PREMO bases its notion of subtype on the interface of the object types.
For T to be a subtype of S, the following conditions must hold:
1. for each operation OPS in S, there should be an operation OPT in T with the same
name;
2. the number of parameters accepted by OPS and OPT should be the same;
3. with the exception of the first parameter, each the type of each parameter to OPT
should be the same as the type of the corresponding parameter to OPS. The excep-
tion is because the first parameter to an operation is taken to be a reference to the
object on which the operation is dispatched. Therefore this type will by definition
be distinct in different object types.
Two concepts which are useful in discussing subtyping are direct instance, and im-
mediate subtype:
• Each object in PREMO is a direct instance of exactly one type, which is called the
object’s immediate type. If O is a direct instance of T, this means that O is an
instance of T, but not an instance of any subtype of T. Intuitively, an object is a
direct instance of the object type used to create the object.
• An object type T is an immediate subtype of object type S, if, informally, T is
‘immediately beneath’ S in the object type hierarchy. More precisely, T must be a
subtype of S, and there must be no other object type U such that U is a subtype of S,
and T is a subtype of U.
An object type can have any number of subtypes, and can also have any number of
supertypes, that is, it can be a subtype of multiple object types. An object type, plus the
collection of its supertypes, plus their supertypes, etc., forms a directed graph called the
type graph of the object type. New subtypes are created by subtyping from some exist-
ing PREMO object type. In particular, all PREMO objects are subtypes (directly or in-
directly) of the type PREMOObject, which defines the minimal functionality of each
object in a PREMO system. This includes:
• operations to enquire an object’s type and type graph;
• an initialize operation that is performed on an instance when it is first created; and

45
type of its immediate type, i.e. an object type that appears in the type graph of the im-
mediate type. This is supported by facilities defined in Part 2 of PREMO that allow all
objects to enquire their type graph.
3.8 Operation Request Modes
In simple models of program execution, operations are usually carried out as synchro-
nous processes, with the caller suspending until the operation has been completed and
the result (if any) is returned. This model of course breaks down in the presence of con-
current processes, and is untenable in a distributed system where the overhead of locat-
ing and communicating with a remote object can be non-trivial compared with actual
execution times. Distribution has been a fundamental design goal in PREMO, and con-
sequently the PREMO object model must address the problem of how operation invo-
cation takes place. In PREMO, each operation on an object is defined to operate in one
of three possible request modes. These modes are called synchronous, asynchronous,
and sampled. The mode is an immutable property of an operation, specified when the
operation is defined. A subtype can override the implementation of an operation, but
cannot change the mode of the operation.
• Asynchronous operation request causes the caller of the request to be suspended
until the request has been serviced and a result returned. This is the usual model
found in and supported by default in most object oriented programming languages.
• The caller of an asynchronous request can continue its own thread of control as
soon as the call is made; at some point the requested operation will be invoked. By
the time the operation has been completed, the caller may be carrying out some dif-
ferent task, and it is not therefore possible to return a result. Communication
between asynchronous processes is a well known engineering problem, involving
techniques such as shared variables with mutual exclusive access.
• A sampled request is similar to an asynchronous request, except that any pending
request for a given operation (i.e. a call that has not been serviced) is over-written
by any new request. Conceptually, each operation has a 1-place buffer for storing
pending requests. Sampled mode has been supported in PREMO for tasks where the
rate at which an object can be informed of, and service, requests may be slower than
the changes that are causing clients to make requests. This may happen for example
where a server graphical objects in a distributed VR environment is being asked for
detailed object geometry by clients managing the interaction of remote participants.
Some care is needed in dealing with the suspension that results from invoking a syn-
chronous operation. When suspended, an object can still receive requests from other ob-
jects, which are managed in accordance with the behaviour described above. For
example, a synchronous request on a suspended object will be held as pending, with the
caller itself suspended, until the callee’s own invocation is completed and it is able to
service the request. While suspended however, an object can continue with its own in-
ternal thread of processing; it just cannot access information related to its own recep-

50The core of this book describes the object interfaces and the semantic behaviour of
the objects only, to give a thorough presentation of the reference model advocated by
PREMO. However, a separate appendix gives also some more details of the implemen-
tation itself, for those who want to understand what happens “behind the scenes”, and
who may feel challenged to provide a full implementation of PREMO.
The choice of the programming language and the environment used for the imple-
mentation has a consequence on the way objects will be presented throughout this book.
The current chapter concentrates on some of the general issues and constraints which
directed our choices and how these constraints will be visible in the various type spec-
ification in the rest of this book. Of course, the problems we will describe, and the an-
swers we have found to those problems, reflect the chosen environment as well as our
own software engineering abilities, and the reader might perfectly be capable of greatly
improving our design. But that is all right; the goal of this book is to understand the ref-
erence model of PREMO, and an efficient implementation would not necessarily be the
most direct and clear account of PREMO.
Figure 4-1 — Object Specification in the ISO PREMO Document
PropertyInquiryabstract
EnhancedPREMOObject
inquireNativePropertyValue
keyin : Key
nativeValueout : seq Value
exceptions : {InvalidKey}
The operation returns the native property values for keyin. This native property value
represents the value or values the project instance can take on for keyin. A native prop-
erty value is available for all properties which are defined as part of the object’s func-
tional specification.
The nativeValueout is typically a sequence of values (e.g., if the type of the correspond-
ing property is defined as a String), or a minimum–maximum range (if the value is a
numerical type). The specification of the property shall define the return type of the re-
sult of operation invocation if this is not the case.
Exceptions raised:
InvalidKey The key is invalid.
PropertyInquiry

52
• are the interfaces of object instances fixed (e.g., is it possible to add new attributes
or methods to an object instance such as, for example, in Python or a delegation
based model);
and the list continues…
As a consequence of this diversity, if a very precise abstract specification is to be de-
fined, this specification must include its own object model description. This has been
the case for such industrial specification as CORBA[71]: OMG has its own specifica-
tion of what objects are in OMG’s point of view. And this is the case with PREMO, too;
an important portion of the so–called “fundamental” part of PREMO (see Chapter 3)
has been devoted to the definition of the object model which PREMO relies on. Al-
though this object model is not particularly different from what people usually think of
objects, it had to be precisely defined, and this model did influence our choice for a pro-
gramming language as far as our implementation was concerned.
The language we have finally chosen is Java. Although the object model of Java is
not 100% identical to PREMO’s model, it is probably the closest we can get. Apart from
its large popularity and wide availability, the following features of Java influenced our
choice:
• Java has exceptions as part of the language (exceptions are also used in PREMO);
• threads are integral part of the Java design;
• Java has a large and very useful set of utilities in terms of the various Java pack-
ages;
• good portability across numerous platforms, including both the language and the set
of ‘core’ Java packages.
As a consequence, all object specifications, example code, etc., in this book will be in
Java1).
Java uses the concept of ‘packages’ which offers us a nice way of structuring the im-
plementation, as well as the various interfaces of PREMO. Our implementation is split
into the following packages:
premo.std.part2
premo.std.part3
premo.std.part4
premo.impl.part1
premo.impl.part2
premo.impl.part3
premo.impl.part4
premo.impl.utils
The content of these packages is straightforward: the ‘std’ packages contain those
classes and interfaces which are the direct counterparts of PREMO specifications, the
‘impl’ packages contain the various implementation specific classes and interfaces.
The premo.impl.utils package separates those implementation classes which are
1) We will rely on the familiarity of the reader with Java. Apart from the book on Java by the designers of the
language[36], there are a large numbers of other books available (e.g., [28] or [39], to refer only to those
used by the authors), and the reader may consult these if necessary.

53
used in all parts of PREMO; the premo.impl.part1, etc., packages contain the im-
plementation classes which are specific to a PREMO part. Part 1 of PREMO does not
include object type specification, i.e., it does not appear in this list as a std package;
however, some of the requirements of the object model lead to the development of spe-
cial facilities, which constitute the premo.impl.part1 package.
4.1.2 Implementation Environment
Having an implementation language is not enough; a full implementation environment
has to be chosen, too. This entails the usual utility and I/O facilities, basic data struc-
tures such as hashtables, vectors, etc. Java offers a standard set of packages (the “java
core”) which are available with all Java implementations, and which are perfectly ap-
propriate for our implementation (e.g., the java.lang, the java.utils, and the
java.io packages). There are no real problems in this respect.
There is, however, one area where a further choice has to be made, and this is distri-
bution. Indeed, one of the main characteristics of PREMO is that it should function in
a fully distributed environment, i.e., some of the PREMO objects can be accessed and
invoked through a network. This means that, beyond the choice of Java, we also have
to choose which kind of tool we would use to manage distribution. It also turns out that
his choice has its (albeit minor) consequences on the way the various specifications in
the book are defined and presented.
At the time of writing this book, there are, broadly speaking, three alternatives one
could choose from to control the distributed access of Java objects (see, e.g., [70] or
[86] for further details):
4. Microsoft’s DCOM (Distributed Component Object Model) used with Java (i.e.,
Visual J++);
5. Some form of an OMG CORBA implementation with a Java interface (ILU,
Netscape’s ONE or Caffeine, OrbixWeb, etc.); or
6. Sun’s (i.e., Javasoft’s) own RMI (Remote Method Invocation) facilities, bundled
into Sun’s JDK (Java Development Kit), starting from version 1.1.
We ruled out the usage of DCOM for portability reasons; we did not want to have an
implementation dependent on only one environment. The choice between RMI and a
CORBA implementation is less obvious. CORBA offers a greater compatibility with
other object environments and programming languages, and a richer set of functionality
with respect to object interface registry (although, by the time the reader reads these
lines, Sun may come up with additional registry services for RMI which are compatible
with CORBA). On the other hand, using CORBA means having to access and use yet
another large piece of software besides the Java environment itself, which may prove to
be a significant burden for portability (not all CORBA implementations are available
on all platforms where Java runs). We have therefore opted for RMI. Provided we use
Java JDK 1.1 or higher, this provides us with the level of portability Java itself has; it
also has the functionality necessary for our prototypical implementation. It must be em-
phasized, though, that a fully marketable implementation of PREMO might well prefer
to choose CORBA rather than RMI; development in this area is so rapid that our choices

54
may have been different had they occurred at a later time. Actually, the design philoso-
phies behind CORBA and RMI are very close to one another, so most of the specifica-
tion and code in this book would be valid for a CORBA environment, too. Both
environments rely on the concepts of client and server stubs, and the major differences
between the two approaches can be localized in the way these stubs (i.e., remote ob-
jects) are created, and the references to the server objects are accessed. As we will see
in Chapter 5, the abstract PREMO specification localizes these functionalities into one
or two objects anyway, so the differences between the RMI and corresponding CORBA
implementation are minor and easily manageable.
There is one important difference, however. In the case of CORBA, all objects in the
CORBA world are, per definition, remotely accessible, hence only their references
should and can be passed as method arguments. Java’s RMI, on the other hand, allows
the user to differentiate between objects which are remotely accessible (i.e., they ‘reg-
ister’ as remote server, have a server and client stubs) and objects which cannot be ac-
cessed through the RMI mechanism but still exist in their own right. When transferring
references to objects in the second category, RMI passes the objects by value[91].
The possibility of passing objects by value is important from PREMO’s point of
view. It so happens that the PREMO specification also differentiates between objects
which have their services available through the network, e.g., full–blown multimedia
players, and objects which are short–lived, simple, and are not supposed to provide
complicated services in a distributed environment, e.g., a geometric point (see
section 5.3.4). The distinction made by the RMI designers between ‘remote’ and ‘local’
objects perfectly suits our needs.1)
4.2 PREMO Specifications in Java and Java RMI
4.2.1 Constraints on the Specification Details
Using RMI for the specification of potentially remote objects creates some constraints
as to how the abstract PREMO object type specifications should be described in Java
terms. First of all, in Java’s RMI, stubs are created for interfaces and not for classes. In
other words, one has to define a remote interface, which is used by the rmic stub gen-
erator program to create the client stub and the implementation skeleton. The real serv-
ice itself is supposed to be defined through a separate remote server class which
implements the remote interface. This means, in practice, that the public interface of all
PREMO objects which are remotely accessible (a more precise definition will be given
in Chapter 5) will be defined as Java interfaces and not as Java classes. These interfaces
will be placed into the ‘std’ packages. The implementation has to provide classes
which implement these interfaces; these will form the ‘impl’ packages.
1) Passing objects by value is not yet defined in CORBA, but OMG has issued a Request for Proposal in
1996, and a specification may become available by the time this book goes to press. So, on long term, this
difference may disappear, too.

56
SysClock sysClock;
sysClock = (SysClock)GetTheObjectReferenceFromSomwhere();
int time;
try {
time = sysClock.inquireTick();
} catch( java.rmi.RemoteException e ) {
System.out.println(e.toString());
}
although, as we said, we will not always include the try and catch clauses into all our
examples. The GetTheObjectReferenceFromSomwhere is obviously a dummy
method call for now; what should be noted, though, is that the method could return ei-
ther a reference to a local object or a reference to a local stub; the code remains identical.
Average users of PREMO may refer exclusively to the interfaces defined in the ‘std’
packages. By the very nature of Java interfaces, these describe the publicly accessible
methods. However, PREMO also defines some protected methods, i.e., which are of im-
portance for subclasses only. These are of interest for programmers who wish to extend
existing objects. These protected methods are included in the ‘_Impl’ classes; indeed,
extending existing PREMO object types means in practice that these classes should be
extended in the Java sense.
As said before, this “dual” structure, i.e., the separation of a separate interface and
its implementation, is valid for those PREMO objects which may be used as server ob-
jects through RMI. PREMO also includes some simpler objects, which are implement-
ed in a more “straightforward” manner; more about it in the coming chapters. Also, the
choice of Java and Java’s RMI have some other, minor consequences on the way spe-
cific abstract PREMO specifications are mapped onto an implementation. These conse-
quences may be better understood if the specific PREMO concepts are also known, so
we will have to come back to those in later chapters.
4.2.2 Registering Server Objects
An issue which has not been addressed up to now is how an object instance, once cre-
ated, becomes a server object, i.e., how would its methods become available for remote
method invocation.
The usual approach when using RMI is that the implementation class would extend
from a java.rmi.RemoteServer class; in the current release of JDK this can be done
by extending it from java.rmi.UnicastRemoteObject. By ensuring that the con-
structor of the superclass is invoked at instantiation time, such an object instance would
then automatically be registered as a server object. We say “usual”, because most of the
Java textbooks, when describing RMI, describe this approach only.
This scheme, however, would lead to problems with the PREMO implementation
classes. Indeed, Java does not allow for multiple inheritance; i.e., if a server class would
already have to extend java.rmi.UnicastRemoteObject in order to become a serv-
er object, the code of, for example, SysClock_Impl above would become invalid. Fur-
thermore, this would mean that all instances of the given object type would become
server objects which, though semantically acceptable, may have some negative conse-
quences on efficiency.

60
1. Basic data types. This is defined as a small set of non–object data types describing
fundamental entities such as integers or floats. All the basic data types are defined
in Part 2 of PREMO.
2. Constructed data types. It is possible to construct new non–object types using exist-
ing ones. A characteristic example is the construction of arrays. All Parts of
PREMO contain a small number of such constructed data types, related to the
proper specification of “real” PREMO objects.
3. Exceptions. These are, strictly speaking, not defined as non–object types in the
PREMO documents, but they logically belong to this category. Error management
in PREMO is based upon the ability to throw exceptions from within a method,
exceptions which can be then caught by the caller. Exceptions carry information
which can be extracted by whoever catches them.
It may be a source of confusion that some of these entities, referred to as “non–objects”
in PREMO, will be represented by Java classes, i.e., objects. This is the consequence of
the nature of Java, which does not have structures, enumerations, etc., and encourages
the programmers to use classes even for those relatively simple entities1). However,
these “non–object classes” have some characteristics in common, which differentiate
them from the Java classes representing “real” PREMO objects. Namely:
• They are all final classes, i.e., they cannot be extended by other classes.
• They are not part of the PREMO object hierarchy.
• They are never defined as possible server objects for RMI.
• They all implement the java.io.Serializable interface, i.e., they can be
passed as arguments through RMI calls.
• They are not active entities (i.e., do not run in separate threads), whereas PREMO
objects make extensive use of threads.
When necessary, we will refer to these classes as “non–object classes”.
The subsequent sections will give some details on each of the non–object categories.
5.2.1 Basic Data Types
Most of the PREMO basic data types are fairly straightforward, and they have their di-
rect counterpart in Java. These are:
1) More fundamentally, there will rarely be a direct mapping between the PREMO view of objects and non–
objects, and that of any target programming language.
PREMO non–object types Java types
Integer (Z) int
Real (R) double
Object type java.lang.Class
Time long

61The type names, as appearing in the PREMO document, are not reused literally, and this
may be a source of confusion for whoever wants to consult the original PREMO docu-
ment while reading this book. The main reason is the relative inflexibility of Java in this
respect. Indeed, whereas in C or C++ it would be possible to say, e.g.,
typedef long Time;
which just gives a new name to an existing type, this facility does not exist in Java. The
only other possibility would have been to wrap all PREMO non–object types into sep-
arate Java classes for the purpose of renaming, which would have been an unnecessary
complication. In this book, we refer to the Java names only.
The “Time” non–object type is used to describe the ticks of a clock, and plays an im-
portant role in multimedia synchronization. PREMO gives the implementors the choice
of choosing either a real number or a large integer number to represent time. We have
chosen the latter, based on the fact that Java’s view on elapsed time (see, e.g., the cur-
rentTimeMillis method of the java.lang.System object) uses long integers, too.
The PREMO standard makes extensive use of a non–object data type called Value,
which acts as a union type encompassing all of the other PREMO non–object data types,
including object references. Java does not provide a union constructor over its basic
data types. Instead, our PREMO Java binding uses the Java class Object in place of
value. PREMO non–object values (which are basic values in Java) are then represented
by instances of the so–called Java “Envelope” classes when they appear as values of this
union type. For example, a value of type int, when used in the union type, is represented
as an object of type java.lang.Integer.
5.2.2 Constructed Data Types
Creating arrays is probably the most important non–object data mechanism used in
PREMO. These are represented by Java arrays. Internally, when variable length arrays
are necessary, java.util.Vector is also used, but this does not appear on the level
of interface specifications.
Another mechanism is the creation of simple classes, called structures, for collecting
attributes (i.e., public variables) of other non–object types. The only method imple-
mented in a structure is the equals method (automatically inherited from
java.lang.Object) to ensure a proper use (comparing the constituent attributes rath-
er than the object references), as well as obvious constructors to fill the attribute values.
They may also appear as nested top–level classes as in the following example:
Object references implicitly part of the language, no
separate type is necessary
Boolean boolean
String String
PREMO non–object types Java types

655.3.2 Simple PREMO Objects
Simple PREMO objects are, conceptually, data structures needed for the proper speci-
fication of various multimedia service objects. They are very similar to constructed
non–object data type classes (see section 5.2.2) in the sense that the intention is a com-
pact representation of entities such as a geometric point, an event, or a constraint spec-
ification. The similarity is also reinforced by the fact that simple PREMO objects do not
define multimedia services, e.g., over a network. The major difference between con-
structed data types and simple PREMO objects is that the latter are part of the full PRE-
MO hierarchy, and they can also be subject to various subtyping patterns. For example,
it is possible to build a complete hierarchy of various events using subtyping (this is
done very frequently in various interactive systems).1)
Formally, simple PREMO objects are defined to be subclasses of the following class:
package premo.std.part2;
public abstract class SimplePREMOObject
implements PREMOObject, java.io.Serializable {
}
The class is abstract, i.e., non–instantiable, and it is a direct subtype, in the PREMO
sense, of PREMOObject. Note that the class also implements the standard
java.io.Serializable interface, i.e., instances of this class can appear, for exam-
ple, as arguments of remote object calls.
1) It must be noted that the strong similarity between simple PREMO objects and constructed data types is an
artefact of the nature of Java, and not of the PREMO specification proper. Other languages may offer much
richer data structuring possibilities, such as enumerations and data structures. In this case, most of the
PREMO non–object data types could be described independently of the object/class hierarchy.
PREMOObject
simple PREMO objects enhanced PREMO objects
callbacks
Figure 5-1 — Main categories of PREMO objects

66
The various simple PREMO objects are all subclasses from SimplePREMOObject,
either directly or indirectly. To reinforce the “data” nature of these classes, they are usu-
ally defined in terms of public variables and not methods. These variables are also re-
ferred to as “structure tags”, and simple PREMO objects are also referred to as
“structures”.1)
Part 2 of PREMO defines only four simple PREMO objects. Other parts of PREMO,
especially Part 4, make a much more elaborate use of them. Two of these simple ob-
jects, ActionElement and SyncElement, are closely related to the semantics of other
objects, such as the Controller in the case of ActionElement, and the synchroniza-
tion objects in the case of SyncElement. They will be defined in later sections. The
two other simple PREMO objects, defined in Part 2, are of a very broad use in the stand-
ard, so it is better to define them in general. They are also very good examples of how
the various constructions described so far converge in concrete type specifications.
5.3.2.1 Event Structures
Event handling and event management play a central role in all dynamic systems, in-
cluding PREMO. Events represent basic building blocks to convey information through
the system in an asynchronous fashion. Events are used to manage interaction, synchro-
nization patterns, to monitor various activities in other objects, etc.
The complete event handling model in PREMO involves several different objects,
which will be defined later (see section 5.4.1). All these objects make use of the simple
PREMO object Event, which carries the necessary information. An Event structure
has a name, which can be used to identify the event itself, it contains an event data
which, at the abstract level, consists of an array of key–value pairs, and an event source,
which is a reference to the object which has created (“raised”) the event instance.
Here is the class specification of Event:
package premo.std.part2;
import java.lang.*;
public class Event extends SimplePREMOObject {
public String eventName;
public static class EventData implements java.io.Serializable {
public String key;
public Object value;
}
public EventData[] eventData;
public EnhancedPREMOObject eventSource;
}
Note the use of a nested top–level class for the specification of a simple data structure
(see also page 61). The type EnhancedPREMOObject is the “root” of all enhanced
PREMO objects, see section 5.3.4.
1) Of course, an implementation may also redefine operations such as equals, inherited from
java.lang.Object.

68
5.3.3 CallbacksManagement of events dynamically is usually achieved by “registering interest” in
some events. This is done by publishing the reference of the interested party. The object
which raises or forwards the event may then notify the interested party of the occurrence
of the event.
The Callback interface is defined to facilitate this mechanism, by defining a single
entry point for any interested party. The interface has the following, very simple defini-
tion:
package premo.std.part2;
public interface Callback extends PREMOObject {
void callback(Event callbackValue);
}
Various enhanced PREMO objects implement this interface, thereby assigning a specif-
ic behaviour to the callback operation.
Although the interface specification is simple, there is an underlying semantics to the
operation callback which must be taken into account when the interface is imple-
mented. Two features should be emphasized:
• If the operation’s semantics is such that the callback value is forwarded to a third
party, or the structure tags are changed, a copy of the event structure should be
made. Indeed, the same event structure can be forwarded (through the callback
operation) to a number of objects, whose identity and number is not known in
advance. Changing the callback values may lead to uncontrollable situations.
• Callbacks are usually used for interaction and synchronization. In other words, in
time critical situations. It is therefore of a paramount importance that the caller to
the callback operation is not suspended for a long time while the operation per-
forms its own activity. The callback operation is therefore defined to be asyn-
chronous (see Section 3.8).
Whereas, in simple cases, the semantics of the callback operation may be defined to
affect the state of the object directly, it is very often the case that this operation acts only
as an entry point to call other operations on the object. To facilitate this second case,
PREMO also defines an interface which extends Callback, called CallbackByName.
This extension does not add any new methods, but overrides the (inherited) asynchro-
nous callback operation:
package premo.std.part2;
public interface CallbackByName extends Callback {
void callback(Event callbackValue) throws OperationNotDefined;
}
The callback of CallbackByName has the following behaviour: the eventName
structure tag of the Event structure (appearing as the input argument of callback) is
interpreted to be the name of a local operation which is then internally invoked by the
callback operation (an exception is raised if the name does not refer to any valid op-

69
eration). By default, all other structure tags of the Event structure are disregarded by
the callback operation. Subtypes of CallbackByName may add an additional behav-
iour to the operation which also takes these tags into consideration.
5.3.4 Enhanced PREMO Objects
Most of the objects defined by PREMO are enhanced PREMO objects. All other cate-
gories of objects, as well as the various non–object types, are defined and used in order
to make the specification of enhanced PREMO objects concise and precise. Enhanced
PREMO objects have a common supertype within the PREMO hierarchy called, not
surprisingly, EnhancedPREMOObject.
5.3.4.1 Enhanced PREMO Objects as Service Objects
A fundamental restriction of PREMO is that enhanced PREMO objects, and only those,
offer services over a distributed PREMO environment. In terms of our implementation
strategy, based on Java RMI, this means that enhanced PREMO objects, and only those,
should be registered as RMI server objects. This means that the “dual” implementation
structure, as described in section 4.2.1, is valid for all enhanced PREMO objects.
Formally, enhanced PREMO objects are those which implement an interface, called
EnhancedPREMOObject, defined as follows:
package premo.std.part2;
import java.lang.*;
public interface EnhancedPREMOObject
extends PREMOObject, java.rmi.Remote {
}
[Note: we have omitted the various methods defined by EnhancedPREMOObject for
now, see section 5.3.4.2]. Note that the interface also extends java.rmi.Remote,
which is necessary to ensure that the object could serve as an RMI server object.
This interface is implemented by a separate class in the premo.impl.part2 pack-
age:
package premo.impl.part2;
import premo.std.part2.*;
public abstract class EnhancedPREMOObject_Impl
implements EnhancedPREMOObject {
}
The various enhanced PREMO objects are implemented by classes extending, either di-
rectly or indirectly, this class (and implementing their corresponding interface, of
course).
5.3.4.2 Property Management
Properties are used to store values with an object that may be dynamically defined and
are outside of the type system. Properties are pairs of keys (i.e., strings) and arrays of
values which are conceptually stored within an enhanced PREMO object (to use anoth-
er terminology, each enhanced PREMO object has an associated dictionary). Opera-

71
void defineProperty(String key, Object[] value)
throws ReadOnlyProperty;
This method adds a new property to the object. If the key identifies a property already
defined for the object, the new value is assigned to the property, replacing the previous
value(s). Otherwise, a new property is created with key and value.
void addValue(String key, Object value)
throws ReadOnlyProperty;
This method adds a value to the properties for the argument key. If the key has not been
used yet, a new property is defined. Both of these methods may raise an exception if the
key refers to a read–only property.
By default, if a property value is defined for a key which already exists for the object
instance, the old value is silently overridden. However, the client has the ability to add
a reference to a Callback object to a property key to monitor those changes. The call-
back is activated whenever a new value is defined for the key. Adding a callback refer-
ence is done through the method:
void setPropertyCallback( String key,
Callback callback,
String eventName )
throws NoKey;
The newly created event instance, forwarded to the callback, uses the name given in the
method argument. The event structure contains the key–value pair corresponding to the
new setting.
5.3.4.2.2 Removal of Properties
Two methods are defined to remove properties from an object. The method
void undefineProperty(String key)
throws ReadOnlyProperty, NoKey;
removes the property altogether, deleting both the key and all the corresponding values,
whereas the operation
void removeValue(String key, Object value)
throws ReadOnlyProperty, NoKey, InvalidValue;
removes a single value from the property defined for a key. (The exception Invalid-
Value is raised if the value does not appear on the property list.) Both of these opera-
tions raise an exception if a read only property is referred to in their argument.
5.3.4.2.3 Property Inquiry Operations
A single property can be inquired through the
Object[] getProperty(String key) throws NoKey;
method. If all the properties are to be inquired, they can also be accessed through the
PropertyPair[] getPairs();

72
method, where PropertyPair is a separately defined non–object data type of the
form:
public final class PropertyPair implements Serializable {
public String key;
public Object[] value;
}
Most of the methods so far could raise an exception if the key was not present, or it re-
ferred to a read–only property. The full set of keys can also be retrieved through the
public static class KeyInfo {
public String key;
public boolean readOnly;
}
KeyInfo[] inquireProperties();
method, which may aid the client to form a proper call sequence.
5.3.4.2.4 Property Matching
Property matching is the most powerful operation among the property management
methods of the EnhancedPREMOObject interface. It allows for a constraint–based re-
trieval of properties, serving as a basis for various negotiation mechanisms occurring in
multimedia systems.
The interface specification of the method is as follows:
public static class MatchPropertyResults {
public PropertyPair[] satisfied;
public PropertyPair[] unsatisfied;
}
MatchPropertyResults matchProperties(Constraint[] constraintList);
(the Constraint structure is defined in Chapter 5.3.2.2 on page 67, and MatchProp-
ertyResults is a top–level nested class.) Semantically, what happens is as follows:
The properties defined for the object are matched against the property sequences in
constraintList. For each key appearing in this constraint list, the values are com-
pared against the value or values stored with an identical key in the object. Comparison
is based on the boolean operation defined by the enumeration ConstraintOp (also de-
fined in Chapter 5.3.2.2), and appearing as the structure tag of constraint list. The left
operand of the operation is the property stored in the object, and the right operand of the
operation is the value appearing in the constraintList structure. If the operation
does not make sense, the result of the comparison is false (for example, “Includes”
for numerical types).
The satisfied array contains those keys with associated values for which the com-
parison has resulted in true. The unsatisfied array contains those keys with asso-
ciated values for which the comparison has resulted in false.
An example will clarify the use of this method. A fictitious audio object may store
the various audio formats it can decode in a property list. The object providing the serv-
ice may define a (read–only) sequence of values for the key “AudioFormatK”, e.g.,
<“AIFF”, “AIFC”>, describing the audio file formats it can recognise. The match-
Properties method may be invoked with a pair consisting of a key and a value:

73[“AudioFormatK”, “AIFF”]
using the comparison operator “Equal”. The result will be:
satisfied: [“AudioFormatK”, <“AIFF”>]
unsatisfied: [“AudioFormatK”, <“AIFC”>]
Another call, using:
[“AudioFormatK”, “IRCAM”]
will result in
satisfied: [“AudioFormatK”, <> ]
unsatisfied: [“AudioFormatK”, <“AIFF”,“AIFC”>]
etc. Based on this information the client can choose the AIFF file format which can be
managed both by itself and the audio service.
The example can be made more complex. For example, using more than one key in
the invocation of the matchProperties operation (e.g., also include sampling size),
and optimizing the calls through the use of arrays of values and the “Includes” oper-
ator instead of “Equal”, further information can be retrieved on the object. This can
serve as a basis for powerful negotiations.
5.3.5 Top Layer of PREMO
Figure 5-2 gives an overview of all the interface and class definitions appearing at the
top level of the PREMO type hierarchy and is a detailed version of Figure 5-1. Only
those simple objects are depicted on the figure which have already been presented.
premo.impl.part2
Figure 5-2 — Top classes and interfaces of PREMO
PREMOObject
SimplePREMOObject Callback EnhancedPREMOObject
Constraint
EnhancedPREMOObject_Impl
Event CallbackByName
premo.std.part2

74
5.4 General Utility Objects
We have, somewhat arbitrarily, re–grouped a few PREMO Part 2 objects under a cate-
gory called general utility objects, although this categorization does not appear in the
original Standard itself. The objects in this category provide some elementary building
blocks which are used in various other places in PREMO, but they are rarely used in
isolation, i.e., without being bound to some other, more complex objects. There are
three groups of general utility objects:
• Event handler objects, which provide an event propagation mechanism in PREMO.
• Controller objects, providing an interface for controlled finite state machines.
• Timer objects, which define the interfaces to measure time in PREMO.
This section will present these objects in more detail.
5.4.1 Event Management
Forwarding information, i.e., data, through operation invocation is a relatively static ac-
tion. The caller has a direct knowledge of the callee (modulo the actual value of an ob-
ject reference), only one callee can be invoked at a time, the callee has no real control
over the occurrence of the call, etc. Whereas this approach to information transfer is ap-
propriate most of the time, it has long been recognized that dynamic systems need a
more flexible way of forwarding information, too. As opposed to a direct call, this dy-
namic form of data transfer should be such that:
• The caller, or the source of the information, should be separated from the callee, or
the possible consumer of the data. The caller does not need to know about the con-
sumer of the data.
• Data transfer should be as asynchronous as possible. The source of the information
should just make the data “known” to its environment, and continue its own activi-
ties.
• It should be possible to have more than one consumer at a time.
• The receiver should have the ability to dynamically control whether it is interested
in the information or not.
Event handling, or event management, has become the standard answer to these con-
cerns, and event management is ubiquitous in dynamic and interactive systems nowa-
days. Events were already present as early as 1982, when the first version of GKS
became more widely available as a technical document. The GKS standard included the
notion of event mode input which had some of the characteristics described above[5].
The notion of events became more familiar through various windowing environments,
such as X11, and is now part of almost all graphics and multimedia systems. Java’s
AWT has the notion of events, too, and the listener–based event model, present in AWT
since Java version 1.1, shares a lot of common characteristics with what will be de-
scribed below for PREMO.

75Just as in the case of a precise object model, various systems have their own view of
event management, but none of the present schemes suffices for PREMO. Consequent-
ly, PREMO defines its own event management model.
5.4.1.1 The PREMO Event Model
Figure 5-3 gives a rough overview of the main notions involved. Events are simple PRE-
MO objects which were already defined in section 5.3.2.1 on page 66. Events have a
name, a source, and they may contain event data. A name is also referred to as event
type, which is a String. A source is the reference to a PREMO object (usually a refer-
ence to the object which creates a specific instance). The event data is conatined in a
sequence of key–value pairs, much like properties. Event management is concerned
about how these units of information are propagated among PREMO objects.
The main feature of event management is the separation between the source of the
events and the objects receiving the events, called also event clients. This separation is
done through the use of a special object in PREMO, called an event handler. Objects,
which intend to propagate an event, send the event instance to this event handler object.
and it is the event handler’s task to broadcast the event instance to a set of event clients.
Objects, which want to become event clients, register to the event handler. If they do
not want to be an event client any more, they can unregister. In other words, event han-
dlers embody a one–to–many propagation of events with a dynamic management of
prospective event receivers. The event source does not know who the event recipients
are for a specific event; it only sends the event to the event handler.
Prospective event clients can register themselves to several event handler instances,
thereby having some control over which category of events they want to receive. How-
ever, registration, and the corresponding event propagation, is much more finely tuned.
Indeed:
1. When registering, the client gives an event name to the event handler, too. When a
new event arrives, the event handler compares the name of the event to the names
which are part of client registrations. Only those registrations are considered where
EventSource
register/unregister
dispatch event c
al
lb
ac
k
Figure 5-3 — Event management in PREMO
EventClient1
EventClient2
EventClientn
EventHandler
dispatchEvent
register

77
The dispatchEvent operation is defined to be asynchronous. Event registration
makes use of the Constraint structure, defined in section 5.3.2.2 (see page 67) and a
simple PREMO enumeration:
public final class AndOr
extends premo.impl.utils.PREMOEnumeration {
public static AndOr And;
public static AndOr Or;
}
The registration operation returns a registration identifier. This identifier should be used
to unregister (an exception is raised if an invalid event registration identifier is used as
argument for unregistration).
Dispatching an event is achieved through the dispatchEvent call although, as pre-
viously stated, the “real” effect of this operation is merely to put a copy of the event
structure into an internal queue; actual dispatch is done in a separate thread. The imple-
mentation of the EventHandler interface must provide an implementation for the
callback operation, too (in order to implement the Callback interface). The effect
of callback is identical to dispatchEvent in this case. The two operations can be
considered as simple synonyms.
The details of event propagation are at the heart of the object’s semantics. Here is
what the object has to do when a new event instance is received:
1. The type of the new event is compared to all registrations. Only those are retained
where the new event’s type matches with the registered event type.
2. For all retained registrations:
2.1. If either the event data or the registered constraint array is empty, the event is
forwarded. This is done by invoking the callback operation on the registered
object with the event instance as an argument.
2.2. If the arrays are not empty, all key–value pairs with identical keys, in the con-
straint and the event data arrays, respectively, are compared. Comparison
means using the operation defined in the ConstraintOp field of the constraint
(which tells whether the comparison means ‘equal’, ‘greater than’, ‘contains’,
etc., see page 67). The left operand of the comparison operator is the value
stored in the event handler, i.e., the registered value, and the right operand is the
value in the new event instance.1)
2.3. If the registered value for the AndOr enumeration is And, the logical AND of all
comparisons is considered. Otherwise, the OR is considered. This leads to the
result of the full constraint checking: If true, the event is forwarded (like in 2.1
above). If false, this registration is not considered for event dispatch for this
event instance.
Let us see some examples. In a very simple case, registration with no constraints is used:
long id = eHandler.register(“PREMOEvent”,null,null,this);
1) If the operation does not make sense, e.g., a ‘contains’ is required for two numerical types, the result is
false.

82The finite machines appearing in an interactive setting have some extra require-
ments. They must be:
• programmable, i.e., the end–user should have means to attach his/her own methods
to each state transition of the object.
• monitorable, i.e., it should be possible to monitor state transitions from the outside
easily, typically through some event propagation mechanism.
The Controller object type of PREMO provides these facilities. The programmabil-
ity of the Controller objects is ensured by the specification of a number of protected
methods as part of the object. The client of a Controller object does not (and should
not) access these methods directly, because their invocation sequence is closely related
to the state transition of the object. That is why they are protected. On the other hand,
by defining a subtype for a Controller object, one can create specialized finite state
machines where the required behaviour is coded into these protected methods.
The ability to monitor state transition is accomplished by dynamically attaching call-
back routines to state transitions. This means that if a state transition occurs, these call-
backs are notified. For example, event handlers can be attached as callbacks to specific
state transitions and, through these event handlers, any object which intends to monitor
the state transitions can express its interest. Furthermore, Controller objects are de-
fined as subtypes of Callback, too, which means that Controller objects can be
chained together to form very complex interaction patterns.
5.4.2.1 Detailed Specification of a Controller
States in Controller are represented by strings. The allowable states, as well as the
current state of the object, can be retrieved by the operations:
premo.std.part2
premo.impl.part2
Figure 5-6 — PREMO types related to the Controller object
premo.std.utils
PREMOEnumeration
ActionType
SimplePREMOObject Callback EnhancedPREMOObject
ActionElement Controller
EnhancedPREMOObject_Impl
Controller_Impl

87
5.4.3 Time Objects5.4.3.1 General Notions
Time is an essential notion in multimedia systems, and also one of the most difficult to
grasp, primarily if full generality and distribution is to be taken into account. Although
there are systems, special hardware equipment, etc., which are capable of providing pre-
cise control over time, these are rarely available in the type of computing environments
in which PREMO is supposed to run. As a consequence, PREMO cannot include object
specifications which rely on real–time control. In most of the cases implementation of
such facilities would be impossible.
The pragmatic approach adopted by PREMO is to define a simple interface to time
control, without requiring a certain accuracy. Instead, the local accuracy value can be
inquired, and it is expected that the client would adapt its behaviour if the accuracy is,
for example, reduced.
PREMO time objects may return elapsed time in different time units, ranging from
picoseconds to years. Which unit to use is set by the client (a separate TimeUnit enu-
meration type is defined for that purpose). The accuracy of the time objects will, of
course, depend on the current setting of the unit. Time objects may easily commit
rounding errors and be off, for example, by one year if the unit is set to years. (For ex-
ample, because our implementation relies on long integer values to measure time, it is
not possible to express the value of 2.5 years!). On another extreme, it is very rare to
have a computing environment which can provide an accurate measurement on the pi-
cosecond level. The typical case is that the various time objects would be reasonably
accurate on the millisecond level, which is enough for multimedia presentation purpos-
es. The current accuracy can be inquired by the client, and the unit used for the accuracy
value can also be chosen. It is possible to measure the returned time value in years, but
expect the corresponding accuracy value in milliseconds.

88On the abstract level, represented by the abstract type called Clock, elapsed time is
measured as the returned value of an operation called inquireTick. Various subtypes
of the Clock object attach a more detailed semantics to what the ticks really mean. This
value and the accuracy obey the following relation. Suppose that the output of in-
quireTick is T, and the value of the accuracy is A (both values are long integers in our
case). If the moment used by inquireTick as a starting point in time is E then, math-
ematically, the real actual time , when inquireTick is called, follows the relation:
This also means that an accuracy of value zero represents the most accurate timing pos-
sible, and increasing values represent a loss in precision.1)
5.4.3.2 Specification of the PREMO Time Objects
The more formal specification of the time objects rely on the enumeration TimeUnit,
which is defined as follows:
1) To be precise, this relation is valid if accuracy and time are measured using the same units. If this is not the
case, A should be replaced by a function f(A), which converts the accuracy value from its own units to the
units used by T.
Figure 5-8 — Time objects
premo.std.part2
premo.impl.part2
Clock
Timer SysClock
Clock_Impl
Timer_Impl SysClock_Impl
Tr
E T A2--–+ Tr E T
A
2--+ +≤ ≤

90public interface Timer extends Clock,java.rmi.Remote
{
int getTimerCurrentState();
void start();
void stop();
void pause();
void resume();
void reset();
}
which include the obvious state transition operations and the explicit reset. Figure 5-
9 shows the meaningful state transition operations. All other state transition calls (e.g.,
calling pause when the object is in the TSTOPPED state) are ignored.
5.5 Synchronization Facilities
One generally accepted and important characterization of multimedia systems is that
they manage continuous media data. “This term refers to the temporal dimension of me-
dia, such as digital video and audio in that at the lowest level, the data are a sequence
of samples — each with a time position. The timing constraints are enforced during
playback or capture when the data are being viewed by humans.”[60] In some cases,
such as animation and synthetic 3D sound, the samples may result from (sometimes
complex) internal calculations (synthesis) whereas, in other cases, the samples are
available through some data capture process.
Maintaining the presentation of a continuous media data stream at a sufficient rate
and quality for human perception represents a significant challenge for multimedia sys-
tems, and may impose significant resource requirements on the multimedia computing
environment. Aside from this inherent constraint (sometimes referred to as the problem
of intra–media synchronization) a further difficulty arises from the fact that multimedia
applications often wish to use several instances of continuous media data at the same
time, such as an animation sequence with some accompanying sound or a video se-
quence with textual annotations. The difficulty here is that not only should the individ-
start
TSTOPPED
TSTARTED
TPAUSED
pause
stop
resume
stop
Figure 5-9 — State transitions in a Timer object

91
ual media data be presented with an acceptable quality, but well–defined portions of the
various media content should appear, at least from a perceptual point of view, simulta-
neously; some parts of a sound track belong to a specific animation sequence, subtitles
should appear with specified frames in a video sequence, etc. This problem is usually
referred to as inter–media synchronization. The specific problems raised by intra–me-
dia synchronization is not addressed by PREMO, because this is media specific and
falls outside the charter of a general reference model. In what follows, the term synchro-
nization is always used to refer to inter–media synchronization.
Synchronization has received significant attention in the multimedia literature, see,
for example, the book by Gibbs and Tsichritzis[34] or the survey paper Blakowki and
Steinmetz[10] for further information and references on the topic. An efficient imple-
mentation of inter–media synchronization represents a major load on a multimedia sys-
tem, and it is one of the major challenges in the field. What emerges from the experience
of recent years is that, as is very often the case, one cannot pin down one specific place
among all the computing layers (from hardware to the application) where the synchro-
nization problem should be solved. Instead, the requirements of synchronization should
be considered across all layers, i.e., in network technology, operating systems, software
architectures, programming languages, etc. and user interfaces.
The synchronization facilities of PREMO concentrate on one aspect of a complete
solution, namely, on a conceptual model and software architecture aimed at inter–media
synchronization. It provides general facilities which can be used to implement various
synchronization specifications which use interval–based, axes–based, or other declara-
tive methods (see again, e.g., the survey paper in [10] for further details and references).
In line with the middleware nature of PREMO, the goal is to provide a general mecha-
nism upon which these various declarations can be implemented, instead of dictating
one specific approach to be used for synchronization.
The PREMO synchronization model is based on the fact that objects in PREMO are
active. Different continuous media (e.g., a video sequence and corresponding sound
track) are modelled as concurrent activities that may have to reach specific milestones
at distinct and possibly user definable synchronization points. This is the event–based
synchronization approach, which forms the basic layer of synchronization in PREMO.
Although a large number of synchronization tasks are, in practice, related to synchroni-
zation in time, the choice of an essentially “timeless” synchronization scheme as a basis
offers greater flexibility. While time–related synchronization schemes can be built on
top of an event–based synchronization model, it is sometimes necessary to support
purely event–based synchronization to achieve special effects required by some appli-
cation (see, for example, the application described on page 98).
In line with the object–oriented approach of PREMO, the synchronization model
uses abstract object types that capture the essential features of synchronization. Some
of them have already been presented in earlier chapters, whereas some are more specif-
ically tailored at synchronization. These are:
• synchronizable objects, and their various subtypes (see Figure 5-10), to be presented
in more detailed in this section.
• synchronization points, (and event handlers in general), see section 5.4.1.3.