16
Face-to-Face Communication
Nadia Magnenat Thalmann
Prem Kalra*
Marc Escher
MIRALab, CUI
University of Geneva
Airplane and car manufacturers created the first computerized human models
20 years ago. The main idea was to simulate a very simple articulated structure
for studying problems of ergonomics.
In the 1970s, researchers developed methods to animate human skeletons,
mainly based on interpolation techniques. Bodies were represented by very prim-
itive surfaces like cylinders, ellipsoids, or spheres. At the same time, the first
experimental facial animation sequences appeared (Parke, 1974).
The Juggler, from Information International, Inc. (1982), was the first realis-
tic human character in computer animation. The results were very impressive;
however, the human shape was completely digitized, the body motion had been
recorded using 3-D rotoscopy, and there was no facial animation. The first 3-D
procedural model of human animation was used in producing the 12-minute film
Dreamflight (Magnenat-Thalmann, Bergeron, & Thalmann, 1982), one of the first
to feature a 3-D virtual human. Simultaneously, an effort with more emphasis on
functional aspects than realism for virtual humans was initiated at the University
of Pennsylvania (Badler & Morris, 1982) through a software package called Jack.
*Currently at the Indian Institute of Technology, New Delhi, India
345
Copyright © 2003 by Lawrence Erlbaum Associates, Inc.

346 MAGNENAT-THALMANN, KALRA, AND ESCHER
In the 1980s, researchers started to base animation on the key frame animation
and parametric animation, and in the late 1980s on the laws of physics. Dynamic
simulation made it possible to generate complex motions with a great deal of
realism. However, an ordinary human activity like walking is too complex to
be simulated by the laws of dynamics alone. Two people, with the same physical
characteristics, do not move in the same way. Even one individual does not move in
the same way all the time. A behavioral approach to human animation is necessary
to lend credibility to such simulations.
The face is a relatively small part of a virtual human, but it plays an essential role
in communication. We look at faces for clues to emotions or even to read lips. It is
a particular challenge to simulate these aspects. Therefore, the ultimate objective
is to model human facial anatomy exactly, including its movements, with respect
both to structural and functional aspects. Recent developments in facial animation
include physically based approximation to facial tissue and the reconstruction of
muscle contractions from video sequences of human facial expressions. Problems
of correlation between emotions and voice intonation have been also studied.
Ensuring synchronization of eye motion, facial expression of emotion and the
word flow of a sentence, as well as synchronization among several virtual humans,
is at the heart of our new facial animation system at the University of Geneva.
In the context of interactive virtual environments and animation systems, the
relationship between the user as animator and virtual human as synthetic actor
needs to be emphasized. A two-way communication is required where not only
can the animator give commands to the actor but the actor must also be able to
respond both verbally and behaviorally.
This chapter is an account of a face to virtual face interaction system where a
clone, representing a real person, can communicate with another virtual human,
who is autonomous, in a virtual world. The dialogue consists of both verbal and
other expressive aspects of facial communication between the two participants.
Gives an overview of the problem and describes major contributions related to the
different aspects. Concentrates on our system and describes different components
of the system. Presents issues related to the standardization of parameters for
defining the shape and animation of the face. Future trends are outlined in the
concluding remarks.
PROBLEM DOMAIN AND RELATED WORK
To clone is to copy. In our context, cloning means reproducing a virtual model of a
real person in the virtual world. Here, our interest is restricted to one component of
the human figure—the face. The face is the most communicative part of the human
figure. Even a passive face conveys a large amount of information, and when it
comes to life and begins to move, the range of motions it offers is remarkable: We
observe the lips, teeth, and tongue for speech, eyes and head movements for addi-
tional elements of dialogue, and flexing muscles and wrinkle lines for emotions.
Copyright © 2003 by Lawrence Erlbaum Associates, Inc.