Augmented Reality for Construction Tasks: Doorlock Assembly∗
Dirk Reiners, Didier Stricker, Gudrun Klinker, Stefan Mu¨ller
Fraunhofer IGD
Department Visualization and Virtual Reality
Rundeturmstraße 6
64283 Darmstadt Germany
{reiners,stricker,klinker,stefanm}@igd.fhg.de http://www.igd.fhg.de/www/igd-a4/
Abstract
Augmented Reality is a technology that integrates
pictures of virtual objects into images of the real
world. Besides the technical problems still to be
solved, for industry to be interested in this tech-
nology the additional amount of work needed to
use the technology in relation to the benefit has
to be visible. Furthermore the question how this
integrates into the information technology infras-
tructure of the company is important. This paper
describes an Augmented Reality demonstrator for
the task of doorlock assembly into a car door that
was developed trying to create a practical, realis-
tic application that can transport the concepts be-
hind Augmented Reality to a casual observer. To
reach that goal a new fast and robust optical track-
ing algorithm was developed and integrated into a
three-dimensional animation and rendering system,
creating a real-time fully three-dimensional HMD-
based training application showing how to assemble
the doorlock into the door. The system was demon-
strated to the general public at the Hannover In-
dustrial Fair 1998 and this demonstration of Aug-
mented Reality for one of the first times to a large
non-expert audience created a lot of interest into
this new area.
1 Introduction
Augmented Reality is a technology that integrates
pictures of virtual objects into images of the real
∗presented at: the 1rst International Workshop on Aug-
mented Reality (IWAR’98), San Francisco, Nov. 1998. AK
Peters, pp. 31-46.
world. These images can either be taken from a
camera or, with the use of a see-through head-
mounted display, the user’s direct view on the world
can be augmented. There are still some technolog-
ical problems to solve, but for industrial partners
to be interested in investing into this technology its
possible benefit and its integration into the whole
company has to be visible.
Many companies are trying to move bigger parts
of their design and construction processes into the
computer, up to the point where physical proto-
types are replaced by virtual prototypes for pack-
aging, assembly and security evaluations. This is
especially evident in the airplane and car construc-
tion companies, where prototypes are costly and
time to market is a significant factor. A side-effect
is that a large number of three-dimensional models
become available at little or no additional cost.
AR can help using this growing amount of digital
product information to help the people who have to
handle the fast changing products. A consequence
of reduced time-to-markets, shorter product cycles
and smaller production counts is the increasing de-
mand for training. Assembling a complex machine
like a car or a plane is a difficult process that takes
time to learn. If that training is only useful for a
small number of production runs, in the worst case
only once, the efficiency of the worker decreases.
The situation is even worse for service personnel, as
the faster product cycles don’t necessarily go along
with shorter life cycles. Thus a service person is
confronted with an ever faster growing number of
different generation systems and has to be able to
handle all of them. Given the far, possibly world-
wide, distribution of service personnel for example
1

for cars makes a centralized training difficult, ex-
pensive or impossible.
The availability of three-dimensional models al-
lows to do that in novel ways that are more di-
rect and more intuitive than written instructions
or even electronic hypertext manuals. Three-
dimensional animated instructions can be inte-
grated into the surrounding environment at the ex-
act place where the action has to be performed,
so that no mental transfer is needed and the ac-
tion can be observed from different viewpoints to
understand the spatial relationships involved.
This paper describes an implementation of an
augmented reality system that builts this connec-
tion between virtual prototyping and real assembly
tasks, in this case the task of assembling a doorlock
into a car door. After reviewing previous work in
using augmented reality for assembly and mainte-
nance work in sec. 2 we introduce the application
task in sec. 3. Following that is a description of the
used hardware (sec. 4.1) and software (sec. 4.4 and
4.3) systems together with motivations and alter-
natives. The system has been shown to the general
public at the Hannover Industrial Fair 1998 (sec.
5) and as a conclusion experiences and open areas
for further work are analyzed in sec. 6.
2 Previous Work
Assembly and maintenance tasks have been a tar-
get area of AR demonstrations for quite a long time.
One of the first demonstrations was using Aug-
mented Reality for a photocopier maintenance task
[3]. This was done using wireframe graphics and a
monochrome monoscopic HMD. Head- and object
tracking were achieved using ultrasonic trackers.
The graphics used were rather simple and not taken
from any CAD system as the main objective was
extending an existing two dimensional automated
instruction generation system to an augmented en-
vironment. The same scenario was revived recently
by [5], this time tracked optically.
Another application in the construction area was
[10], where augmented reality technology was used
for spaceframe assembly support. Tracking was
done using an active optical system using blink-
ing LEDs coupled to a passive inertial system. The
graphics used were again rather stylized, as the ob-
jective was not to convey a sense of spatial presence
but to bring rather simple instructions across.
Boeing has used an Augmented Reality system
for wire bundle assembly support. To the best of
our knowledge the tracking was done using an in-
ertial/ultrasonic system. The augmentations used
(2D lines showing the path of the cable to be added
to the bundle) were simple and could be generated
from a databse without manual involvement or a
CAD system.
3 Application
The application targeted in this demonstration is
the assembly of the doorlock into a car door (s.
fig. 12). This task was examined as an example
for virtual prototyping and an assembly feasibility
study by the virtual environments group of our de-
partment for BMW (s. fig. 1). Thus CAD data
taken from the system used to construct the actual
objects was available not only for the lock but also
for the whole door.
Figure 1: Virtual doorlock assembly
The task itself poses several challenges. It is very
spatial and three-dimensional in nature, the move-
ment of the hand holding the lock in the small space
inside the door necessitates precise preparation and
movement. A little lever that has to engage in the
lock deep inside the door, which is not easily visible
from the outside (s. fig. 10), has to be pushed to
the right position beforehand, otherwise the final
engaging of the lock will be impossible. Further-
more the space inside the door is just big enough
2