A New Temporal Domain Optical Flow Measurement Technique for Focal Plane VLSI
Implementation
R. Etienne-Cummings, S . Fernando, N. Takahashi, V. Shtonov, J. Van der Spiegel
University of Pennsylvania, Department of Electrical Engineering, Center for Sensor Technologies, Philadelphia, Pa
19104-6390
P. Mueller
Corticon Inc., Philadelphia, Pa 191 04
Abstract
A new temporal domain technique for optical flow
measurement is presented. This approach, which has been
developed primarily for VLSI implementation, requires
only the sign of spatiotemporal derivatives, 1-bit Boolean
multiplication and integer arithmetic to compute image
velocity. Hence, it can be easily and efficiently
implemented in hardware and software. It is composed of
a hybrid of the Reichardt and Ullman-Marr motion models
and measures both speed and direction at every pixel. For
image sequences, it measures the number of frames
required for an edge to translate over a pixel, while in
analog hardware, it measures the time. Direction is given
by correlating the disappearance of an edge at one pixel
with its reappearance at a neighboring pixel. A test chip
of this scheme has been implemented in 2 pm VLSI and
was found to measure 2D velocity over 3 orders of
magnitude. A total of 0.1 mW of power was consumed
per pixel in room light. Therefore, this technique offers
simple, computationally efficient, and direct means for
measuring wide spatiotemporal band width image motion
in both hardware and software.
Introduction
Visual motion detection is one of the fundamental
computations required for visual perception1. From
visual motion, many important cues are obtained to
facilitate high level perception of 3D shape, scene depth,
egomotion, and for navigation and avoidance, just to name
a few. Hence, computer visionist have developed many
techniques for measuring visual m ~ t i o n ~ - ~ . These
techniques typically require large numbers of accurate
spatiotemporal derivatives, multiplications and divisions.
As a result, they have been implemented mostly in
software since these computations are difficult to
implement in digital or analog hardware6g7. However,
motion computation is usually a prerequisite for higher
level processing. Hence, large amounts of computation
resources are dedicated to motion detection causing the
intended higher level processing to become slow. Using
special parallel hardware improves this problem, but is
usually expensiveg. Therefore, using optical flow
detection techniques which do not require accurate
spatiotemporal derivatives, multiplications and divisions
can reduce the amount of computation required for visual
motion measurement, and can be easily mapped onto
VLSI hardware. Furthermore, the constraints of VLSI
requires algorithms which are compact and
computationally efficient. These algorithms lend
themselves nicely to software implementation too. We
present such a technique which requires only the sign of
spatiotemporal derivatives, and 1-bit Boolean
multiplications to compute image velocity. This
technique, which has been implemented in software and
VLSI hardware, measures velocity in the temporal
domain.
Algorithm
This optical flow technique is derived from a hybrid of
the Ullman-Marr and Reichardt methods for computing
visual motion9-11. It uses the simplicity of the Reichardt
correlation of neighboring pixels for sensing direction, and
Ullman-Marr temporal derivative of zero-crossing for
determining speed. By combining the two models, the
advantages of both models are coupled, while their
disadvantages are eliminated. Figure 1 shows the
computational flow of the algorithm.
One of the drawbacks of the Reichardt model is its
limited spatiotemporal frequency response. It correlates
delayed versions of pixels with the outputs of their
neighbors. Since the delay element is implemented as a
first order lowpass filter, the model is very narrowly
temporally band limited. Furthermore, the spatial filters
are also narrowly tuned at each desired scale. Hence, only
motion of edges with specific spatial and temporal
frequency components can be detected with the model.
For optical flow measurement with this technique,
multiple versions of the model tuned to all the expected
velocities and spatiotemporal frequencies of the image
must be used at every pixel. On other hand, special
combinations of different versions of the model which
implicitly gives the output for a wider range of image
motionlspatiotemporal frequency pairs can be used.
However, in general, this model is very cumbersome to
implement in both hardware and software. This is also
true for other models which depend on spatiotemporal
filters such as the energy model based techniques23I3.
On the other hand, the Ullman-Marr model uses first
order differentials to detect both speed and direction.
Hence, it is inherently a wide band process. However, the
model uses complex combination of spatial and temporal
derivatives to detect direction, which makes its
implementation impractical.
241
0-8186-5420-L/93 $03.00 0 1993 IEEE