Minput: Enabling Interaction on Small Mobile Devices with
High-Precision, Low-Cost, Multipoint Optical Tracking

Chris Harrison Scott E. Hudson
Human-Computer Interaction Institute, Carnegie Mellon University
5000 Forbes Avenue, Pittsburgh, PA 15213
{chris.harrison, scott.hudson}@cs.cmu.edu
Figure 1. Our prototype Minput-augmented device running an audio player application. Two optical sensors on the back of the
device enable x/y translation and rotational tracking on ad hoc surfaces, such as tables, walls, clothes, and the palm of one’s hand.
ABSTRACT
We present Minput, a sensing and input method that en-
ables intuitive and accurate interaction on very small de-
vices – ones too small for practical touch screen use and
with limited space to accommodate physical buttons. We
achieve this by incorporating two, inexpensive and high-
precision optical sensors (like those found in optical mice)
into the underside of the device. This allows the entire de-
vice to be used as an input mechanism, instead of the
screen, avoiding occlusion by fingers. In addition to x/y
translation, our system also captures twisting motion, ena-
bling many interesting interaction opportunities typically
found in larger and far more complex systems.
ACM Classification: H.5.2 [Information interfaces and
presentation]: User Interfaces - Input Devices and Strate-
gies, Interaction Styles, Graphical User Interfaces.
General terms: Human Factors
Keywords: Mobile devices, touch screens, optical tracking,
pointing, input, sensors, spatially aware displays, gestures.
INTRODUCTION
Small mobile devices offer the promise of significant com-
putational power that can be carried with us into variety of
circumstances and environments. As advances in electron-
ics allow devices to become smaller and smaller, we begin
to reach limits not of the electronics, but in the area needed
to provide a usable human interface. Buttons, for example
either begin to consume a significant fraction of available
surface area, or become too small for comfortable and ef-
fective use. Techniques such as touch screens become less
effective, especially for fine-granularity operations, when
the size of a human finger begins to take up a significant
fraction of the entire display.
Considerable work has attempted to address issues in this
area. Although many approaches can operate in small to
medium form factors, each suffers from at least some
drawbacks. Vision-based approaches are perhaps the most
compelling (see e.g., [9,15,16]). These, however, require
the integration of a camera into, e.g., the backside, forcing
the user to grasp the (small) device in a very particular
fashion. Furthermore, vision processing is computationally
expensive and error prone in dynamic contexts such as
walking or waiting for the bus (with high levels of non-
input related optical flow).
Acoustics offer another approach [7], although such meth-
ods will always have to contend with environmental noise
and face privacy and social issues in shared settings. Accel-
erometers also suffer from high false positives in dynamic
contexts, such as walking and riding on public transporta-
tion. Moreover, they tend to offer a lower level of fine con-
trol and expressivity – barring them from applications re-
quiring varied and high accuracy interactions.
Several point solutions have attempted to address this prob-
lem. Of note is SideSight [2], which uses infrared proxim-
ity detection around the device periphery to perform multi-
touch tracking. It is unclear, however, how well irregular
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee.
CHI 2010, April 10–15, 2010, Atlanta, Georgia, USA.
Copyright 2010 ACM 978-1-60558-929-9/10/04....$10.00.
CHI 2010: Devising Input April 10–15, 2010, Atlanta, GA, USA
1661

surfaces (e.g., the palm or clothing) affect tracking due to
line-of-sight issues. NanoTouch [1] cleverly incorporates a
touch-sensitive surface on the underside of a device, allow-
ing for direct finger manipulation without screen occlusion.
On a very small device, the focus of our efforts, it is not
clear if one can even comfortably place two fingers on the
underside for accurate multitouch gestures, such as pinch-
ing. More important, however, is that both systems essen-
tially provide a 1:1 control-device (C-D) gain, tightly cou-
pling the resolution of input to the size of the device.
MINPUT
The mass production of optical mice has made the highly
sophisticated sensors on which they rely very inexpensive.
Additionally, advances in electronics and optics have
yielded sensors that are both small and extremely precise.
A generic optical mouse, costing only a few dollars, is ca-
pable of capturing and comparing surface images several
thousand times per second. Often, this high resolution en-
ables their use on a variety of surfaces - both traditional
(e.g., mouse pads, tables) and ad hoc (e.g., palms, pants,
bed covers, papers) (Figure 1). Vision-based interaction
techniques (e.g., [9,15,16]) tend to heavily tax even modern
mobile device processors and batteries. Fortunately, the
optical sensors we employ have dedicated, highly efficient
processors that handle most of the computation with negli-
gible power consumption.
The central idea behind Minput is simple: place two optical
tracking sensors on the back of a very small device. This
allows the whole device to be manipulated for input, ena-
bling many interesting interactions with excellent physical
affordances [6]. This could allow for the creation of e.g., a
matchbook-sized media player that is essentially all screen
on its front side. The device could be operated using any
convenient surface, e.g., a table or palm. The use of two
tracking elements enables not only conventional x/y track-
ing, but also rotation [8], providing a more expressive de-
sign space. The latter motion is calculated by taking the
difference in velocities of the two sensors.
This configuration allows Minput to operate like a spatially
aware display [5]. Previous systems, however, have tended
to be large and complex. For example, [4] and [13] were
tethered to high-cost and stationary tracking systems, while
[11] used an equally immobile augmented table and vision
system. Minput provides much of the same capability, but
in an inexpensive, low-power, compact and mobile form.
PROTOTYPE
To investigate the usability and accuracy of our input ap-
proach, we constructed a small prototype device (Figure 2).
For a display, we used a NHJ VTV-101 TV wristwatch
modified to receive video from a conventional desktop
computer (where interface control and rendering for our
proof-of-concept device took place). The device features a
1.5” TFT LCD (30x23mm) with a resolution of 280x220.
On the underside of the device, we mounted optical sensors
extracted from two SlimG4 mice. The sensor and optics
package is a diminutive 9x17x3mm, allowing it to be read-
ily integrated into mobile device hardware. At the heart of
the sensor is an ATA2198 processor, manufactured by At-
Lab (http://atlab.co.kr), which samples at 3.4kHz (800
CPI). Translation data from the two sensors is transmitted
over USB to the aforementioned PC.
INPUT MODALITIES
Minput is an enabling technique on top of which numerous,
distinct input modalities can be built. To illustrate this, we
highlight three interaction techniques we believe to be of
particular utility: gestures, virtual windows, and cursor con-
trol. We also introduce a new interaction: twisting for
zooming and selection.
Gestures
The high precision motion captured by our approach makes
gestures a strong candidate for input. As a proof of concept,
we developed software that detected two basic forms: flick-
ing and twisting (Figure 3). To flick, users simply rapidly
swipe the device in a particular direction. We primarily
used up, down, left and right, but diagonals and other an-
gles are possible. Twisting is achieved by rotating the de-
vice around its center point. This feels much like twisting a
physical knob, and offers many of the same affordances.
More complex gestures are, as they are with mice or styli,
eminently performable.
In piloting, we observed two distinct ways people perform
such gestures. Some users held the device above the sur-
face. When a gesture was to be performed, it was only then
that the device made contact with the surface (for a mo-
ment). It then returned to a central, hovering position. Con-
versely, some users tended to prefer resting the device on
the surface. This allowed gestures to be performed immedi-
ately. However, after the gesture was complete, users lifted
the device, re-centered it, and placed it back on the surface.
The latter is similar to clutching in mice (e.g., when the
edge of the mouse pad is reached). In both methods, contact
with the surface acts as a clutch for input.

Figure 2. Two optical sensors operate on the underside
of our proof-of-concept Minput device.


Figure 3. Possible gestures include omni-directional flicking,
twisting clockwise and counter clockwise, and motion paths.

CHI 2010: Devising Input April 10–15, 2010, Atlanta, GA, USA
1662