Mid-Air Text Input Techniques for Very Large Wall Displays

Garth Shoemaker, Leah Findlater, Jessica Q. Dawson, and Kellogg S. Booth
Department of Computer Science
University of British Columbia
2366 Main Mall, Vancouver, BC, Canada
{garths, lkf, ksbooth}@cs.ubc.ca
ABSTRACT
Traditional text input modalities, namely keyboards, are often not
appropriate for use when standing in front of very large wall
displays. Direct interaction techniques, such as handwriting, are
better, but are not well suited to situations where users are not in
close physical proximity to the display. We discuss the potential
of mid-air interaction techniques for text input on very large wall
displays, and introduce two factors, distance-dependence and
visibility-dependence, which are useful for segmenting the design
space of mid-air techniques. We then describe three techniques
that were designed with the goal of exploring the design space,
and present a comparative evaluation of those techniques.
Questions raised by the evaluation were investigated further in a
second evaluation focusing on distance-dependence. The two
factors of distance- and visibility-dependence can guide the design
of future text input techniques, and our results suggest that
distance-independent techniques may be best for use with very
large wall displays.

KEYWORDS: text input, wall displays, interaction techniques

INDEX TERMS: H.5.2 [User Interfaces]: Input devices and
strategies
1 INTRODUCTION
Designers of very large interactive display environments (Figure
1) have yet to establish a standard model for interaction.
Traditional devices (keyboard and mouse) and onscreen
metaphors (windows, icons, menus, and pointers) long established
for desktop use are often not ideal for the form factors of new wall
and table displays. An ongoing effort in the research community
has been to investigate different interaction possibilities, with the
goal of designing better methods for input and manipulation. The
development of these methods will play a critical role in the
evolution of large displays from research systems to universally
adopted tools.
In this paper we consider English language text input
techniques for very large wall displays. Text input is one of the
primitives of interaction identified by Foley et al. [6], and must be
supported by any general purpose interactive system, even if that
system is not used for lengthy uninterrupted text input.
Researchers recognize the limitations of keyboard input, the most
significant being that keyboards are designed for use while sitting
stationary at a desk or a table, but users of wall displays are often
standing or walking. Alternative approaches have been
investigated, including pen-based text input, where a user writes
as on a whiteboard. This method, while natural, has the significant
drawback that a user must be within physical reach of the display
surface in order to write. Users of large wall displays are often not
within physical reach of the display surface [21, 23].
To address this problem we identified and investigated the
under-explored design space of mid-air text input techniques,
those that can be used by a standing, mobile user. In this context,
techniques that allow for input independent of visibility or
distance to the display might be particularly important. Thus, we
segment this design space along the axes of visibility-dependence
and distance-dependence, both of which are useful for
categorizing candidate interaction techniques.
We developed three text input techniques that differ in terms of
distance- and visibility-dependence: Circle, QWERTY, and Cube.
For each technique the user manipulates a handheld device in
mid-air, at some distance from the display surface. We conducted
a controlled experiment to compare the techniques in terms of
speed and accuracy for a text input task. Results suggest that the
QWERTY technique is suitable for adoption, and that the other
two techniques have potential for future development but need
improvements to be competitive with the QWERTY technique.
A second, follow-up experiment was performed with the goal of
developing a deeper understanding of the distance-dependence
factor of interaction techniques. We evaluated Circle and
QWERTY techniques, which we hypothesized would be distance-
independent and –dependent, respectively, at two different
distances. Results showed that the factor of distance had a
significant impact on performance. This highlights the need for
further work on distance-independent techniques.
The primary contributions of this paper are three-fold. First, we
investigate the design space of mid-air text input techniques for
large wall displays, and identify important factors segmenting this
space. Second, we report a comparative analysis of three
candidate text input techniques representative of the design space,
and we provide conclusions regarding the usefulness of those
techniques. Third, we report a second experiment that highlights
the importance of distance-independence when designing
interaction techniques for large wall displays.

Figure 1. A mockup of collaboration around a large wall display.


231
Graphics Interface Conference 2009
25-27 May, Kelowna, British Columbia, Canada
Copyright held by authors. Permission granted to CHCCS/SCDHM
to publish in print form, and ACM to publish electronically.

2 RELATED WORK
Relevant related work falls into the categories of large display
interaction and text input. The large display work serves to define
the special requirements of these environments, while the
substantial body of text input research serves as a foundation for
text input considerations in general.
2.1 Large Display Interaction
Large physical surfaces can provide valuable inspiration in the
design of large electronic surfaces. Traditional whiteboards are
widespread in modern workplaces, and have been shown to be
important tools for collaborative information sharing and
visualization [4]. It has been argued that the 19
th
century
replacement of personal slates by large shared blackboards was an
important innovation in the domain of education [3]. This,
combined with quantitative evidence that large display surfaces
can enhance performance [5], makes a strong argument that large
interactive wall displays have a useful role to play.
Consistent with these conclusions, recent work on Shadow
Reaching [20], Soap [2], and freehand pointing [24] has begun
looking at mid-air techniques that allow for relatively
unconstrained movement on the part of the user. The Shadow
Reaching work demonstrates the use of full-body shadows for
interaction, whereas the findings of the work on Soap and
freehand pointing are about device-based and device-less
pointing, respectively.
2.2 Text Input
Text input techniques in a variety of domains can serve as
inspiration for the design of techniques for large wall displays.
2.2.1 Large Wall Display Text Input
Large wall displays are physically similar to whiteboards, so there
is an understandable tendency for designers to employ pen-based
handwriting for text input, as was done in Flatland [16] and Tivoli
[18]. Handwriting is not always the best choice for text input,
however. Writing input speed, known to be at best around 20
wpm [1], is worse than many mechanized techniques.
Furthermore, recognition algorithms for the purpose of digitizing
and archiving written data are error-prone. As a result of these
limitations, researchers have considered alternatives to
handwriting input. Pavlovych and Stuerzlinger evaluated text
entry performance using direct touch input with a variety of
keyboard layouts [17]. They found that a standard QWERTY
layout resulted in a mean text entry rate of 17.6 wpm, which is
roughly comparable to hand writing performance. Magerkurth and
Stenzel took a different approach, supporting text input from a
small personal input device, with feedback provided on a shared
display [14].
2.2.2 Small Display Text Input
Small handheld devices, such as phones and personal digital
assistants (and hybrids of the two) are increasingly being used for
text-heavy tasks, such as writing email and web browsing.
Techniques for small displays are relevant to large displays
because users of both systems are often standing and moving, and
successful approaches in one domain may apply to the other. But
it is difficult to incorporate a full QWERTY layout keyboard into
a small display. As an alternative to this layout, many small
display devices employ disambiguation techniques such as T9 or
multitap. T9 and multitap are widely deployed, but performance
by any except highly expert users is poor when compared to
standard keyboard input. Other disambiguation approaches have
been explored. TiltText, as an example, uses tilt information from
an accelerometer to filter how characters are selected by button
presses on the keypad [27].
More recently, stylus and touch-based text input approaches
have gained popularity on small mobile devices. In this context,
MacKenzie and Zhang showed that soft keyboards can provide
impressive performance with either standard QWERTY or other
optimized character layouts [13]. Similarly, a number of 2D
gesture techniques have been shown to hold promise [15, 22, 28].
2.2.3 Other Text Input Approaches
Many innovative methods for text input are not easily categorized
as being either small or large display techniques. Dasher, one such
technique, makes use of a continuous gesture through a 2D
landscape of characters generated by a predictive model [26].
Another gestural technique developed by Liu et al. [9] allows
users to input text by tracing letters in mid-air with their fingers.
There has also been work done on device-specific text input
techniques. Input using game controllers has been explored [7], as
have techniques that employ chording keyboards [8].
3 DESIGN SPACE
As we have already noted, many large display text input
techniques require the user to be within physical reach of the
display, but in many cases users of such displays are frequently at
a distance. In contrast, techniques that allow for free body
movement within the space near, but not at, the display while
interacting have received relatively little attention. These are
termed mid-air techniques. They can be used while standing or
walking, and at any practical distance from the display.
We have identified two properties related to mid-air techniques
that are important to consider. The first property is distance-
dependence. The physical action of a distance-dependent
technique changes as the distance between the user and the
display changes, whereas action of a distance-independent
technique is invariant with distance. As an example, pointing
using a ray-casting model is a distance-dependent technique: as
the user moves farther from the display, the effect of pointing
motions is magnified. We hypothesized that large displays will
benefit from the development of distance-independent techniques,
because these techniques will not constrain the movement of users
within space.
The second property is visibility-dependence. A visibility-
dependent technique requires that the user refer to visible
feedback during use, whereas a visibility-independent technique
does not. For example, touch-typing is visibility-independent, but

Figure 2. Text input interaction techniques as used in Experiment 1. From left to right: Circle Keyboard, QWERTY Keyboard, Cube Keyboard.
232