Visualization and direct manipulation in user interfaces: Are we overdoing it?

Abstract

The current tendency on the part of software designers to place more and more information onto the computer screen is resulting in an increased burden on end-users' physiological capabilities and cognitive resources. Users must constantly survey a large amount of data, resulting in a considerable strain on their eyes. Furthermore, the work on their primary task is often seriously disrupted when direct manipulation techniques are used to issue commands to the application program. In this paper we propose the use of bimodal input - speech coupled with direct manipulation - as a means of allowing users to take fuller advantage of the visualization of information. We describe an empirical study with 72 senior high school students and manufacturing apprentices working with two prototypical interfaces for a program to produce technical diagrams. One interface had the usual unimodal interaction (i.e., direct manipulation only), while the second allowed bimodal interaction. An improvement of over 15% in the subjects’ working-speed was attained with the bimodal interface. Further, no significant increase in their cognitive workload was observed. Our results suggest that speech may be a viable complement to direct manipulation techniques in user interfaces as a way of supporting complex visualizations.