Papers CHI 2 0 0 0 ��� 1 - 6 APRIL 2 0 0 0 Programming and Enjoying Music with Your Eyes Closed Steffen Pauws, Don Bouwhuis IPO, C e n t e r for U s e r - S y s t e m Interaction D e n D o l e c h 2 5 6 0 0 M B E i n d h o v e n , the N e t h e r l a n d s +31 4 0 2 4 7 5 2 5 0 { S . C . P a u w s , D . G . B o u w h u i s } @ tue.nl Berry Eggen Philips R e s e a r c h L a b o r a t o r i e s E i n d h o v e n Prof. H o l s t l a a n 4 5 6 5 6 A A E i n d h o v e n , the N e t h e r l a n d s +31 4 0 2 7 4 5 1 6 0 b e r r y . e g g e n @ p h i l i p s . c o m ABSTRACT Design and user evaluation of a multimodal interaction style for music programming is described. User requirements were instant usability and optional use of a visual display. The interaction style consists of a visual roller metaphor. User control of the rollers proceeds by manipulating a force feedback trackball. Tactual and auditory cues strengthen the roller impression and support use without a visual display. The evaluation investigated task p e r f o r m a n c e and procedural learning when performing music programming tasks with and without a visual display. No procedural instructions were provided. Tasks could be completed successfully with and without a visual display, though programming without a display needed more time to complete. Prior experience with a visual display did not improve performance without a visual display. When working without a display, procedures have to be acquired and remembered explicitly, as more procedures were remembered after working without a visual display. It is demonstrated that multimodality provides new ways to interact with music. Keywords multimodal interaction, nonvisual interaction, interface design, user evaluation, interactive music system INTRODUCTION Considering the wide assortment of music available, instant access to a large music collection and, particularly, the task of music programming becomes increasingly important. Music listeners are already tempted to organize their music electronically and prefer extended play facilities without the need to handle physical storage media. For instance, music encoded in computer files, jukeboxes and portable players are becoming increasingly popular. However, it takes time to explore a music collection and select the favourite recordings with only sequential access to the music. In Permission to make digital or hard copies of all or par~ of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on seiwers or to redistribute to lists, requires prior specific permission and/or a fee. CHI '2000 The Hague, Amsterdam Copyright ACM 20001-58113-216-6/00/04...$5.00 addition, current jukebox players intended for home use are inconvenient to operate [1,2]. They generally contain numerous control elements. They often have inadequate visual displays that lack relevant information required for music programming, or that are poorly legible in dimly lit situations or from a large viewing distance. Multimodality may enhance interaction with music. This paper describes the design, implementation and a user evaluation of a multimodal interaction style for music programming. Music programming is defined here as the serial selection of multiple music recordings from a music collection. The evaluation was focused on assessing whether or not the user requirements were met by the interaction style. REQUIREMENTS The most important user requirements for the multimodal interaction style are instant usability and optional use of a visual display. Instant usability Usually, first-time users of a home device attempt to operate the device immediately without the aid of procedural instructions. A user manual is sometimes never consulted or is simply lost. Since users of home devices have no opportunities for training or are not willing to take these opportunities, learnability is considered a fundamental usability criterion for home devices [3]. Therefore, an interaction style for a home device should be as transparent, intuitive or self-explanatory as possible meaning that users are able to perceive at a glance the most effective and efficient ways to use the interaction style without procedural instructions. In other words, rather than leamability, a home device should allow instant usability. Some methods to design for instant usability are consistency of operation, minimality of features, the use of a small number of control elements and the use of a conceptual metaphor for interaction. Consistency of operation (or similarity of protocols [3]) means that the same pattern of actions can be used in different situations, allowing users to learn such a pattern only once. Minimality of features means that infrequently used or more complex functions are implemented in such a way that they do not interfere with initial learning, A small number of control elements allows users to learn the meaning of only a small 376 ~k,~l~ ~I=IX ~ O O O CHI Letters volume 2 ��� issue 1

CHI 2 0 0 0 , 1 - 6 APRIL 2 0 0 0 Papers Figure 1: Visual miler metaphor of the interaction style. The music programme contains two recordings. The currently selected music style is 'postbop' and a recording of Miles Davis is playing. Three visible recommendations are linked to this recording. set of controls or actions, for instance, the meaning of a few buttons. The use of a conceptual metaphor may form a starting point to understand an interaction style. It may aid learning domain knowledge [4], may explain the expression and meaning of actions [5] or may facilitate the learning of procedures [6,7]. Optional use of a visual display Current CD j u k e b o x players have an i n a d e q u a t e presentation of visual information [1,2]. Some visual displays lack the presentation of relevant information required to operate the jukebox effectively. They are often too small or do not have enough contrast to be legible in dim light or from a large viewing distance. Portable players even lack a convenient visual display. The need for visual inspection of information can also be less desirable, for instance, when relaxing while going through a music collection. It is for these reasons that the use of a visual display should be optional in a music programming task, without sacrificing instant usability. DESIGN The interaction style comprises a manual input modality and various output modalities: the visual, tactual and auditory modalities. User control proceeds entirely by manipulating a force feedback trackball. The auditory modality consists of three different audio streams: synthetic speech, non-speech audio and music audio. Visual roller metaphor The concept of a music programming for the purpose of selecting favourite music from a collection is generally self- evident, irrespective of whether it is a request or carried out at will. However, first-time users of a home device often find difficulties to transfer declarative domain knowledge and a task purpose into procedural knowledge on how to achieve the desired result, though they generally like to know how- to-use-a-device first. Without procedural instructions, the initial problem that first-time users have to overcome is to discover what exactly constitutes an action, what consequences can be expected from an action, and whether or not these consequences are effective with respect to their purpose [8]. In order to partly solve this initial problem for users, a metaphor is used as a conceptual interaction model. During an iterative design process, several metaphors starting from a spherical object were considered as an conceptual interaction model. The use of a spherical object was prompted by the use of a trackball as input device (see Figure 3). While adding actions that are essential to navigation and selection, the model was assessed on appropriateness. The main criteria for assessment were consistency of operation, a minimum number of actions required and compliance with the envisaged metaphor. Secondary criteria were implementation feasibility and predicted computational resources required. As shown in Figure 1, the final interaction model resembles a fruit machine consisting of four rollers on which title and artist of music recordings are projected. The left-hand roller represents a music programme that the user creates. A counter is positioned over this roller and displays the number of recordings added to the programme. The next roller represents the music styles in the collection. Music styles are arranged in a chronological order, that is, in the order in which the music styles (in this case, jazz styles) have emerged in time. Next, the music collection roller displays all available recordings in the collection or just the subset of recordings that belongs to a particular music style. Figure 1 shows that this roller has input focus and is therefore high-lighted. Recordings on this roller are first alphabetically ordered on artist, then grouped by album and placed in the order in which they came out on the album. Finally, albums are chronologically ordered on year of publication. The numbers displayed on this roller are intended to indicate the number of recordings available in a music style. The right-hand roller contains a list of music recommendations that corresponds to the music recording that is at the front of the music collection roller. Music recommendations are intended to ease and speed up the music selection process. ~--k.~lBI 377