Computer Supported Cooperative Work 11: 153���180, 2002. �� 2002 Kluwer Academic Publishers. Printed in the Netherlands. 153 Physical and Virtual Tools: Activity Theory Applied to the Design of Groupware MORTEN FJELD1,���, KRISTINA LAUCHE2, MARTIN BICHSEL3, FRED VOORHORST1, HELMUT KRUEGER1 & MATTHIAS RAUTERBERG4 1Institute for Hygiene and Applied Physiology (IHA), Swiss Federal Institute of Technology (ETH), CH-8092 Zurich, Switzerland 2Institute for Work and Organisational Psychology (IfAP), Swiss Federal Institute of Technology (ETH), CH-8092 Zurich, Switzerland 3Institute of Mechanical Systems (IMES), Swiss Federal Institute of Technology (ETH), CH-8092 Zurich, Switzerland 4Center for User-System Interaction (IPO), Technical University Eindhoven (TUE), NL-5612 AZ Eindhoven, The Netherlands (���Corresponding author E-mail: morten@fjeld.ch) Abstract. Activity theory is based on the concept of tools mediating between subjects and objects. In this theory, an individual���s creative interaction with his or her surroundings can result in the production of tools. When an individual���s mental processes are exteriorized in the form of tools ��� termed objectification ��� they become more accessible to other people and are therefore useful for social interaction. This paper shows how our understanding of activity theory has shaped our design philosophy for groupware and how we have applied it. Our design philosophy and practice is exemplified by a description of the BUILD-IT system. This is an Augmented Reality system we developed to enhance group work it is a kind of graspable groupware which supports cooperative planning. The system allows a group of people, co-located around a table, to interact, by means of physical bricks, with models in a virtual three-dimensional (3D) setting. Guided by task analysis, a set of specific tools for different 3D planning and configuration tasks was implemented as part of this system. We investigate both physical and virtual tools. These tools allow users to adjust model height, viewpoint, and scale of the virtual setting. Finally, our design practice is summarized in a set of design guidelines. Based on these guidelines, we reflect on our own design practice and the usefulness of activity theory for design. Key words: activity theory, Augmented Reality, computer, configuration, co-located interaction, cooperation, design, graspable, groupware, objectification, physical tools, planning, social, Virtual Reality, virtual tools 1. Introduction The aim of this paper is to explain and illustrate our design philosophy for devel- oping graspable groupware. Our philosophy is based mainly on the concepts of tools and exteriorization found in Leont���ev���s (1978, 1981) and Engestr��m���s (1990, 1996) work on activity theory. Our practice rests on B��dker���s (1991) and Kaptelinin���s (1996) work, where they apply activity theory to human-computer interaction. In our construction of a system we employed a recent technology called Augmented Reality (AR). Before we began to develop the system, we studied users needs by employing task analysis methodology.

154 MORTEN FJELD ET AL. According to Leont���ev, not only is activity shaped by physical surroundings, activity in turn shapes the surroundings. When activity shapes those surround- ings what happens is that internal mental activity materializes into artifacts. This process of turning mental activity into an object or objectification is what Leont���ev called exteriorization. While it is obvious that for any individual the moment of exteriorization is an important step in his or her creative design activity, it is perhaps less obvious that this is a crucial step in making ideas accessible to others. From this perspective exteriorization is an important social moment which supports mutual understanding in a collective creative design process. In our design philosophy we take account of the physical and social surround- ings as well as the physical and mental faculties of human beings. We draw on B��dker���s (1991) and Kaptelinin���s (1996) work as a basis for our design process. While we do not consider aspects of human consciousness and emotionality in human development, we note the possible connection between these aspects and human-computer interaction (Nardi, 1996b). A vital part of our design philosophy is the tradition of AR, which enriches natural communication with virtual features. Backed up by activity theory and the usage of AR, we developed groupware for layout planning and configuration tasks this groupware is called the BUILD-IT system (Rauterberg et al., 1997a, 1997b, 1998 Fjeld et al., 1998a, 1998b, 1998c, 1999a, 1999b, 2000, 2001 Fjeld, 2001). This system enables end-users, grouped around a table, to cooperate in the active design of a virtual setting, thus supporting co-located, instead of distributed, inter- action. The multi-user functionality of BUILD-IT overcomes a serious drawback often seen with computer-supported cooperative work (CSCW) systems, namely that they are based on single-user applications1 (Grudin, 1988). We believe that co-location is an indispensable factor for the early stages of a complex planning process. Input and output, however, can be prepared and further developed off- line, using a conventional Computer-Aided Design (CAD) system. Other major projects where graspable groupware was constructed to support planning processes are the metaDESK (Ullmer and Ishii, 1997) and the Environment and Discovery Collaboratory (EDC) (Arias et al., 2000). Section 2 introduces our theoretical background, which stems from activity theory we discuss concepts such as tools, objectification and collective action regulation. The difference between goal-directed pragmatic action and exploratory epistemic action is emphasized. Section 3 describes our design philosophy in terms of AR. Task analysis and the incorporation of both physical and virtual tools are two methods which we show to emerge naturally out of our design philos- ophy. Section 4 demonstrates how our design philosophy was used to develop the groupware called BUILD-IT. Our design philosophy is subsequently applied to the design of tools for graspable groupware. We describe the basic principles of human interaction with graspable tools and show how such tools are developed according to the results of task analysis, whereby we focus on the challenges and problems

PHYSICAL AND VIRTUAL TOOLS 155 we encountered. In Section 5, the benefits of activity theory for groupware design are discussed and a set of design guidelines is outlined. 2. The concept of tools in activity theory To provide a theoretical background, we show how our understanding of activity theory has shaped our goals and the design process of AR groupware. First, our account of the tool concept and objectification is given. Then, collective action regulation is explained. Finally, we introduce two types of complete action regu- lation cycles for goal-directed pragmatic action and for exploratory epistemic action. 2.1. TOOLS AND OBJECTIFICATIONS In the most general sense, activity means a subject���s interaction with his or her surroundings. Modern activity theory originated from Soviet cultural-historical psychology (Vygotsky, 1978 Leont���ev, 1978, 1981), which in turn is rooted in both eighteenth and nineteenth century classical German philosophy ��� from Hegel���s idealism to the historical materialism of Marx and Engels, in which the concept of activity was extensively elaborated. These roots are quite unfamiliar to most Anglo- American readers and have therefore been partly neglected (Kuutti, 1996). Yet, Engestr��m (1991) claims that activity theory today is transcending these origins, becoming truly international and multidisciplinary. Two results of this development are Engestr��m and Middleton (1996) and Nardi (1996a). Fundamental to modern activity theory is the idea that the development of thoughts and cognitive activity requires social interaction and exchange with a physical environment. Via the process of internalization, social interaction turns into mental activity. Handling ever more abstract objects and concepts is part of an individual���s cognitive development. Nevertheless, the physical environment remains important, since it is used for the externalization of thoughts and as external memory. This is particularly important for the abstract planning and configuration tasks we focus on in this paper. Individuals are confronted with tasks that life puts in front of them and they use artifacts as tools or create tools out of their understanding. These tools then become part of the cultural context of other people. A tool mediates an activity, thereby connecting a human being, not only to the world of objects ��� his or her physical surroundings ��� but also to other human beings. At the same time the use of a tool appropriates the collective experience of humanity embodied in that tool (Leont���ev, 1982). In this sense we view the developmental processes of human beings, their physical surroundings and social culture as co-evolutionary. As Engestr��m (1991) puts it: ���The idea is that humans can control their own behavior ��� not ���from the inside���, on the basis of biological urges, but from the outside, using and creating artifacts���.

156 MORTEN FJELD ET AL. Figure 1. Tool production during the eighteenth century (Diderot and D���Alembert, 1778). For planning activities Engestr��m���s idea has been applied to artifacts and tools, resulting in sketches, documents, and three-dimensional (3D) objects or devices. (At this point in our research process, we view Engestr��m���s artifacts and tools as corresponding to Leont���ev���s (1978) objectifications of physical nature.) Hacker et al. (1998) describe the importance of grasping design ideas by sketching and by low-cost prototyping. Such methods may help to achieve design results in a faster and better way than by using abstract design processes. Physical interaction handles, for instance bricks (Fitzmaurice et al., 1995), can be seen as physical devices for exteriorization in a planning process. To illustrate the tradition of tool development in activity theory, a historic example might be of interest. In the historical collection of the ETH library, we found that mathematicians of the eighteenth century (Diderot and D���Alembert, 1778) employed a variety of physical tools2 (Figure 1). Modern mathematics has become even more an abstract field of study. However, the view of tools presented above can also impose some limitations on the potential applications of activity theory. In Virtual Reality (VR) ���the border between a tool and reality is rather unclear information technology can provide the user not only with representations of objects of reality but also with a sort of reality as such, which does not obviously represent anything else and is intended to be just one more environment with which the individual interacts��� (Kaptelinin, 1996, p. 64). This unclear border is a problem VR presents to activity theory it might be solved by enriching activity theory���s basic principles with new ideas from cultural-historical traditions or other approaches for studying the use of artifacts. One answer may be found in the distributed cognition approach, in which internal and external representations of artifacts are examined (Flor and Hutchins, 1991 Hutchins 1991). As in activity theory, this approach describes how we may take advantage of artifacts designed by others in collaborative manipulation. Compared with activity theory, ���distributed cognition has taken most seriously the study of persistent structures, especially artifacts��� (Nardi, 1996c, p. 85). However, the two closely related frameworks show one distinct difference. In activity theory artifacts mediate human thought and human behavior and there is no intrinsic

PHYSICAL AND VIRTUAL TOOLS 157 symmetry between people and their tools. Based on the 19th century debates on epistemology and on what a human being is, gaining knowledge can be seen as an individual process, a process of knowing, which can only take place in an individual. In contrast, distributed cognition puts people and things at the same level they are both ���agents��� in a system (Nardi, 1996c, p. 86). Hence, the distributed cognition approach ignores the faculties of human beings not found within computers, like motive, emotionality, and consciousness. It also ignores for computers their non-human traits, namely their ability to execute programs in a precise and predictable manner. By focusing on a common capability of humans and computers, much is lost on both sides. Another answer is found in B��dker���s (1996, pp. 151���152) characterization of the different focuses in the use activity: ��� The physical aspects ��� support for operations toward the computer application as a physical object. The physical aspects are the conditions for the physical handling of the artifact. . . . ��� The handling aspects ��� support for operations toward the computer applica- tion. . . . The handling aspects are the conditions for transparency of the artifact that allow the user to focus on the ���real��� objects and subjects of the activity. . . . ��� The subject/object-directed aspects ��� the conditions for operations directed toward objects or subjects that we deal with ���in��� the artifact or through the artifact. . . . In our work we understand the physical aspects as how to use the hardware to operate the groupware, the handling aspects as how to operate the groupware, and the subject/object-directed aspects as how to use groupware to solve a task. B��dker talks about ���real��� objects and subjects of the activity. Working with VR, the object of the activity is represented by a virtual world. What a subject situated within that virtual world would see, is represented by a virtual viewpoint. By inter- preting the term ���real��� in the handling aspects as ���virtual���, we may overcome the limitations pointed out by Kaptelinin (1996), i.e. we may clearly define the border between tool and reality. This interpretation takes on particular importance when we develop our so-called navigation methods (Section 4.3.3), which are purely based on the handling aspects. In a more general way, we will use all three focuses in the use activity to structure the description of our design process (Section 4). 2.2. COMMON OBJECTIFICATION In more recent developments, the scope of activity theory has broadened to encompass interaction within a community (Engestr��m, 1991). Since our design philosophy is centered on supporting co-located groups, this new development has been of particular interest to us. As pointed out above, a tool connects an individual to other human beings by mediating activity, thereby becoming part of a cultural context. Of particular interest to us are tools that are made or used by groups.