A 2D-3D Integrated Environment for Cooperative Work Kousuke Nakashima��, Takashi Machida��, Kiyoshi Kiyokawa�� and Haruo Takemura�� ��Graduate School of Information Science, Osaka University ��Cybermedia Center, Osaka University Toyonaka Educational Research Center 1-32 Machikaneyama, Toyonaka, Osaka 560-0043, Japan E-mail: machida, kiyo, takemura @ime.cmc.osaka-u.ac.jp ABSTRACT This paper proposes a novel tabletop display system for natural communication and flexible information sharing. The proposed system is specifically designed for integration of 2D and 3D user in- terfaces, using a multi-user stereoscopic display, IllusionHole. The proposed system takes awareness into consideration and provides both 2D and 3D information and user interfaces. On the display, a number of standard Windows desktop environments are provided as personal workspaces, as well as a shared workspace with a ded- icated graphical user interface. In personal workspaces, users can simultaneously access existing applications and data, and exchange information between personal and shared workspaces. In this way, the proposed system can seamlessly integrate personal, shared, 2D and 3D workspaces with conventional user interfaces and effec- tively support communication and information sharing. To demon- strate capabilities of the proposed display system, a modeling appli- cation has been implemented. A preliminary experiment confirmed the effectiveness of the system. Categories and Subject Descriptors J.6 [Computer-Aided Engineering]: Computer-aided design H.5.3 [Information Interfaces And Presentation]: Group and Organi- zation Interfaces���Collaborative computing, Computer-supported cooperative work General Terms Design Keywords 2D - 3D Integrated User Interface, VNC, IllusionHole 1. INTRODUCTION This paper proposes a novel tabletop display system for natu- ral communication and flexible information sharing. The proposed system is specifically designed for awareness support and integra- Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. VRST���05, November 7���9, 2005, Monterey, California, USA. Copyright 2005 ACM 1-59593-098-1/05/0011 ...$5.00. tion of 2D and 3D user interfaces. Figure 1 shows the proposed system in use. Human-to-human communication and information sharing are crucial for cooperative work. Information sharing means that infor- mation in the collaborative workspace is properly seen and manip- ulated by the participants. Collaboration is promoted by discussing such shared information. There are many types of information to share in collaboration including drawings, documents, maps, and maquettes. To establish an effective cooperative workspace, fol- lowing issues need to be considered. �� Workspace layout for awareness support. �� Compatibility with conventional desktop environment. �� Support of various kinds of information. First issue is a workspace layout. To share information, a ta- ble, a white board, and a large screen display are typically used in cooperative work. When a shared workspace is on the wall (e.g. a white board and a large screen), participants cannot see each other���s faces and gestures while paying attention to shared information on the wall. In this setup, they cannot see information on the wall ei- ther while seeing each other. On the other hand, when participants sit around a table facing each other, it is easy to communicate by exchanging non-verbal communication cues such as facial expres- sions, poses, gestures and viewing directions. These cues play great roles in communication to attract attention. Attentions from non- verbal information are referred to as ���awareness.��� With respect to awareness, it is useful to use a table-type screen [1, 2, 3]. Second issue is compatibility with a conventional desktop envi- ronment. When information is shared on the screen, it is impor- tant that the entire workspace is divided into a shared workspace and a number of personal workspaces, that information in the per- sonal workspace can be presented in the shared workspace, and that shared information can be manipulated by any participant co- operatively. A shared workspace is always accessible by all users, encouraging discussion in collaboration. On the other hand, per- sonal workspaces placed near to the shared workspace are said to have an important role in terms of independent activity and col- laboration efficiency [2]. In addition, a personal workspace should be provided in a way that each participant can easily access and process information in it. In a typical meeting where each partici- pant brings his or her own laptop computer, only a single computer screen is shared at a time and participants need to switch screens frequently. But each participant knows how to use his or her own personal workspace. To provide a better workspace, several studies have developed a collaboration system that seamlessly integrates a conventional desktop graphical user interface (GUI) environment with a shared screen[1, 2, 4, 5]. Some studies have developed a set 16

(a) Proposed system in use (b) A screenshot of the system Figure 1: A tabletop envrionment. of dedicated interaction techniques for shared information on the table[6]. Third issue is about variation of shared information. Shared in- formation can be either 2D or 3D, according to the collaboration task. To support 3D information, a collaborative workspace is ex- pected to present 3D (stereoscopic) images correctly to every par- ticipant so that the entity appears at a single location. Some stud- ies try to integrate conventional computer environments into 3D workspace using augmented reality or mixed reality techniques[7, 8]. In this paper, proposed is a new cooperative tabletop system which integrates four types of workspaces 2D, 3D, personal and shared workspaces. In the following, characteristics and functions of the proposed system are explained in Section 2. Section 3 de- scribes an experiment and discussions. Section 4 describes details of implementation are given. 2. THE COOPERATIVE TABLETOP SYSTEM In this section, an overview of the proposed system is described. First, its display design to support 2D and 3D information is given. Then, its tabletop environment is explained that supports both a conventional desktop environment as a personal workspace, and a shared workspace. Finally, a modeling application is described to the capability of the proposed system. IllusionHole Mask panel Projector Tracker Board Mouse Stick Tracker (a) Entire display system (b) Stick mouse (c) Operation plate Figure 2: Display System. 2.1 A design of 2D-3D display system Our system consists of an IllusionHole [9] to provide a 3D coop- erative workspace, and another overhead projector to display a 2D workspace onto a mask panel of the IllusionHole as shown in Fig- ure 2. A pair of stereoscopic images is displayed using a 60-inch IllusionHole whose hole diameter is 26 cm. Since IllusionHole���s mask panel exclusively shows a pair of stereoscopic images to a user and hides those from other users, multiple (practically up to four) users can simultaneously observe the same 3D scene from their own viewpoints through a pair of polarized glasses with head tracking facility. Unlike a normal IllusionHole, our system uti- lizes a mask panel of IllusionHole for projection of 2D workspace. Note that a pair of polarized grasses is advantageous over a head mounted display (HMD) in terms of easy attachment and periph- eral vision. Peripheral vision is important to recognize other par- ticipants��� faces and gestures. The proposed system is designed to display both 2D and 3D in- formation. In this environment, if different devices are provided for 2D and 3D environments, users need to switch them repeatedly with physical and cognitive burdens. A stick mouse (Figure2(b)) is then provided which is seamlessly available in both 2D and 3D workspaces. A stick mouse works as a normal mouse when it is manipulated on the mask panel on which a 2D pointer appears (2D mode). On the other hand, when it is manipulated inside a cylinder- shaped 3D volume above the mask hole, it works as a 3D pointing device (3D mode) using a Polhemus tracker with six degree-of- freedom (6DOF). In the 3D mode, an arrow-headed 3D cursor ap- pears at the tip of the stick in the 3D workspace. An operation plate (Figure2(c)) is a plastic board with a Polhe- mus tracker with which a variety of 2D-3D coordinated functions can be performed. An operation plate is considered as a second, independent 2D workspace, and it also acts as a bridge between 2D and 3D workspaces. For example, a user can invoke and operate a 2D hierarchical menu on the operation plate, or select a 2D cross section of a 3D model for 2D editing at an arbitrary orientation by placing the board inside the model, or scoop a 3D object to move to the 2D workspace like a palette. For a 2D image on the operation plate, a projection frustum is precisely adjusted to a real frustum defined by the overhead projec- tor and the mask panel (see Figure 2(c)). A 2D image to be shown on the operation plate is rendered as a virtual plate in space at the position and orientation of the actual operation plate and is trans- 17