Integrating Printed and Digital Documents: Interaction Models and Techniques for Collaborative Knowledge Work

Abstract

Today's knowledge work is characterized by the simultaneous use of paper and digital documents. This implies disruptive transitions from paper to digital media and vice versa, particularly because interaction techniques differ in the two worlds. For instance, traditional references are created and followed differently than digital hyperlinks. This thesis advances the integration of paper and digital documents. It contributes empirically-grounded conceptual work in the fields of interaction theory, interaction techniques and interactive systems. Three initial field studies explore how paper and digital media are used in learning at universities. They provide evidence for the high importance of paper and show that, in many settings, the use of printed information exists in parallel with using digital documents. Based on these results, an ecological perspective of knowledge work is chosen as the approach for developing the theoretical basis of this thesis. This perspective advocates an integral view on the ensemble of collaborating users, of physical and digital artifacts, of work practices and of their interplay. By generalizing the findings of the empirical field studies and based on the ecological perspective, we develop a theoretical interaction model of Pen-and-Paper User Interfaces (PPUIs). This model is of analytical value and provides guidance on how to design PPUIs that are easy and intuitive to use. Its underlying principle is a systematic separation between a semantic and a syntactic level of interaction. Based on this separation, syntactic interaction primitives are identified, which act as building blocks for PPUIs that support semantic activities. Furthermore, we contribute novel interaction techniques and visualizations for cross-media knowledge work with documents. These are based on the interaction model and provide support for annotating, linking and tagging, all in a hybrid setting of printed and digital documents. First, in the field of paper-based annotation, we introduce the concept of user-adaptable printed interfaces. Moreover, we significantly advance asynchronous sharing by presenting a paper-based mechanism for sharing annotations and a visualization that integrates handwritten annotations of multiple users. Second, we contribute a pen-based interaction technique for creating and following cross-media hyperlinks between printed and digital documents. The same digital pen and the same interactions can be used both on printed documents and on digital documents on a tabletop screen. Third, we contribute four novel techniques for tagging documents and processes. These offer a rich user experience being inspired by the varied practices of using paper. They leverage tangible stickers, paper cards for defining and applying tags, printed buttons, and other tangible objects. Moreover, this thesis contributes CoScribe, a system framework for collaborative cross-media knowledge work. This integrates the interaction techniques and visualizations into a consistent and coherent concept. Following the ecological perspective, CoScribe covers entire workflows with a strong emphasis on collaboration. Finally, we present evaluation results. We implemented a working prototype of CoScribe, which was used in three evaluation studies. Their results provide evidence that CoScribe significantly enhances both work performance and user satisfaction.