Interacting at a distance using semantic snarfing

Abstract

It is difficult to interact with computer displays that are across the room, which can be important in meetings and when controlling computerized devices. A popular approach is to use laser pointers tracked by a camera, but interaction techniques using laser pointers tend to be imprecise, error-prone, and slow. Therefore, we have developed a new interaction style, where the laser pointer (or other pointing technique such pointing with a finger or even eye tracking) indicates the region of interest, and then the item there is copied ("snarfed") to the user’s handheld device, such as a Palm or PocketPC handheld. If the content changes on the PC, the handheld’s copy will be updated as well. Interactions can be performed on the handheld using familiar direct manipulation techniques, and then the modified version is sent back to the PC. The content often must be reformatted to fit the properties of the handheld to facilitate natural interaction.