Multimodal interface towards smartphones: The use of pico projector, passive RGB imaging and active infrared imaging

Abstract

This paper proposes a study regarding smartphone-oriented mobile devices capable of simultaneous passive RGB imaging and active infrared imaging for both projection and image sensing. Using RGB and infrared wavelengths together enables foreground interactive projection and background vision-based analysis to be done without unwanted crosstalk between the two spectrums or visible interruption to audiences. Our proposal includes detachable and rotatable mobile configuration designs, general computing paradigm and multimodal interface strategy; all are presented in a smartphone-oriented manner. Experiments are conducted to clarify efficiency and limitation of our proposal using a proof-of-concept setup. Despite of internal optic and mechanism which requires cooperation from technologys owner to fully accomplish, we believe that our proposal is useful and sustainable, enabling easy compatibility and maintenance with future mobile devices. © 2014 Springer International Publishing.