Finger gesture-based three-dimension mobile user interaction using a rear-facing camera

Abstract

In this paper, we propose a 3D mobile user interaction for 3D space using a mono camera in mobile devices. The proposed system can support a UI using only the index finger of the hand holding the mobile device. Proposed fingertip detection algorithm was developed to overcome the limitation of the skin color segmentation of previous studies under varying lighting conditions and complex background using MMGC and AdaBoost algorithm. Also in proposed 3D mobile user interface, the estimated coordinates and area of the fingertip can be mapping 3D space of implemented application. The experimental result indicates the feasibility of the proposed algorithm for finger movement-based mobile user interfaces. © 2013 SERSC.