Development of a Bi-level Web Connected Home Access System using Multi-Deep Learning Neural Networks

Abstract

Home entrance is a vital entry point that should be secured at all times. A bi-level home access system was designed and developed using face authentication and hand gesture recognition. The system’s mainframe runs on a Raspberry Pi 3 minicomputer. The board serves as the computing platform to process various deep learning algorithms for face authentication and hand gesture recognition. It also serves as a communication hub which allows registered users to communicate with the system remotely via mobile application. Home occupants may also register emergency contacts such as their neighbours’ for quick response at their property. An Android mobile application was developed for remote user interface. Google Firebase platform was used to store user profile and historical data. The face authentication consists of two steps, namely face detection and face recognition. The Multitask Cascaded Convolutional Neural Network (MTCNN) was employed for face detection, while the Inception ResNet was used for face recognition. Upon successful face authentication, the system proceeds to read the user’s hand gesture. First, the system detects the hand using Single Shot MultiBox Detector (SSD) that runs on a Convolutional Neural Network (CNN). Next, a sequence of hand pose is recognised using the conventional CNN method. Based on experiments, the average detection/recognition accuracy under normal operating conditions using real face and realvideo captured by the system is approximately 95.7%. An occupant needs approximately 7s to complete the process to enter the house.