Smart Surveillance System using Deep Learning

Abstract

Traditional closed circuit television (CCTV) requires that a human actively and closely monitoring the output feed of the camera 24/7, which is inefficient and costly. Therefore, there is a need for a smart system that can monitor the feed of a CCTV camera and recognize human actions in real time with little or no human intervention. In this paper, we put forward a smart surveillance system, where images from various devices (for instance a colored CCTV camera) can be fed into, and in these images, human agents are detected using an object detection algorithm and annotated with a bounding box. The human subjects in these bounding boxes are then consequently fed into a Deep Learning model to classify activities. Hence, we propose a pipeline of first detecting and then classifying human activities in images captured by surveillance devices such as a CCTV camera. Our work therefore can be divided into two parts, object detection and classification, for detection, we utilize the state of the art object detection algorithm YOLOv3 and for classifying the human activities we train a deep learning model on the Stanford 40 action dataset. The dataset includes simple human actions such as jumping and running to more complex and composite actions such as playing guitar or riding a bike. We also compare the performance in terms of accuracy of our base CNN model with other approaches, which are trained for human activity recognition on the Stanford 40 action dataset. Our deep learning model obtained the maximum classification accuracy of 87% on parent images obtained from the Stanford 40 action dataset and a classification accuracy of 66% on images where human subjects were extracted from the parent images.