Human Emotion Detection and Face Recognition System

Abstract

This study presents an integrated Human Emotion Detection and Face Recognition system, combining computer vision and deep learning to perform real-time facial analysis.  The system processes live video to recognize individuals and classify emotions into seven categories (angry, disgust, fear, happy, neutral, sad, surprise) using a Convolutional Neural Network (CNN) trained on the augmented and filtered FER-2013 datasets. Face recognition is achieved through OpenCV’s Haar Cascade for detection and SVM (Support Vector Machine)/KNN (K-Nearest Neighbor) for matching facial features. The system pre-processes the image data which includes grayscale conversion for the optimal CNN and SVM input. The system features an interactive interface with secure authentication, real-time overlays for emotion and identity visualization, and dynamic thresholding to enhance accuracy. Moreover, the system generates dataset of face and emotion detected in CSV file format and generates chart accordingly. The test accuracy obtained from custom CNN model is 74.78%. This project offers significant opportunities for future research, as it intersects with a variety of fields including AI, computer vision, healthcare, education, human behavior, security, and ethics.