A hybrid algorithm for human interaction recognition from drone videos: experimental analysis to enhance disaster response and rescue

Abstract

Human interaction recognition leverages computer vision and machine learning to detect and identify people, objects, and their relationships in videos. However, challenges such as video quality, interpersonal differences, large dataset requirements, scale variations, and computational complexity persist. This study introduces a novel method that incorporates Convolutional Neural Networks (CNNs) to enhance the efficiency of recognizing human interactions in drone videos, particularly for emergency response and disaster management. Our approach semantically predicts human behaviour based on object recognition and scene understanding. The method extracts First and Last Images (FLIs) from videos and employs a Support Vector Machine (SVM) to classify high-level features in key frames using CNN, eliminating the need for human tracking or processing entire video sequences. We examined the approach on UCF Sports Action and Olympic Sports benchmark datasets before applying it to flood scenarios. Additionally, drone footage was collected as a benchmark dataset, and the algorithm was then applied to YouTube flood videos to evaluate the model’s performance. Results show that the proposed method achieves 90.42% accuracy, improving existing approaches by 3.82% while reducing computational complexity. This technique has significant applications in video surveillance, human-computer interaction, real-time monitoring, and rescue operations.