Hybrid Deep Visual Intelligence Framework for Public Health and Safety Enforcement

Abstract

This study highlights the urgent need for enhanced visual monitoring technologies to support public health and safety enforcement in crowded situations, in which individuals could potentially violate compliance and other norms. Most traditional surveillance methodologies lack either the scalability or precision to squash complaints about compliance violations – for example, improper mask usage, failure to maintain a minimum social distance, or other visible anomalies. To address these issues, we designed a Hybrid Deep Visual Intelligence Framework that integrates multiscale convolutional neural networks (e.g., ResNet-101, YOLOv8) for object detection with a sequence of 3D CNNs for temporal activity modeling. We also utilized Vision Transformers with cross-attention that spans external contextual metadata. The input data source was multimodal (including closed-circuit television footage, drone footage, and thermal imagery) to derive compliance scores or alerts in real-time. The evaluation for the constrained performance refers to independent public datasets representing four scenarios-indoor hospitals, outdoor markets, nighttime transport hubs, and large event stadiums using UCF-Crime datasets, Okutama-Action datasets, and AI City Challenge for autonomous vehicle innovative technology. The performance results demonstrated superior performance, achieving up to 96.4% detection accuracy and anomaly detections with F1-scores of 0.89, compared to traditional approaches, which were improved by 5-10%. The results included significant processing frame rates of greater than 24 frames per second that facilitated near-real-time execution. In conclusion, our proposed solutions represent a valid, privacy-compliant deployment option that adapts and scales for enforcement purposes, demonstrating significant options for supporting public health compliance and security in complex public spaces.