Pluggable Drone Imaging Analysis Framework for Mob Detection during Open-air Events

Abstract

Drones and thermal cameras are often combined within applications such as search and rescue, and fire fighting. Due to vendor specific hardware and software, applications for these drones are hard to develop and maintain. As a result, a pluggable drone imaging analysis architecture is proposed that facilitates the development of custom image processing applications. This architecture is prototyped as a microservice-based plugin framework and allows users to build image processing applications by connecting media streams using microservices that connect inputs (e.g. regular or thermal camera image streams) to image analysis services. The prototype framework is evaluated in terms of modifiability, interoperability and performance. This evaluation has been carried out on the use case of detecting large crowds of people (mobs) during open-air events. The framework achieves modifiability and performance by being able to work in soft real-time and it achieves the interoperability by having an average successful exchange ratio of 99.998%. A new dataset containing thermal images of such mobs is presented, on which a YOLOv3 neural network is trained. The trained model is able to detect mobs on new thermal images in real-time achieving frame rates of 55 frames per second when deployed on a modern GPU.