Hardware Implementation of KLT Tracker for Real-Time Intruder Detection and Tracking Using On-board Camera

Abstract

This paper presents the hardware implementation of a new hybrid KLT (Kanade–Lucas–Tomasi) algorithm for the sake of real-time intruder detection and tracking. The proposed algorithm takes advantage of BRISK sampling scheme to improve the feature detection and tracking performance. The RTL level hardware logics for feature detection and tracking are designed to implement the hybrid KLT tracker using on-board SoC FPGA. In particular, the hybrid KLT tracker is realized via a five-stage pipelining architecture to increase the throughput rate but also the availability of continuous output. The RTL hardware logics are written in VHDL codes and are executed on an Intel’s Cyclone V SoC development board while the video frames of VGA resolution are obtained from the on-board camera at 60 fps. The performance of the hybrid KLT tracker algorithm has been tested and validated via the software simulation based on the reference codes written in C/C++. The speed acceleration due to the hardware implementation has been verified so that the computational throughput increases by 24.46 times faster than the software implementation, while the object tracking performance remains the same. Consequently, it has been shown that the hardware implementation of KLT tracker for intruder detection and tracking can operate in real-time while providing the supplement surveillance information using on-board camera for mid-air collision avoidance of UAV.