DeepTrackNet: Camera Based End to End Deep Learning Framework for Real Time Detection, Localization and Tracking for Autonomous Vehicles

Abstract

Vehicle detection and tracking in real time remains as the most challenging tasks for the autonomous vehicles and mobile robots. The designed algorithms should have less latency and with high accuracy and performance. This paper proposed a unified end to end deep learning framework for real-time detection, localization and tracking for autonomous vehicles called DeepTrackNet. In the proposed DeepTrackNet architecture Mobilenet SSD is used for vehicle detection, localization and deep regression network is used for vehicle tracking. Based on the experimental analysis performed we infer that the proposed DeepTrackNet architecture has produced satisfactory results for real-time vehicle detection and tracking on an Nvidia Jetson embedded computing platform using monocular camera with a reasonable latency. Mobilenet SSD took 84 (ms) median detection time and deep regression tracker has the least overall average failure frames at 4.388 on the VTB TB-100 dataset (http://cvlab.hanyang.ac.kr/tracker_benchmark/datasets.html ). The code along with detailed results are available in the GitHub repository (https://github.com/dineshresearch/DeepTrackNet ).