Real‐time and deep learning based vehicle detection and classification using pixel‐wise code exposure measurements

Abstract

One key advantage of compressive sensing is that only a small amount of the raw video data is transmitted or saved. This is extremely important in bandwidth constrained applications. Moreover, in some scenarios, the local processing device may not have enough processing power to handle object detection and classification and hence the heavy duty processing tasks need to be done at a remote location. Conventional compressive sensing schemes require the compressed data to be reconstructed first before any subsequent processing can begin. This is not only time consuming but also may lose important information in the process. In this paper, we present a real‐time framework for processing compressive measurements directly without any image reconstruction. A special type of compressive measurement known as pixel‐wise coded exposure (PCE) is adopted in our framework. PCE condenses multiple frames into a single frame. Individual pixels can also have different exposure times to allow high dynamic ranges. A deep learning tool known as You Only Look Once (YOLO) has been used in our real‐time system for object detection and classification. Extensive experiments showed that the proposed real‐time framework is feasible and can achieve decent detection and classification performance.