Towards a smart exploitation of GPUs for low energy motion estimation using full HD and 4K videos

Abstract

Video processing and more particularly motion tracking algorithms present a necessary tool for various domains related to computer vision such as motion recognition, depth estimation and event detection. However, the use of high definitions videos (HD, Full HD, 4K, etc.) cause that current implementations, even running on modern hardware, no longer respect the requirements of real-time treatment. In this context, several solutions have been proposed to overcome this constraint, by exploiting graphic processing units (GPUs). Although, they benefit from the high power of GPU, none of them is able to provide efficient dense and sparse motion tracking within high definition videos efficiently. In this work, we propose a GPU and Multi-GPU based method for both sparse and dense optical flow motion tracking using the Lucas-Kanade algorithm. Our method presents an efficient exploitation and management of single or/and multiple GPU memories, according to the type of applied implementation: sparse or dense. The sparse implementation allows tracking meaningful pixels, which are detected with the Harris corner detector. The dense implementation requires more computation since it is applied on each pixel of the video. Within our approach, high definition videos are processed on GPUs while low resolution videos are treated on CPUs. As result, our method allows real-time sparse and dense optical flow computation on videos in Full HD or even 4K format. The exploitation of multiple GPUs presents performance that scale up very well. In addition to these performances, the parallel implementations offered lower power consumption as result of the fast treatment.