An efficient online hierarchical supervoxel segmentation algorithm for time-critical applications

Abstract

Video segmentation has been used in a variety of computer vision algorithms as a pre-processing step. Despite its wide application, many existing algorithms require preloading all or part of the video and batch processing the frames, which introduces temporal latency and significantly increases memory and computational cost. Other algorithms rely on human specification for segmentation granularity control. In this paper, we propose an online, hierarchical video segmentation algorithm with no latency. The new algorithm leverages a graph-based image segmentation technique and recent advances in dense optical flow. Our contributions include: 1) an efficient, yet effective probabilistic segment label propagation across consecutive frames; 2) a new method for label initialization for the incoming frame; and 3) a temporally consistent hierarchical label merging scheme. We conduct a thorough experimental analysis of our algorithm on a benchmark dataset and compare it with state-of-the-art algorithms. The results indicate that our algorithm achieves comparable or better segmentation accuracy than state-of-the- art batch-processing algorithms, and outperforms streaming algorithms despite a significantly lower computation cost, which is required for time-critical applications.