Co-recognition of Images and Videos: Unsupervised Matching of Identical Object Patterns and Its Applications

Abstract

In this chapter, we address the problem of detecting, matching, and seg-menting all identical object-level patterns from images or videos in an unsupervised way, called the " co-recognition " problem. In an unsupervised setting without any prior knowledge of specific target objects, it relies entirely on geometric and pho-tometric relations of visual features. To solve this problem, a multi-layer match-growing framework is proposed which explores given visual data by intra-layer ex-pansion and inter-layer merge. We demonstrate the effectiveness of this approach on identical object detection, image retrieval, symmetry detection, and action recogni-tion. These applications will validate the usefulness of co-recognition to several vision problems. 5.1 Introduction In detection and recognition of visual objects in images or actions in videos, most of computer vision approaches require some level of supervision to specify or learn a model [5, 12, 14, 24, 34, 45, 50, 52]. In such cases, labeled images with bound-ing boxes or uncluttered video clips are usually adopted for the purpose. Recent categorization methods based on latent topic models have proposed weakly super-vised or unsupervised learning approaches using a decent amount of training im-ages [23, 53, 55]. In real-world images or videos, however, multiple objects of simi-lar appearance often show up even in a single view. For example, an image can con-tain replicas or multiple shots of identical objects, and a video clip often includes the same actions in different spatio-temporal regions. Humans have no difficulty