What and Where: 3D Object Recognition with Accurate Pose

Abstract

Many applications of 3D object recognition, such as aug- mented reality or robotic manipulation, require an accurate solution for the 3D pose of the recognized objects. This is best accomplished by building a metrically accurate 3D model of the object and all its fea- ture locations, and then fitting this model to features detected in new images. In this chapter, we describe a system for constructing 3D met- ric models from multiple images taken with an uncalibrated handheld camera, recognizing these models in new images, and precisely solving for object pose. This is demonstrated in an augmented reality applica- tion where objects must be recognized, tracked, and superimposed on new images taken from arbitrary viewpoints without perceptible jitter. This approach not only provides for accurate pose, but also allows for integration of features from multiple training images into a single model that provides for more reliable recognition.