Visual servoing

Abstract

Vision allows a robotic system to obtain geometrical and qualitative information on the surrounding environment to be used both for motion planning and control. In particular, control based on feedback of visual measurements is termed visual servoing. In the first part of this chapter, some basic algorithms for image processing, aimed at extracting numerical information referred to as image feature parameters, are presented. These parameters, relative to images of objects present in the scene observed by a camera, can be used to estimate the pose of the camera with respect to the objects and vice versa. To this end, analytical pose estimation methods, based on the measurement of a certain number of points or correspondences are presented. Also, numerical pose estimation methods, based on the integration of the linear mapping between the camera velocity in the operational space and the time derivative of the feature parameters in the image plane, are introduced. In cases in which multiple images of the same scene, taken from different viewpoints, are available, additional information can be obtained using stereo vision techniques and epipolar geometry. A fundamental operation is also camera calibration; to this end, a calibration method based on the measurement of a certain number of correspondences is presented. Then, the two main approaches to visual servoing are introduced, namely position-based visual servoing and image-based visual servoing, as well as a scheme, termed hybrid visual servoing, which combines the benefits of both approaches.