Recognition and Grasping Objects from 3D Environment by Combining Depth and Color Stereo Image in the Mobile Picking Robot System

Abstract

This paper proposes recognition and grasping objects from 3D environment by combining depth and color stereo image in the mobile picking robot system. To do this task, the followings are done. Firstly, an image processing system including Kinect camera sensor is described. Secondly, RGB color map and new depth map for image inpainting are obtained using Kinect SDK mapping function to align RGB image with depth image. Thirdly, the new depth map are segmented to distinguish between the image background and the objects that should be recognized. The feature colours are generated based on colour histograms. Euclidean distance is used to measure the similarity between the feature vectors computed from the colour image and the feature vectors stored in a database. Fourthly, by converting RGB map and new depth map into 3D point clouds, an algorithm for localizing handle-like grasp affordances is proposed. The main idea is to search the point cloud for neighborhoods that satisfy handle-like grasp affordances and can be grasped by the end-effector of the manipulator. Finally, the effectiveness of the proposed algorithms is verified by using experiment. The experimental results show that the mobile picking robot successfully detects an object and finds its grasping points with an acceptable small error.