Object semantic segmentation in point clouds-comparison of a deep learning and a knowledge-based method

Abstract

Through the power of new sensing technologies, we are increasingly digitizing the real world. However, instruments produce unstructured data, mainly in the form of point clouds for 3d data and images for 2d data. Nevertheless, many applications (such as navigation, survey, infrastructure analysis) need structured data containing objects and their geometry. Various computer vision approaches have thus been developed to structure the data and identify objects therein. They can be separated into model-driven, data-driven, and knowledge-based approaches. Model-driven approaches mainly use the information on the objects contained in the data and are thus limited to objects and context. Among data-driven approaches, we increasingly find deep learning strategies because of their autonomy in detecting objects. They identify reliable patterns in the data and connect these to the object of interest. Deep learning approaches have to learn these patterns in a training stage. Knowledge-based approaches use characteristic knowledge from different domains allowing the detection and classification of objects. The knowledge must be formalized and substitutes the training for deep learning. Semantic web technologies allow the management of such human knowledge. Deep learning and knowledge-based approaches have already shown good results for semantic segmentation in various examples. The common goal but the different strategies of the two approaches engaged our interest in doing a comparison to get an idea of their strengths and weaknesses. To fill this knowledge gap, we applied two implementations of such approaches to a mobile mapping point cloud. The detected object categories are car, bush, tree, ground, streetlight and building. The deep learning approach uses a convolutional neural network, whereas the knowledge-based approach uses standard semantic web technologies such as sparql and owl2to guide the data processing and the subsequent classification as well. The lidar point cloud used was acquired by a mobile mapping system in an urban environment and presents various complex scenes, allowing us to show the advantages and disadvantages of these two types of approaches. The deep learning and knowledge-based approaches produce a semantic segmentation with an average f1 score of 0.66 and 0.78, respectively. Further details are given by analyzing individual object categories allowing us to characterize specific properties of both types of approaches.