DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Part semantic segmentation of complete and partial point clouds using deep learning

Judeh, Raja (2020) Part semantic segmentation of complete and partial point clouds using deep learning. DLR-Interner Bericht. DLR-IB-RM-OP-2020-169. Master's. Technische Universität München.

[img] PDF - Only accessible within DLR


The availability of the 3D sensors and the computational power in the recent years has allowed the computer vision community to start using deep learning approaches for processing 3D data, inspired by the great advancements that deep learning achieved on 2D data to exploit the rich geometrical information that 3D data provides. 3D data can be easily captured and compactly stored when represented by a 3D data representation called point clouds, which describe 3D data solely using their 3D coordinates in the 3D space. Due to their unstructured nature, processing point clouds using deep learning is tricky and cannot be achieved using the traditional deep learning architectures such as CNNs. PointNet and PointNet++ are the pioneer deep learning architectures in processing raw point clouds. This thesis was done in the German Aerospace Center (DLR) in the Institute of Robotics and Mechatronics as part of a project that focuses on the development of service robots that use point clouds as the main source of visual information to perform proper grasping of household objects. Such a complicated task does not only require detailed geometrical information of the objects but also the semantic understating of these objects and their parts. Therefore, the main problem that this thesis addresses is the part semantic segmentation of point clouds. The PointNet++ architecture is capable of solving this problem by being trained, in a supervised manner, to classify the individual points in each point cloud as belonging to one object’s part or another. However, the analyses performed in this thesis show that the PointNet++ architecture fails to segment partially occluded point clouds, which is the case of real-world point clouds. Hence, this thesis mainly focuses on improving the original PointNet++ architecture, not only on partial data but also on complete data. In addition to modifying the original architecture to become translation-invariant, which has shown significant improvements on partial data, this thesis introduces three main variations of PointNet++: SkipNet, intelligent feature propagation (IFP), and point-pair similarity loss (PPSLoss), to further improve the original architecture. The results of this thesis show that combining the first two variations in a translation-invariant architecture outperforms the original architecture on both, complete and partial data. However, when the third variation is added to the new architecture, further improvements are achieved on complete data, but a slight drop in the performance is observed on partial data.

Item URL in elib:https://elib.dlr.de/138294/
Document Type:Monograph (DLR-Interner Bericht, Master's)
Title:Part semantic segmentation of complete and partial point clouds using deep learning
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Judeh, RajaTechnische Universität MünchenUNSPECIFIEDUNSPECIFIED
Refereed publication:No
Open Access:No
Keywords:point clouds, semantic segmentation, object parts, deep learning
Institution:Technische Universität München
Department:Department of Electrical and Computer Engineering
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Space System Technology
DLR - Research area:Raumfahrt
DLR - Program:R SY - Space System Technology
DLR - Research theme (Project):R - Vorhaben Multisensorielle Weltmodellierung (old)
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013) > Perception and Cognition
Deposited By: Hillenbrand, Ulrich
Deposited On:26 Nov 2020 09:58
Last Modified:26 Nov 2020 09:58

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.