DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

3D Semantic Scene Reconstruction from a Single Viewport

Denninger, Maximilian and Triebel, Rudolph (2023) 3D Semantic Scene Reconstruction from a Single Viewport. In: Proceedings of the 3rd International Conference on Image Processing and Vision Engineering (IMPROVE 2023), 1, pp. 15-26. SciTePress. 3rd International Conference on Image Processing and Vision Engineering, 2023-04-21 - 2023-04-23, Prague, Czech Republic. doi: 10.5220/0011747700003497. ISBN 978-989-758-642-2. ISSN 2795-4943.

[img] PDF

Official URL: https://dx.doi.org/10.5220/0011747700003497


We introduce a novel method for semantic volumetric reconstructions from a single RGB image. To overcome the problem of semantically reconstructing regions in 3D that are occluded in the 2D image, we propose to combine both in an implicit encoding. By relying on a headless autoencoder, we are able to encode semantic categories and implicit TSDF values into a compressed latent representation. A second network then uses these as a reconstruction target and learns to convert color images into these latent representations, which get decoded after inference. Additionally, we introduce a novel loss-shaping technique for this implicit representation. In our experiments on the realistic benchmark Replica dataset, we achieve a full reconstruction of a scene, which is visually and in terms of quantitative measures better than current methods while only using synthetic data during training. On top of that, we evaluate our approach on color images recorded in the wild.

Item URL in elib:https://elib.dlr.de/195220/
Document Type:Conference or Workshop Item (Speech)
Additional Information:Best Student Paper Award
Title:3D Semantic Scene Reconstruction from a Single Viewport
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Denninger, MaximilianUNSPECIFIEDhttps://orcid.org/0000-0002-1557-2234UNSPECIFIED
Triebel, RudolphUNSPECIFIEDhttps://orcid.org/0000-0002-7975-036XUNSPECIFIED
Date:April 2023
Journal or Publication Title:Proceedings of the 3rd International Conference on Image Processing and Vision Engineering (IMPROVE 2023)
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In ISI Web of Science:No
Page Range:pp. 15-26
Keywords:3D Reconstruction, 3D Segmentation, Single View, Sim2real
Event Title:3rd International Conference on Image Processing and Vision Engineering
Event Location:Prague, Czech Republic
Event Type:international Conference
Event Start Date:21 April 2023
Event End Date:23 April 2023
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Multisensory World Modelling (RM) [RO]
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013) > Perception and Cognition
Institute of Robotics and Mechatronics (since 2013)
Deposited By: Strobl, Dr. Klaus H.
Deposited On:30 May 2023 07:15
Last Modified:24 Apr 2024 20:55

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.