Denninger, Maximilian und Triebel, Rudolph (2023) 3D Semantic Scene Reconstruction from a Single Viewport. In: Proceedings of the 3rd International Conference on Image Processing and Vision Engineering (IMPROVE 2023), 1, Seiten 15-26. SciTePress. 3rd International Conference on Image Processing and Vision Engineering, 2023-04-21 - 2023-04-23, Prague, Czech Republic. doi: 10.5220/0011747700003497. ISBN 978-989-758-642-2. ISSN 2795-4943.
PDF
14MB |
Offizielle URL: https://dx.doi.org/10.5220/0011747700003497
Kurzfassung
We introduce a novel method for semantic volumetric reconstructions from a single RGB image. To overcome the problem of semantically reconstructing regions in 3D that are occluded in the 2D image, we propose to combine both in an implicit encoding. By relying on a headless autoencoder, we are able to encode semantic categories and implicit TSDF values into a compressed latent representation. A second network then uses these as a reconstruction target and learns to convert color images into these latent representations, which get decoded after inference. Additionally, we introduce a novel loss-shaping technique for this implicit representation. In our experiments on the realistic benchmark Replica dataset, we achieve a full reconstruction of a scene, which is visually and in terms of quantitative measures better than current methods while only using synthetic data during training. On top of that, we evaluate our approach on color images recorded in the wild.
elib-URL des Eintrags: | https://elib.dlr.de/195220/ | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Dokumentart: | Konferenzbeitrag (Vortrag) | ||||||||||||
Zusätzliche Informationen: | Best Student Paper Award | ||||||||||||
Titel: | 3D Semantic Scene Reconstruction from a Single Viewport | ||||||||||||
Autoren: |
| ||||||||||||
Datum: | April 2023 | ||||||||||||
Erschienen in: | Proceedings of the 3rd International Conference on Image Processing and Vision Engineering (IMPROVE 2023) | ||||||||||||
Referierte Publikation: | Ja | ||||||||||||
Open Access: | Ja | ||||||||||||
Gold Open Access: | Nein | ||||||||||||
In SCOPUS: | Nein | ||||||||||||
In ISI Web of Science: | Nein | ||||||||||||
Band: | 1 | ||||||||||||
DOI: | 10.5220/0011747700003497 | ||||||||||||
Seitenbereich: | Seiten 15-26 | ||||||||||||
Verlag: | SciTePress | ||||||||||||
ISSN: | 2795-4943 | ||||||||||||
ISBN: | 978-989-758-642-2 | ||||||||||||
Status: | veröffentlicht | ||||||||||||
Stichwörter: | 3D Reconstruction, 3D Segmentation, Single View, Sim2real | ||||||||||||
Veranstaltungstitel: | 3rd International Conference on Image Processing and Vision Engineering | ||||||||||||
Veranstaltungsort: | Prague, Czech Republic | ||||||||||||
Veranstaltungsart: | internationale Konferenz | ||||||||||||
Veranstaltungsbeginn: | 21 April 2023 | ||||||||||||
Veranstaltungsende: | 23 April 2023 | ||||||||||||
HGF - Forschungsbereich: | Luftfahrt, Raumfahrt und Verkehr | ||||||||||||
HGF - Programm: | Raumfahrt | ||||||||||||
HGF - Programmthema: | Robotik | ||||||||||||
DLR - Schwerpunkt: | Raumfahrt | ||||||||||||
DLR - Forschungsgebiet: | R RO - Robotik | ||||||||||||
DLR - Teilgebiet (Projekt, Vorhaben): | R - Multisensorielle Weltmodellierung (RM) [RO] | ||||||||||||
Standort: | Oberpfaffenhofen | ||||||||||||
Institute & Einrichtungen: | Institut für Robotik und Mechatronik (ab 2013) > Perzeption und Kognition Institut für Robotik und Mechatronik (ab 2013) | ||||||||||||
Hinterlegt von: | Strobl, Dr. Klaus H. | ||||||||||||
Hinterlegt am: | 30 Mai 2023 07:15 | ||||||||||||
Letzte Änderung: | 24 Apr 2024 20:55 |
Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags