DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

ReSyRIS: A Real-Synthetic Rock Instance Segmentation Dataset for Training and Benchmarking

Boerdijk, Wout and Müller, Marcus Gerhard and Durner, Maximilian and Triebel, Rudolph (2023) ReSyRIS: A Real-Synthetic Rock Instance Segmentation Dataset for Training and Benchmarking. In: 2023 IEEE Aerospace Conference, AERO 2023. IEEE. 2023 IEEE Aerospace Conference, 2023-03-04 - 2023-03-11, Big Sky, USA. doi: 10.1109/AERO55745.2023.10115802. ISBN 978-166549032-0. ISSN 1095-323X.

[img] PDF

Official URL: https://ieeexplore.ieee.org/document/10115802


The exploration of our solar system for understanding its creation and investigating potential chances of life on other celestial bodies is a fundamental drive of human mankind. After early telescope-based observation, Apollo 11 was the first space mission able to collect samples on the lunar surface and take them back to earth for analysis. Especially in recent years this trend accelerates again, and many successors were (or are in the process of being) launched into space for extra-terrestrial sample extraction. Yet, the abundance of potential failures makes these missions extremely challenging. For operations aimed at deeper parts of the solar system, the operational working distance extends even further, and communication delay and limited bandwidth increase complexity. Consequently, sample extraction missions are designed to be more autonomous in order to carry out large parts without human intervention. One specific sub-task particularly suitable for automation is the identification of relevant extraction candidates. While there exists several approaches for rock sample identification, there are often limiting factors in the form of applicable training data, lack of suitable annotations of the very same, and unclear performance of the algorithms in extra-terrestrial environments because of inadequate test data. To address these issues, we present ReSyRIS (Real-Synthetic Rock Instance Segmentation Dataset), which consists of real-world images together with their manually created synthetic counterpart. The real-world part is collected in a quasi-extra-terrestrial environment on Mt. Etna in Sicily, and focuses recordings of several rock sample sites. Every scene is re-created in OAISYS, a Blender-based data generation pipeline for unstructured outdoor environments, for which the required meshes and textures are extracted from the volcano site. This allows not only precise re-construction of the scenes in a synthetic environment, but also generation of highly realistic training data with automatic annotations in similar fashion to the real recordings. We finally investigate the generalization capability of a neural network trained on incrementally altered versions of synthetic data to explore potential sim-to-real gaps. The real-world dataset together with the OAISYS config files to create its synthetic counterpart are publicly available at https://rm.dlr.de/resyris_en. With this novel benchmark on extra-terrestrial stone instance segmentation we hope to further push the boundaries of autonomous rock sample extraction.

Item URL in elib:https://elib.dlr.de/194113/
Document Type:Conference or Workshop Item (Speech)
Title:ReSyRIS: A Real-Synthetic Rock Instance Segmentation Dataset for Training and Benchmarking
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Boerdijk, WoutUNSPECIFIEDhttps://orcid.org/0000-0003-0789-5970UNSPECIFIED
Müller, Marcus GerhardUNSPECIFIEDhttps://orcid.org/0000-0003-4283-6693UNSPECIFIED
Durner, MaximilianUNSPECIFIEDhttps://orcid.org/0000-0001-8885-5334UNSPECIFIED
Triebel, RudolphUNSPECIFIEDhttps://orcid.org/0000-0002-7975-036XUNSPECIFIED
Journal or Publication Title:2023 IEEE Aerospace Conference, AERO 2023
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In ISI Web of Science:Yes
Keywords:rock, stone, instance segmentation, detection, autonomous exploration, moon, dataset
Event Title:2023 IEEE Aerospace Conference
Event Location:Big Sky, USA
Event Type:international Conference
Event Start Date:4 March 2023
Event End Date:11 March 2023
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Multisensory World Modelling (RM) [RO]
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013)
Deposited By: Boerdijk, Wout
Deposited On:27 Apr 2023 10:16
Last Modified:24 Apr 2024 20:54

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.