DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulation

Liu, Ziyuan and Liu, Wei and Qin, Yuzhe and Xiang, Fanbo and Gou, Minghao and Xin, Songyan and Roa Garzon, Máximo Alejandro and Calli, Berk and Su, Hao and Sun, Yu and Tan, Ping (2022) OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulation. IEEE Robotics and Automation Letters, 7 (1), pp. 486-493. IEEE - Institute of Electrical and Electronics Engineers. doi: 10.1109/LRA.2021.3129136. ISSN 2377-3766.

[img] PDF - Only accessible within DLR - Preprint version (submitted draft)

Official URL: https://ieeexplore.ieee.org/document/9619915/


In this paper, we propose a cloud-based benchmark for robotic grasping and manipulation, called the OCRTOC benchmark. The benchmark focuses on the object rearrangement problem, specifically table organization tasks. We provide a set of identical real robot setups and facilitate remote experiments of standardized table organization scenarios in varying difficulties. In this workflow, users upload their solutions to our remote server and their code is executed on the real robot setups and scored automatically. After each execution, the OCRTOC team resets the experimental setup manually. We also provide a simulation environment that researchers can use to develop and test their solutions. With the OCRTOC benchmark, we aim to lower the barrier of conducting reproducible research on robotic grasping and manipulation and accelerate progress in this field. Executing standardized scenarios on identical real robot setups allows us to quantify algorithm performances and achieve fair comparisons. Using this benchmark we held a competition in the 2020 International Conference on Intelligence Robots and Systems (IROS 2020). In total, 59 teams took part in this competition worldwide. We present the results and our observations of the 2020 competition, and discuss our adjustments and improvements for the upcoming OCRTOC 2021 competition.

Item URL in elib:https://elib.dlr.de/147155/
Document Type:Article
Title:OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulation
AuthorsInstitution or Email of AuthorsAuthor's ORCID iD
Liu, ZiyuanAlibaba AI LabsUNSPECIFIED
Liu, WeiAlibaba AI LabsUNSPECIFIED
Xiang, FanboUC San DiegoUNSPECIFIED
Gou, MinghaoAlibaba AI LabsUNSPECIFIED
Xin, SongyanUniversity of EdinburghUNSPECIFIED
Roa Garzon, Máximo AlejandroMaximo.Roa (at) dlr.dehttps://orcid.org/0000-0003-1708-4223
Calli, BerkWorcester Polytechnic InstituteUNSPECIFIED
Sun, YuUniversity of South FloridaUNSPECIFIED
Tan, PingAlibaba AI LabsUNSPECIFIED
Date:January 2022
Journal or Publication Title:IEEE Robotics and Automation Letters
Refereed publication:Yes
Open Access:No
Gold Open Access:No
In ISI Web of Science:Yes
DOI :10.1109/LRA.2021.3129136
Page Range:pp. 486-493
Publisher:IEEE - Institute of Electrical and Electronics Engineers
Keywords:benchmark, robotic manipulation, robotic competition
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Autonomy & Dexterity [RO]
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013) > Autonomy and Teleoperation
Deposited By: Roa Garzon, Dr. Máximo Alejandro
Deposited On:10 Dec 2021 00:10
Last Modified:10 Dec 2021 00:10

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Copyright © 2008-2017 German Aerospace Center (DLR). All rights reserved.