Learning from Simulation to Improve Robotic Sensing

Abstract

The ability of the robot to sense its environment is essential for its autonomous operation. Precise object detection and pose estimation, derived from the sensing capabilities, is crucial for the autonomy of any robotic system. Deep learning and neural networks have already revolutionized this field. The research community has successfully developed powerful deep learning algorithms using neural networks for faster and accurate object detection and pose estimation. However, these methods require a large amount of annotated dataset for the training of neural networks. Preparation of such high quality annotated dataset is expensive and time-consuming. An alternative method is to make use of the easily available simulation data instead of real data for training. Because of the domain gap between the simulation and the real data, networks trained with simulation data fails to perform well on real images. This thesis explores the possibility of generating realistic looking images using Generative adversarial network (GAN), in order to overcome the existing domain gap between the simulated and real images. Recent research on GANs has shown that these networks are capable of producing photo-realistic images and is a promising new area of the research field for producing realistic synthetic images. Inspired from one of the variants of GAN known as CycleGAN, the proposed work explores the possibility of generating realistic images from simulated images using modified CycleGAN architecture. Using the T-LESS and YCB dataset as the benchmark, the generated images are then evaluated by training an object detector network. The quality of the generated images are measured by the accuracy of the detector and then compared with the real domain images.