Investigating the Sim-to-Real Generalizability of YOLO Object Detection Models

Abstract

In machine learning, access to real-world data can be a limiting factor, creating the need to understand the full implications of training a machine learning model on synthetic data, and deploying it in a real setting. One component of this issue is the sim-to-real generalizability of the model, characterized by the sim-to-real gap, a frequently encountered performance drop when testing on real versus synthetic data. This work investigates the YOLO family of object detection models on their ability to generalize across domains regarding model iteration, size, and release date. Our experiments show that the models display a sim-to-real gap while the influence of size and model recency on performance is not apparent on our visually simple dataset. We furthermore carefully examine several factors that partly explain how the gap arises and also investigate the connection between generalizability and performance.