Bridging the Sim-to-Real Gap for Monocular 3D Object Detection via Foundation Model-Guided Unsupervised Domain Adaption

Abstract

For autonomous driving, monocular 3D object detection provides a scalable perception solution using inexpensive cameras, while synthetic data offers perfectly annotated and diverse training data. The primary obstacle to combining these technologies is the significant sim-to-real domain gap, where models fail to generalize from simulation to reality. To bridge this gap, this thesis introduces and evaluates a novel Unsupervised Domain Adaptation framework that uses foundation models to improve a Mean Teacher self-training process. The framework leverages semantic segmentation masks to filter noisy pseudo-labels, thereby stabilizing the teacher-student learning dynamic. I investigate both semantic and geometric guidance, finding that while semantic filtering is highly effective, current metric depth estimation is not yet suitable for this task. Through a series of experiments, the proposed semantic-guided framework demonstrates a significant performance improvement of over 34% compared to a standard UDA baseline, confirming the viability of this approach.