Representation Learning with Weak Labels in Remote Sensing

Abstract

Semantic understanding of the Earth's surface from satellite and aerial imagery supports many critical societal and scientific applications. In recent years, DL has substantially advanced the semantic interpretation of RS data. Yet, its success relies heavily on the availability of large-scale, accurately annotated data. In real-world settings, this dependence is difficult to satisfy, particularly for label-intensive tasks like semantic segmentation and multi-label classification. As an alternative, weak supervision sources--such as annotations from OSM and various LULC products--offer a more scalable, cost-efficient form of supervision. However, these weak labels inevitably contain noise. Such label noise introduces bias into representation learning when used as supervision for model training.

Building on these observations, this dissertation examines how weak labels can be systematically leveraged as a foundation to learn robust and transferable representations for RS semantic understanding. It investigates how different weak supervision types shape representation learning and how different learning strategies give rise to representations with distinct properties. To this end, the work is organized around two interrelated dimensions. From the supervision perspective, it considers increasingly complex forms of weak labels, ranging from incompleteness-dominant noisy annotations, to mixed and more intricate single-source noisy labels, and further to multi-source weak supervision with divergent and inconsistent class definitions. From the learning perspective, it advances from label-noise-robust optimization, to weakly supervised representation pretraining, and ultimately to weakly supervised open-vocabulary semantic learning.