Semantic segmentation of remote sensing images with sparse annotations

Abstract

Training convolutional neural networks (CNNs) for very high-resolution images requires a large quantity of high-quality pixel-level annotations, which is extremelylabor-intensive and time-consuming to produce. Moreover, professional photograph interpreters might have to be involved in guaranteeing the correctness of annotations. To alleviate such a burden, we propose a framework for semantic segmentation of aerial images based on incomplete annotations, where annotatorsare asked to label a few pixels with easy-to-draw scribbles. To exploit these sparse scribbled annotations, we propose theFEature and Spatial relaTional regulArization (FESTA) method to complement the supervised task with an unsupervised learn-ing signal that accounts for neighborhood structures both inspatial and feature terms. For the evaluation of our frame-work, we perform experiments on two remote sensing image segmentation data sets involving aerial and satellite imagery, respectively. Experimental results demonstrate that the exploitation of sparse annotations can significantly reduce labeling costs, while the proposed method can help improve the performance of semantic segmentation when training on such annotations. The sparse labels and codes are publicly available for reproducibility purposes.