Semantic Annotation in Earth Observation Based on Active Learning

Abstract

As the data acquisition capabilities of Earth Observation (EO) satellites have been improved significantly, a large amount of high resolution images are downlinked continuously to ground stations. The data volume increases rapidly beyond the users' capability to access the information content of the data. Thus, interactive systems that allow fast indexing of high resolution images based on image content are urgently needed. In this paper, we present an interactive learning system for semantic annotation and content mining at patch level. It mainly comprises four components: primitive feature extraction including both spatial and temporal features, relevance feedback based on active learning, a Human Machine Communication (HMC) interface, and data visualization. To overcome the shortage of training samples and to speed up the convergence, active learning is employed in this system. Two core components of active learning are the classifier training using already labeled image patches, and the sample selection strategy which selects the most informative samples for manual labeling. These two components work alternatively, significantly reducing the labeling effort and achieving fast indexing. In addition, our data visualization is particularly designed for multi-temporal and multi-sensor image indexing, where efficient visualization plays a critical role. The system is applicable to both optical and SAR images. It can index patches and it can also discover temporal patterns in satellite image time series. Three typical study cases are included to show its wide variety of use in EO applications.