An Overview of Multimodal Remote Sensing Data Fusion: From Image to Feature, from Shallow to Deep

Abstract

With the ever-growing availability of different remote sens-ing (RS) products from both satellite and airborne platforms,simultaneous processing and interpretation of multimodal RSdata have shown increasing significance in the RS field. Dif-ferent resolutions, contexts, and sensors of multimodal RSdata enable the identification and recognition of the materialslying on the earth’s surface at a more accurate level by de-scribing the same object from different points of the view. Asa result, the topic on multimodal RS data fusion has graduallyemerged as a hotspot research direction in recent years.This paper aims at presenting an overview of multimodalRS data fusion in several mainstream applications, which canbe roughly categorized by 1) image pansharpening, 2) hyper-spectral and multispectral image fusion, 3) multimodal fea-ture learning, and (4) crossmodal feature learning. For eachtopic, we will briefly describe what is the to-be-addressed re-search problem related to multimodal RS data fusion and givethe representative and state-of-the-art models from shallow todeep perspectives.