A Multi-Spectral and Multi-Angle 3-D Convolutional Neural Network for the Classification of ZY-3 Satellite Images over Urban Areas

Abstract

The recent availability of high-resolution multiview ZY-3 satellite images, with angular information, can provide an opportunity to capture 3-D structural features for classification. In high-resolution image classification over urban areas, objects with diverse vertical structures make urban landscape more heterogeneous in 3-D space and consequently can make the classification challenging. In this article, a novel multiangle gray-level cooccurrence tensor feature is proposed based on the multiview bands of the ZY-3 imagery, namely, GLCMMA-T. The GLCMMA-T feature captures the distributions of the gray-level spatial variation under different viewing angles, which can depict the 3-D textures and structures of urban objects. The spectral and GLCMMA-T tensor features are interpreted by two 3-D convolutional neural network (CNN) streams and then concatenated as the input to the fully connected layer. This novel multispectral and multiangle 3-D convolutional neural network (M²-3-DCNN) combines the spectral and angular information, and the fused feature has the potential to provide a comprehensive description of urban objects with complex vertical structures. The experimental results on ZY-3 multiview images from four test areas indicate that the proposed method can significantly improve the classification accuracy when compared with several state-of-the-art multiangle features and deep-learning-based image classification methods.