Conception and assessment of semantic feature descriptors for Earth Observation images

Abstract

The volume of civil high resolution Earth Observation (EO) images has steeply increased during the past decade due to numerous advances in airborne and spaceborne imaging technologies and has already leveraged a number of new applications. On the other hand, the large quantity of available images has extremely increased the challenge of exploring and understanding the full content of the images (i.e., their semantics). Therefore, the development of new image mining systems providing satisfactory results with reasonable computational effort became highly demanded. The existing EO image mining systems are usually based on extracted image features provided by various feature descriptors which can represent either pixel level patterns or the higher level semantics of images. Thus, developing feature descriptors which are able to represent the content of images relevant to the users' requirements helps to improve the accuracy and efficiency of image mining systems. As a consequence, this dissertation introduces new approaches based on Latent Dirichlet Allocation (LDA), a topic model for low and high level image feature descriptions. Moreover, the dissertation proposes novel methods based on LDA and information theory for evaluating various image feature descriptors independent of their application case. Since users usually evaluate image mining results based on their semantics, we conducted user studies for assessing the issues such as the sensory and the semantic gaps which affect the user acceptance of the results. Furthermore, this dissertation shows the importance of prior knowledge about the semantic structure of images in shortening the semantic gap between users and computers. All corresponding experiments are conducted on multispectral and SAR (airborne and space-borne) images; the results are validated by employing standard classification and clustering methods (e.g., SVM and k-means) in order to be comparable to previously obtained results in our discipline. The results demonstrate that by using higher level feature descriptors, the user acceptance of image mining results increases because the images are described by their semantic content. Furthermore, the results show that an evaluation of the feature descriptors regardless of their application allows us to generalize the evaluation outcomes to various applications. In addition, our studies and experiments indicate that the sensory and the semantic gaps should not be overlooked due to their high impact upon the user acceptance of image mining results. Finally, our analyses show that exploring the space of image features leverages an understanding of the image semantics.