Towards Explainable Machine Learning from Remote Sensing to Medical Images—Merging Medical and Environmental Data into Public Health Knowledge Maps

Abstract

Both remote sensing and medical fields benefited a lot from the machine learning methods, originally developed for computer vision and multimedia. We investigate the applicability of the same data mining-based machine learning (ML) techniques for exploring the structure of both Earth observation (EO) and medical image data. Support Vector Machine (SVM) is an explainable active learning tool to discover the semantic relations between the EO image content classes, extending this technique further to medical images of various types. The EO image dataset was acquired by multispectral and radar sensors (WorldView-2, Sentinel-2, TerraSAR-X, Sentinel-1, RADARSAT-2, and Gaofen-3) from four different urban areas. In addition, medical images were acquired by camera, microscope, and computed tomography (CT). The methodology has been tested by several experts, and the semantic classification results were checked by either comparing them with reference data or through the feedback given by these experts in the field. The accuracy of the results amounts to 95% for the satellite images and 85% for the medical images. This study opens the pathway to correlate the information extracted from the EO images (e.g., quality-of-life-related environmental data) with that extracted from medical images (e.g., medical imaging disease phenotypes) to obtain geographically refined results in epidemiology.