Using Street-level imagery and multi-task deep learning for multi-hazard risk related building characterization

Abstract

Accurate building characterization is a key component of multi-hazard risk analysis. Collecting such information for large areas is often highly cost and time intensive. Especially in developing countries, where reference data is only scarcely available, automatized methods for comprehensive building inventory creation are highly demanded. In recent years, more and more areas are captured by publicly available street-level imagery such as Google Street View. Several studies of various research fields have aimed to extract high-level information from this massive data set using state-of-the-art Deep Convolutional Neural Networks. In many cases, however, target variables are learned by separate deep learning models, disregarding highly valuable information encoded in inter-class relationships and prolonging model training time. This study quantitatively evaluates the potential of multi-task Learning techniques to further optimize building characterization workflows. A data set containing nearly 30,000 Street View images collected in Santiago de Chile is populated with key structural building components. Multiple preprocessing algorithms to address class imbalance in multi-task data sets are proposed and evaluated. Analysis of single-task and multi-task neural network performance is carried out within a comparable experimental setting. The best performing model reaches an overall accuracy of 82.27 % and a F 1 score of 82.29 %. Inference results of multi-task networks indicate significant improvements in classification accuracy of most tasks while drastically reducing training time. Further, a novel neural network architecture, incorporating class dependency modeling is developed and tested. The proposed model is capable of outperforming baseline multi-task models in the majority of the addressed tasks. The deployed data balancing strategies, however, do not influence model performance significantly. This thesis therefore marks a solid basis for further studies in the domain of building expose mapping and the evaluation of new deep learning techniques.