Deep transfer learning on street-level imagery for classification of seismic building types in Lima, Peru

Abstract

Comprehensive exposure models for seismic risk assessment require accurate building inventories in the endangered areas. The learning capabilities of Deep Learning (DL) can be combined with street-level imagery to categorize buildings in an automated way. Since the training of a DL model requires large amounts of data, in this thesis, a transfer learning approach will be employed to adapt an already trained model to a new study area and a different image dataset while minimizing the labeling requirements. The used model was trained by Aravena Pelizari, et al. (2021) with Google Street View (GSV) images in Santiago de Chile and will be adapted to the Peruvian capital Lima and to street-level imagery of the open-source platform Mapillary. Three data-driven active learning (AL) strategies are designed and implemented with a pre-labeled pool of images: an initial cluster-based sampling with subsequent margin sampling and two additional augmentation strategies exploring the impact of either the most similar source domain images or secondly the addition of high-confidence semi-labeled target domain images. The methods could achieve an improvement of the F1 accuracy score from 0.31 to 0.67 with a comparatively small amount of labeled images. Though the methods did all perform similarly, the initial clustering and the semi-labeling of additional target domain images were the most promising approaches. The class accuracies indicate that the class differences between the domains could be learned at least partly quite successfully, while the performance of the data-driven AL methods was presumably limited by the noisiness of the dataset. The methods provided promising first results and could be further improved with diversity-based batch sampling as well as an extension of the semi-supervised learning approach.