Generation of Annotated Data Using Pre-Trained Diffusion Models: Alleviating the data pain in the context of architectural facade segmentation

Kurzfassung

The success of modern machine learning methods such as deep learning has broadened its influence on almost all research fields, including architecture and the built world. A common requirement for these data-centric technologies is the availability of data for training. Dataset creation and curation is a time-consuming and error-prone process. Depending on the complexity of annotations needed, this can become a significant bottleneck. Particularly in tasks such as in semantic segmentation, where labelling a single urban image may require several minutes. Segmented data is essential for analysing and developing methodologies for objects in the built world. Yet existing façade segmentation datasets often suffer from inconsistencies, limited size and varying annotation standards. This work proposes an approach to bridge the data gap, using generative models. A pretrained stable diffusion model is finetuned on a custom dataset created with the purpose of providing a wide range of architectural styles and image types. In a subsequent stage, the pretrained model is used to train an ensemble of pixel classifiers to predict class labels for generated images. A data generation pipeline is proposed and prototypically tested, focusing on the generation of segmented data for facade labels.