AI-Based Classification of Disaster-Related Images: A Comparative Study of Models

Abstract

Natural disasters have become an increasingly severe threat. This study evaluates the reliability of image classification across multiple deep learning models in the context of natural disaster detection, using a subset of manually validated images derived from the GDELT dataset. Due to the absence of ground-truth labels and challenges in data quality, a smaller, curated sample was employed to ensure analytical validity. The evaluated models include EfficientNet-B1, ResNet-101, OpenCLIP, and CoCa. Results indicate that EfficientNet-B1 and ResNet-101—particularly when utilizing MEDIC’s pretrained weights—achieved consistent and reliable performance, especially in distinguishing between disaster and non-disaster imagery. In contrast, OpenCLIP and CoCa exhibited lower classification accuracy, with CoCa performing weakest, primarily due to difficulties in interpreting abstract disaster categories and additional uncertainty introduced through semantic textual similarity. Identified sources of error include inconsistencies in image labeling, sampling biases, and ambiguities within both statistical and semantic evaluation procedures. Despite these limitations, the study highlights critical differences in model behavior and reliability, emphasizing the need for specialized fine-tuning when applying general-purpose vision-language models to disaster recognition tasks.