Utilizing CNNs for classification and uncertainty quantification for 15 families of European fly pollinators

Kurzfassung

Pollination is vital for biodiversity and food security, yet monitoring pollinators remains challenging. While bees and butterflies have been extensively studied, flies (Diptera) also provide crucial pollination services. However, their classification is difficult due to subtle morphological differences and the limited availability of image datasets. Traditional monitoring methods are costly, time-consuming, and dependent on expert knowledge. Advances in deep learning enable automation in pollinator classification, providing a scalable and efficient alternative. This study explores the use of Convolutional Neural Networks (CNNs) for classifying 15 European pollinating fly families, with a focus on uncertainty quantification to improve prediction reliability. We examine the performance of ResNet18, MobileNetV3, and EfficientNetB4 architectures and evaluate the impact of image cropping on classification accuracy and model confidence. Our dataset consists of 29,374 images sourced from the Global Biodiversity Information Facility (GBIF). Images include full-frame representations and expert-cropped images focusing solely on the fly specimen. A stratified approach was used to ensure balanced representation across the 15 families, with 60% of the images allocated for training, and 20% each for validation and testing. Three CNN architectures, ResNet18, MobileNetV3, and EfficientNetB4, were trained on both full-frame and cropped images. MobileNetV3 (5.4M parameters) was selected for efficiency, ResNet18 (11.3M parameters) for robustness, and EfficientNetB4 (17.6M parameters) for accuracy. To quantify classification uncertainty, we implemented Monte Carlo based uncertainty estimation. Aleatoric uncertainty, stemming from inherent data variability, was assessed using Test-Time Augmentation (TTA). Epistemic uncertainty, reflecting model limitations, was captured through Test-Time Dropout (TTD). Evaluation metrics included overall accuracy (OA), Kappa scores, and mean confidence levels. EfficientNetB4 achieved the highest OA: 90.35% (full-frame) and 95.61% (cropped). ResNet18 reached 88.58% (full-frame) and 93.16% (cropped), while MobileNetV3 scored 88.78% (full-frame) and 93.87% (cropped). Cropping images improved classification accuracy by approximately 5% across all models, demonstrating the importance of targeted image preprocessing for pollinator identification.