Enhancing Safety and Reliability of Object Detection in Aerial Imagery through Explainable AI

Abstract

Object detection in aerial imagery plays a critical role in safety-critical applications including disaster response, surveillance, and autonomous systems, yet modern deep learning detectors operate as black boxes, providing limited insight into their decision-making processes. This lack of interpretability presents significant challenges for building trustworthy AI systems where understanding why a model makes specific predictions is as important as achieving high accuracy. This project addresses this gap by systematically investigating explainable AI (XAI) techniques for aerial object detection and developing a novel method that overcomes fundamental limitations of existing approaches. Working with the EAGLE dataset containing 215,986 annotated vehicles across 748 high-resolution aerial images, we trained YOLOv11l, achieving mAP50 of 0.78. We conducted a comprehensive evaluation of both black-box XAI methods (D-RISE, D-CLOSE, SODEx) and white-box methods (GradCAM, HiResCAM) adapted for object detection. Our analysis revealed an unavoidable trade-off: black-box methods achieve faithful explanations but require 1,000–5,000 forward passes per explanation, while white-box methods offer single-pass efficiency but produce spatially inaccurate explanations due to sparse gradients and naive upsampling that ignore true receptive fields. To overcome this limitation, we propose Receptive-Field-Based HiResCAM, which explicitly reconstructs each grid cell’s receptive field through input-image gradient computation, accounting for complex architectural elements including Feature Pyramid Networks and multi-scale detection heads. Quantitative evaluation using insertion–deletion metrics demonstrates that our method achieves faithfulness scores of 0.589–0.868, matching or exceeding black-box methods while being an order of magnitude faster computationally. The generated explanations reveal that the detector relies primarily on vehicle roofs, shadows, and contextual information, providing actionable insights for systematic model improvement. This work makes several key contributions: (1) the first comprehensive evaluation of XAI techniques specifically for small object detection in aerial imagery, (2) a novel explanation method that breaks the conventional efficiency–faithfulness trade-off through principled receptive field reconstruction, (3) demonstration that white-box-level efficiency and black-box-level faithfulness are simultaneously achievable, and (4) practical insights enabling interpretable, trustworthy object detection systems suitable for mission-critical deployments.