Exploring Deep Learning for Hazardous Material Detection: A Study in Anomalous Object Identification

Abstract

Autonomous systems increasingly require reliable detection of unknown materials to ensure safety. These pose significant risks to both vehicular traffc and environmental ecosystems, such as chemical spills on roadways that cause vehicle accidents, fuel leaks that contaminate soil and groundwater, or agricultural runoff containing fertilizers and pesticides that affect local water sources. Traditional detection systems, such as gas chromatography and physical sampling, create contamination risks and cause operational delays, while laser-based remote sensing techniques require pre-localization of potential hotspots. To address these critical safety and environmental challenges, this thesis examines how deep learning techniques are utilized to automatically detect hidden dangers without requiring labeled anomaly examples during training. The experiments show that unsupervised and self-supervised deep learning can work for detecting materials and defects. The research develops and evaluates four different deep learning models to address these challenges. We implement two unsupervised autoencoder architectures: a ResNet encoder-based autoencoder and a Vision Transformer (ViT) encoder-based autoencoder. In addition, two self-supervised learning techniques are implemented using SimCLR and Barlow Twins. The experimental evaluation is conducted using two distinct datasets that address different aspects of anomaly detection. The primary dataset consists of multispectral road images that are captured by the German Aerospace Center (DLR) using specialized cameras covering visible to near-infrared wavelengths, specifically designed to identify unknown materials on road surfaces. However, due to the time-intensive nature of multispectral image acquisition and the limited availability of ground truth annotations, the open-source MVTec RGB hazelnut dataset is additionally employed to provide a comprehensive quantitative evaluation. This supplementary dataset enables testing the models’ capability in detecting surface defects with ground truth labels. The autoencoder with the ViT encoder outperforms all other models on both datasets, achieving higher precision, recall, and F1 scores. The ResNet encoder detects most anomalies successfully. However, it occasionally misinterprets normal viitexture variations as actual defects. In addition, Barlow Twins performs better than the SimCLR method for anomaly detection, providing better anomaly localization results. Morphological operations are used to refine the detection results by removing noise and filling gaps. The 2D UMAP visualizations are used to analyze feature representations of the training, test, and validation sets for anomaly detection. In summary, this thesis presents deep learning models that are capable of detecting unknown and unusual patterns, contributing to enhanced road safety and environmental protection by enabling early detection of hazardous substances before they cause traffc accidents or ecological damage.