Detection of Unknown Substances in Operation Environments Using Multispectral Imagery and Autoencoders

Abstract

Autonomous vehicles and robotic systems are increasingly used to perform operations in environments that bear potential risks to humans (e.g. areas affected by natural disasters, warfare, or planetary exploration). One source of danger is the contamination with hazardous substances. In order to improve situational awareness and planning, such substances must be detected using the sensors of the autonomous system. However, training a supervised machine learning model to detect different substances requires a labelled dataset with all potential substances to be known in advance, which is often impracticable. A possible solution for this is to pose an anomaly detection problem where an unsupervised algorithm detects suspicious substances that differ from the normal operation environment.

In this paper we propose SpectrAE, a convolutional autoencoder-based system that processes multispectral imaging data (covering visible to near-infrared ranges) to identify surface anomalies on roads. Unlike traditional detection methods such as gas chromatography and physical sampling that risk contamination and cause operational delays, or laser-based remote sensing techniques that require pre-localisation of potential hot spots, our approach offers near real-time detection capabilities without prior knowledge of specific hazardous substances. The system is trained exclusively on normal road conditions and identifies potential hazards through localised reconstruction loss patterns, generating Areas of Interest for further investigation. Our contributions include a robust end-to-end detection pipeline, comprehensive evaluation of system performance, and a roadmap for future development in this emerging intersection of autonomous systems and crisis response technologies.