Sensor Fusion-Based Learning for the Improvement of Person Segmentation by Means of a Low-Resolution Thermal Infrared Array Sensor

Abstract

Low-resolution thermal infrared array sensors can be used to detect human bodies and motion. Segmentation and deriving features from the segmented shape using such devices remains challenging. For improving and testing segmentation results, a sensor fusion approach using a Kinect sensor can be used to automatically receive ground-truth data. After performing a spatial calibration, experiments were performed to receive data for training and testing. A measure of difference to the ground-truth data is defined as error rate. Probability functions can be derived to determine whether a human is present, appearing or disappearing at a specific pixel. Optimization using Gaussian blur results in shapes ready for segmentation. A machine learning approach that uses conditional random fields on the ground-truth data generated by sensor fusion can be trained to reconstruct the ground-truth data. Testing different models showed that a spatial model that consists of a 4-connected neighborhood achieves better results than a temporal-spatial model.