Bayesian Inference on LSTM-based Object Recognition from Tactile and Kinesthetic Information

Abstract

Recent advances in the field of intelligent robotic manipulation pursue providing robotic hands with touch sensitivity. Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations). This letter focuses on multimodal object recognition and proposes a Bayesian methodology for the optimal fusion of tactile- and kinesthetic-based classification results. The procedure is as follows: a three-finger actuated gripper with an integrated high-resolution tactile sensor performs squeeze-and-release Exploratory Procedures (EPs). The tactile images and kinesthetic information acquired using angular sensors on the finger joints constitute the time-series datasets of interest. Each temporal dataset is fed to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify in-hand objects. The LSTMs provide an estimation of the posterior probability of each object given the corresponding measurements, which after fusion allows to estimate the object through maximum a posteriori (MAP). An experiment with 24-classes is carried out to evaluate and compare the performance of the fused, tactile, and kinesthetic perception systems. The results show that the fusion-based algorithm improves capabilities for object-recognition.