Material Classification through Knocking and Grasping by Learning of Structure-Borne Sound under Changing Acoustic Conditions

Abstract

Structure-borne sound is an interesting sensory modality for inferring contact information in robotics due to comparably cheaply available sensor hardware, the possibility to integrate it into existing systems with little effort, and due to the richness of information in the acquired signals. In this work we investigate whether its is feasible to fit a robotic system with piezo acoustics sensorics in order to infer on properties about objects in the workspace of the robot during contact events. In contrast to existing works regarding object and material identification by evaluating sound the challenge in our experimental setup is that the sensor is integrated in the structure of the robot, hence, the measured audio signal are not only governed by the acoustic properties of the objects we try to identify but are also strongly influenced by the changing resonance properties of the robot due to its kinematic configuration and the ego-noise during operation. Therefore, we investigate whether it is possible to learn a classifier that is invariant and robust to these configuration dependent changes in the acquired audio signals. We exemplarily show the feasibility of this approach for contact inference in a material classification experiment and compare the performance of a deep learning classifier to several baseline machine learning methods. We found that a representation learning approach using a deep neural network shows the highest invariance to the changing acoustics properties and outperforms the baseline methods in our experiments. The results encourage the further investigation of the uses of structure-borne sound in robotic applications.