Multimodal myocontrol for bimanual manipulation

Abstract

Human hands provide multiple functions like powerful and precise grasping and are part of interpersonal communication. Thus the loss of an upper limb affects everyday life. Intuitively controllable prostheses have the aim to replace the missing functionality. Reliability of the myocontrol system is the key criterion to develop user-oriented prostheses. Simultaneous and proportional control using muscle signals realizes intuitive control. However reliability is still missing. One solution to solve this issue is incremental learning. This approach takes care of unreliable performance by allowing model updates on-demand, for instance whenever the user wants to teach the system or whenever a failure occurred. Thus current incremental learning methods lead to interactive myocontrol; their drawback, however, is the need to stop the prediction and update the model manually. An automatic detection offailures, as reliable as possible, is desirable. By developing a new method called observer model we seek to automatically identify failures of the performance of the myocontrol system. The observer model is fed with multimodal data and according observer’s labels. An offline analysis on training data includes extraction of time- and frequency domain features and classification according to their class affiliation. This classifier is used to detect poor performance of the myocontrol system. Recognition rate is used as a measure for the performance of the classifier; it gives a percentage of concurrence of actual and predicted labels. An experiment modeling daily-life activities was conducted for 11 subjects. We see an average outcome of a 60.3% recognition rate as a proof for the possibility to detect failures of the myocontrol system that leaves space for further improvement. It opens the door to further investigations towards automatic failure detection to aim reliable myocontrol.