Implementation of a 3D Hand Pose Estimation System

Abstract

The variety of means for human-computer interaction has experienced a surge in recent years and currently covers several methods for haptic communication and voice control. These technologies have proven to be a vital tool for information exchange and are already a fundamental part of daily human interaction. However, they are mainly restricted to a two dimensional input and the interpretation of formulated commands. To overcome these limitations, several approaches of 3D hand pose detection systems were made that often imply computation intensive models or large databases with synthetic training sets. Therefore, this thesis presents a 3D hand pose estimation system that provides real-time processing at low computational cost. The first parts describes methods for the hand-background segmentation and the tracking of specific feature points, which include the center of the palm and the fingertips. In the second part, a novel approach is proposed for inferring the position of finger base joints and numbering the correlated fingertip-joint clusters. This method clearly distinguishes from other approaches, as it deduces the joint location from the relative configuration of finger clusters to the center of the palm, omitting the need for complex models or databases. Finally, all obtained information is combined to calculate the angles at each joint and to drive a 3D simulation model, which illustrates the imitated user hand pose. Results presented in the evaluation show that this approach is well suited for hand pose estimation, as it achieves high accuracy in detection and additionally enables real-time interaction. Precision and stability can further be improved by upgrading the utilized camera to a system with higher resolution. This modification also allows considering multiple feature points for a more differentiated angle calculation. Overall, the user feedback was very positive and all test participants mentioned an incline of their success rate over time.