Person Activity Recognition for sitting postures using RGB-D data

Kurzfassung

The estimation and analysis of the activity of a person has become an important component in many robotic applications such as Human-Robot Collaboration and assisted living systems. However, many existing approaches lack generalizability due to (a) their sensory set-up or (b) the use of a set of features which might be difficult to extract in a variety of application scenarios. This thesis focuses on the head of a person and its purpose is to create a framework that extracts features from the head that can be used for activity recognition. Key requirements for the framework are (1) its generic usability independent of the particular application/scene, (2) real-time capability, (3) a non-intrusive set-up using a low-priced RGB-D camera and (4) robustness to lighting conditions, partial occlusions and facial expressions. To meet these requirements an extension of the existing method for head pose estimation of [1] was developed. The extending component estimates the 2D elliptical region of interest around the human head in a depth image. The proposed combined method is a frame-by-frame approach and runs in real-time without the need of GPU computations. The method has been trained, tuned and validated in an indoor office environment using existing benchmarking databases as well as a new recorded database, later referred to as DLR FC-PEAR. Additionally, we recorded another database, DLR CAR, in a car environment which exhibits realistic and simulated driving situations. Both databases were recorded with a Kinect v2. These databases were used in the experiments to evaluate the performance of the head features for estimating the visual focus of attention of the subjects. The results showed the applicability of the method in these different use cases as well as its generalizability from office to car. During the experiments it also became apparent that various characteristics of the human (head) have to be reflected in the training stage to guarantee this generalizability. [1] G. Fanelli, M. Dantone, J. Gall, A. Fossati, and L. Van Gool. Random forests for real time 3d face analysis. Int. J. Comput. Vision, 101(3):437–458, February 2013.