Model-Based Process-Oriented Human Operator Cognitive Performance Measurement for the Evaluation of Human-Machine Systems.

Kurzfassung

In this thesis, a model-based process-oriented human operator cognitive performance measurement for Human-Machine Systems is developed. Existing measures of human operator performance concentrate on the results of interactions. They provide no information about the decisions leading to the measured performance. Instead, the developed measure allows evaluating the performance during an interaction. According to the evaluation process developed in this thesis, measured interaction sequences are completed by generated interaction sequences, which represent the best available option. By repeating this step for different situations, the reachable performance is determined at any point during interaction. Subsequently, the evaluation of decisions of human operator is based on changes of the reachable performance. By comparing the effects of the available options with the effects of the selected options, the impact of different types of errors and kind of actions on the overall performance can be identified. A human operator cognitive planning model is developed in this thesis to generate the required interaction sequences. This planning model contains a model of the controlled system realized with Coloured Petri Nets and a model of human operators’ behavior. This model is implemented as a set of rules in this thesis and deducted from the objectives and available options. To not tempt human operators to a counterproductive change of their working methods by a comparison with the optimal calculations of the model, the effect of prediction uncertainty is integrated into the model. Consequently, more realistic predictions of human behavior are possible. Finally, the developed model-based process-oriented performance measure is validated with a study. This study demonstrates that the performance measure can not only replace conventional result-oriented performance measures but gives a lot of additional information. The addition benefits of this thesis are the insights made possible by the developed measurement. For example, the available options and their consequences at different times during an interaction as well as the consequences of the human operators’ decisions can be determined. These insights support the goal-oriented improvement and development of assistance. Moreover, the developed measurement enables estimating the benefits of assistance before its implementation. Additionally, human operator can be confronted with the measured performance. On the one hand, this can be used for training purposes. On the other hand, potential Improvements of Human-Machine Systems can be identified by a discussion with human operators about the reasons and consequences of detected erroneous actions.