Development and validation of a construct-based situational judgment test for pilot selection

Kurzfassung

Assessment Centers (ACs) are widely used but resource-intensive procedures for assessing social competencies in personnel selection. Situational Judgment Tests (SJTs) represent an established and more efficient simulation-oriented method for predicting professional success. When developed in a construct-oriented manner, they enable a valid assessment of job-relevant characteristics. In the present study, a construct-based SJT consisting of 12 situations was developed to assess leadership and teamwork for use in the selection of aviation personnel. A scoring key was derived from effectiveness ratings provided by N = 13 experts (aviation psychologists). In a second step, the test was analyzed with regard to its suitability as a selection instrument in the context of a multi-stage selection procedure. In the first selection stage, N = 172 applicants for the profession of pilot completed various cognitive and knowledge tests as well as a personality questionnaire, and the newly developed construct-based SJT. In the second selection stage, N = 82 applicants, who had passed stage one, participated in an AC consisting of various simulation-based procedures, assessing dimensions of leadership, cooperation, and communication (among others). A subsample completed a work sample test (flight simulator task, N = 41) in the third selection stage and a final interview (N = 33). The SJT results correlated significantly with different ratings of the AC observers: positive relationships were found for communication, teamwork, and leadership. Furthermore, the SJT-scores showed positive relations with the flight simulator performance and the final interview. Additionally, applicants who passed the cognitive test battery (stage 1) scored significantly higher on the SJT than those who failed. This confirms previous research that found positive relationships between knowledge-based SJTs and cognitive abilities, as SJTs require cognitive processing. Moreover, only two marginal relationships were found between personality scales and the SJT results. Overall, the observed associations provide first evidence for hypothesis-consistent convergent relationships of the newly developed construct-based SJT with the existing selection procedures. The finding suggests that the SJT validly captures relevant behavioral competencies and represents a promising and efficient pre-selection-tool for aviation personnel selection.