Validation of Advanced Surface Movement Guidance and Control Systems (A-SMGCS): Lessons learnt from EMMA2

Abstract

E-OCVM gives a comprehensible structure in shape of its six sub-steps which facilitates planning of validation studies for both laymen and experts. In the same way E-OCVM’s levels of maturity are more helpful and user friendly than Technology Readiness Levels in determining the status of a new concept; however good advice about how to find the appropriate tools for each maturity level are lacking. The sketchiness and novelty of concepts with levels V1/V2 should lead to measure “Operational Feasibility” but not “Operational Improvements” in terms of Efficiency, Capacity, Safety, Human Factors issues etc. It is unlikely to expect meaningful or even statistically significant effects e.g. for increased capacity or enhanced situation awareness, if the system is too unclear or maybe even too complex or simply too novel for the user. Operational Feasibility consists of acceptance and usability of the new system by tomorrow’s user. It should be measured for example with a tailor-made questionnaire consisting of items directly derived from the operational concept. Hence the number of baseline scenarios may be reduced as far as possible because scenarios with today’s concepts won’t shed too much light in the acceptance of new systems. Given a transformation of a new concept to more mature levels like V3/V4, additional measurements are recommended to be applied to test finally Operational Improvements. Such kind of tests should only be conducted in shape of real-time simulations: Effects of increased efficiency or decreased workload can only be measured if an accurate designing of test scenarios took place in sub-step 2.5. It has to be guaranteed that baseline scenarios make the results of a new applied system comparable. Here the use of standard questionnaires like NASA-TLX or simple scales like ISA is suitable to measure operators’ reaction to new systems. For experiments in a complex and increasingly cooperative ATM the need for distributed- or coupled simulation setups is getting more and more relevant. The efficient use of such an experimental environment for validation requires the provision of standardised ways to connect simulators and exchange simulation data. To improve interoperability between validation platforms the EUROCAE WG81 is therefore developing the required standards and recommendation.