Deliverable D4.4 - User Evaluation Methods

Kurzfassung

As higher levels of automated vehicles (AVs) start to enter the market, it is important to understand how these technologies will be used by the general population, what effects they will have on the ordinary driver, and how this human-technology interaction influences traffic flow and road safety. Since AVs are deployed in mixed traffic environments, which include pedestrians, cyclists, and other drivers, understanding how interactions of the AV with these actors influence our traffic system is also important. Consequently, the user sub-project (SP6) of Hi-Drive will conduct empirical studies to develop a firm understanding of user behaviour, expectations, and limitations when interacting with AVs, as drivers/onboard users, external road users, and remote operators. The current deliverable provides an overview of the methods and measures that will be used for all evaluation areas within SP6, and the research questions that will be addressed using these methods. It provides a summary of the current state-of-the-art in relation to measuring user interactions with AVs, highlighting the importance of the project-level research questions for understanding the experiences of onboard users, external road users, and remote operators. A multi-modal and interdisciplinary approach is taken to investigate road user behaviours and experiences during interactions with AVs, with multiple, complementary methods used for data collection. Specifically, 15 different simulator studies, nine test track studies, and 12 real-world driving studies are planned or completed. These will incorporate a wide range of use cases relating to the comfort and motion sickness of onboard users, driver state monitoring, and external road user behaviours. Across these experimental studies, data will be collected from over 1,000 ordinary drivers, pedestrians, and cyclists, along with data from 40 safety drivers and 35 professional engineers acting as remote operators. The combination of a range of methods will allow us to establish whether similar patterns of results are obtained in simulator and real-world environments, while the data collected through lab-based experiments can be used to inform knowledge when real-world studies are not possible, due either to safety concerns or the absence of advanced AVs. These studies will help the design of future systems and ensure optimum research effectiveness. A wide range of objective measures will be used to capture participant behaviours when interacting with AVs. For onboard users, these will include measures of vehicle control after a takeover request, including hands on wheel or brake reaction time. User state during and after takeover can be measured with physiological metrics such as skin conductance, or eye- tracking-based measures such as pupil dilation and direction of gaze. Finally, performance in non-driving related tasks (NDRTs) can inform how drivers divide their attention between secondary tasks and driving, and what effect this has on safe resumption of control. Measures of external road user behaviour will include the percentage and time of crossings made by pedestrians; cyclists’ yielding behaviours at intersections; and the accepted time gap of other drivers when interacting with an AV. Finally, measures of remote operator performance will include eye tracking and cognitive demand metrics, along with measures of performance on primary and secondary task elements, e.g., time to accept the primary task, and performance in NDRTs. Subjective data from questionnaires, interviews, and focus groups will be used to supplement the data collected through experimental research. This will allow us to draw conclusions on road users’ experiences of automation and their attitudes towards the various automated systems under investigation. The development of a set of Common Questionnaires will enable comparison across studies, although any responses obtained will be context specific, which must be taken into account when comparing across sites. The Hi-Drive Global Survey will focus on exploring both onboard and other road user expectations towards different levels of AVs. In particular, the survey investigates what AV capabilities are required to promote acceptance and use by potential customers. This will allow us to understand what factors influence the uptake of the automated driving (AD) functions (ADFs), and the related technology Enablers being developed within the Hi-Drive project (SP2). In addition, large- scale questionnaires investigating propensity towards motion sickness in various European populations will allow us to understand at-risk groups for carsickness during AD, also informing us about the motion factors and driving styles that will lead to increased risk of sickness. Information about users’ attitudes towards AVs, the factors affecting their comfort, and the prevalence of motion sickness will be sought across six different questionnaire studies, from approximately 20,000 unique questionnaire respondents, and at least 8 different countries. Proposed questions include an understanding of users’ acceptance of ADFs and human-machine interfaces (HMI), along with ratings of perceived safety, trust, and comprehension of AV communication, as well as an overview of driver and remote operator situation awareness and task loads. Finally, this deliverable provides a detailed overview of the specific studies which will be used to address each of the 44 medium-level project-based research questions on users (outlined in Hi-Drive Deliverable 4.1 Research Questions). The studies are grouped according to their main research topic and provide an evaluation plan for each of the four work packages of the User Sub-project (SP6): user acceptance and awareness (WP6.3), user comfort (WP6.4), driver monitoring (WP6.5), and other road users (WP6.6). As outlined in this deliverable, many of the studies provide a multi-pronged approach to user interactions with AV, allowing an exploration of how different research contexts, scenarios, and methodologies can be used to address a particular research question. The relationships between the findings obtained in these studies will be summarized at the end of the project in Deliverables D6.1 (User acceptance and awareness results); D6.2 (Human-like driving and user comfort), D6.3 (User monitoring and related HMI), and D6.4 (Interactions with other road users)