research-article

Open access

“Can you rely on me?” Evaluating a Confidence HMI for Cooperative, Automated Driving

Authors:

Jakob Benedikt Peintner,

Andreas RienerAuthors Info & Claims

AutomotiveUI '22: Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Pages 340 - 348

https://doi.org/10.1145/3543174.3546976

Published: 17 September 2022 Publication History

All formats PDF

Abstract

First concepts for cooperative driving already illustrate the potential of the cooperation between human drivers and automated vehicles. However, the main influencing factors that determine an efficient and effective cooperation still need to be investigated. We therefore examined the effects of displaying the vehicle’s intended decision in a critical situation in combination with a confidence value. In a driving simulator study (N=49), the automated vehicle communicated uncertainty in predicting the behavior of a pedestrian and the participants could support the automation by entering their own decision to stop or drive through. The results show that the time until entering the own decision (system override) was longer when a pending system decision was indicated with a confidence value in the HMI. A low confidence value resulted in the longest interaction times. In addition, trust in automation and usability were lower compared to the baseline cooperative HMI without a confidence value.

References

[1]

Matthias Beggiato, Franziska Hartwich, Katja Schleinitz, Josef Krems, Ina Othersen, and Ina Petermann-Stock. 2015. What would drivers like to know during automated driving? Information needs at different levels of automation., 6 pages. https://doi.org/10.13140/RG.2.1.2462.6007

[2]

Johannes Beller, Matthias Heesen, and Mark Vollrath. 2013. Improving the Driver–Automation Interaction: An Approach Using Automation Uncertainty. Human Factors 55, 6 (2013), 1130–1141. https://doi.org/10.1177/0018720813482327 arXiv:https://doi.org/10.1177/0018720813482327PMID: 24745204.

[3]

John Brooke. 1996. SUS – a quick and dirty usability scale(1 ed.). CRC Press, Boca Raton, FL, USA, 189–194.

[4]

Heiner Bubb. 2012. Information Ergonomics. In Information Ergonomics: A theoretical approach and practical experience in transportation, M. Stein and P. Sandl (Eds.). Springer Berlin Heidelberg, Berlin, Germany.

[5]

Weiya Chen, Tetsuo Sawaragi, and Toshihiro Hiraoka. 2020. Driver’s Mental Workload Measurement Concerning Cognitive Channels. In 2020 59th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE). IEEE, Chiang Mai, Thailand, 1458–1463. https://doi.org/10.23919/SICE48898.2020.9240327

[6]

K Christoffersen and D Woods. 2001. How to make automated systems team players. Advances in Human Performance and Cognitive Engineering Research 2 (11 2001), 1–12. https://doi.org/10.1016/S1479-3601(02)02003-9 ISSN: 14793601.

[7]

Simon Danner, Alexander Feierle, Carina Manger, and Klaus Bengler. 2021. Context-Adaptive Availability Notifications for an SAE Level 3 Automation. Multimodal Technologies and Interaction 5, 4 (2021), 14 pages. https://doi.org/10.3390/mti5040016

[8]

Mica R. Endsley and Esin O. Kiris. 1995. The Out-of-the-Loop Performance Problem and Level of Control in Automation. Human Factors: The Journal of the Human Factors and Ergonomics Society 37, 2 (June 1995), 381–394. https://doi.org/10.1518/001872095779064555

[9]

Anna Feldhütter, Christian Gold, Sonja Schneider, and Klaus Bengler. 2017. How the Duration of Automated Driving Influences Take-Over Performance and Gaze Behavior. In Advances in Ergonomic Design of Systems, Products and Processes, Christopher Marc Schlick, Sönke Duckwitz, Frank Flemisch, Martin Frenz, Sinem Kuz, Alexander Mertens, and Susanne Mütze-Niewöhner (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 309–318. https://doi.org/10.1007/978-3-662-53305-5_22

[10]

Paul H Fitts. 1951. Human engineering for an effective air-navigation and traffic-control system.

[11]

Frank Ole Flemisch, Klaus Bengler, Heiner Bubb, Hermann Winner, and Ralph Bruder. 2014. Towards cooperative guidance and control of highly automated vehicles: H-Mode and Conduct-by-Wire. Ergonomics 57, 3 (March 2014), 343–360. https://doi.org/10.1080/00140139.2013.869355

[12]

Christian Gold, Daniel Damböck, Lutz Lorenz, and Klaus Bengler. 2013. “Take over!” How long does it take to get the driver back into the loop?Proceedings of the Human Factors and Ergonomics Society Annual Meeting 57, 1 (Sept. 2013), 1938–1942. https://doi.org/10.1177/1541931213571433

[13]

Chunshi Guo, Chouki Sentouh, Jean-Baptiste Haué, and Jean-Christophe Popieul. 2019. Driver–vehicle cooperation: a hierarchical cooperative control architecture for automated driving systems. Cognition, Technology & Work 21, 4 (2019), 657–670.

Digital Library

[14]

Tobias Hecht, Simon Danner, Alexander Feierle, and Klaus Bengler. 2020. Does a Confidence Level for Automated Driving Time Estimations Improve the Subjective Evaluation of an Automation HMI?Multimodal Technologies and Interaction 4, 3 (2020), 12 pages. https://doi.org/10.3390/mti4030036

[15]

Tove Helldin, Göran Falkman, Maria Riveiro, and Staffan Davidsson. 2013. Presenting System Uncertainty in Automotive UIs for Supporting Trust Calibration in Autonomous Driving. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications(Eindhoven, Netherlands) (AutomotiveUI ’13). Association for Computing Machinery, New York, NY, USA, 210–217. https://doi.org/10.1145/2516540.2516554

Digital Library

[16]

Kai Holländer, Philipp Wintersberger, and Andreas Butz. 2019. Overtrust in External Cues of Automated Vehicles: An Experimental Investigation. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Utrecht, Netherlands) (AutomotiveUI ’19). Association for Computing Machinery, New York, NY, USA, 211–221. https://doi.org/10.1145/3342197.3344528

Digital Library

[17]

G. Klein, D.D. Woods, J.M. Bradshaw, R.R. Hoffman, and P.J. Feltovich. 2004. Ten Challenges for Making Automation a ”Team Player” in Joint Human-Agent Activity. IEEE Intelligent Systems 19, 06 (Nov. 2004), 91–95. https://doi.org/10.1109/MIS.2004.74

Digital Library

[18]

Alexander Kunze, Stephen J. Summerskill, Russell Marshall, and Ashleigh J. Filtness. 2018. Augmented Reality Displays for Communicating Uncertainty Information in Automated Driving. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, Toronto ON Canada, 164–175. https://doi.org/10.1145/3239060.3239074

Digital Library

[19]

Alexander Kunze, Stephen J. Summerskill, Russell Marshall, and Ashleigh J. Filtness. 2019. Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces. Ergonomics 62, 3 (March 2019), 345–360. https://doi.org/10.1080/00140139.2018.1547842

[20]

Alexander Kunze, Stephen J. Summerskill, Russell Marshall, and Ashleigh J. Filtness. 2019. Conveying Uncertainties Using Peripheral Awareness Displays in the Context of Automated Driving. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, Utrecht Netherlands, 329–341. https://doi.org/10.1145/3342197.3344537

Digital Library

[21]

David Large, Gary Burnett, Andrew Morris, Arun Muthumani, and Rebecca Matthias (Bartlett). 2017. Design Implications of Drivers’ Engagement with Secondary Activities During Highly-Automated Driving – A Longitudinal Simulator Study.

[22]

Frederik Naujoks, Katharina Wiedemann, Nadja Schömig, Sebastian Hergeth, and Andreas Keinath. 2019. Towards guidelines and verification methods for automated vehicle HMIs. Transportation Research Part F: Traffic Psychology and Behaviour 60 (2019), 121–136. https://doi.org/10.1016/j.trf.2018.10.012

[23]

On-Road Automated Driving (ORAD) committee. 2018. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. https://doi.org/10.4271/J3016_201806

[24]

Jürgen Pichen, Tanja Stoll, and Martin Baumann. 2021. From SAE-Levels to Cooperative Task Distribution:An Efficient and Usable Way to Deal with System Limitations?. In 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, Leeds United Kingdom, 109–115. https://doi.org/10.1145/3409118.3475127

Digital Library

[25]

Jonas Radlmayr, Christian Gold, Lutz Lorenz, Mehdi Farid, and Klaus Bengler. 2014. How Traffic Situations and Non-Driving Related Tasks Affect the Take-Over Quality in Highly Automated Driving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 58, 1 (Sept. 2014), 2063–2067. https://doi.org/10.1177/1541931214581434

[26]

Natalie Tara Richardson. 2019. Konzeption und Langzeittest der Mensch-Maschine-Schnittstelle für das hochautomatisierte Fahren im Lkw. dissertation. Technical University of Munich.

[27]

Andreas Riener, Myounghoon Jeon, and Ignacio Alvarez. 2021. User Experience Design in the Era of Automated Driving -. Springer Nature, Singapore.

[28]

Clemens Schartmüller and Andreas Riener. 2021. The Driving Experience Lab: Simulating the Automotive Future in a Trailer. In 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Leeds, United Kingdom) (AutomotiveUI ’21 Adjunct). Association for Computing Machinery, New York, NY, USA, 225–228. https://doi.org/10.1145/3473682.3481877

Digital Library

[29]

Clemens Schartmüller, Andreas Riener, and Philipp Wintersberger. 2018. Steer-By-WiFi: Lateral Vehicle Control for Take-Overs with Nomadic Devices. In Adjunct Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Toronto, ON, Canada) (AutomotiveUI ’18). Association for Computing Machinery, New York, NY, USA, 121–126. https://doi.org/10.1145/3239092.3265954

Digital Library

[30]

Nadja Schömig, Katharina Wiedemann, Sebastian Hergeth, Yannick Forster, Jeffrey Muttart, Alexander Eriksson, David Mitropoulos-Rundus, Kevin Grove, Josef Krems, Andreas Keinath, Alexandra Neukum, and Frederik Naujoks. 2020. Checklist for Expert Evaluation of HMIs of Automated Vehicles—Discussions on Its Value and Adaptions of the Method within an Expert Workshop. Information 11, 4 (2020), 16 pages. https://www.mdpi.com/2078-2489/11/4/233

[31]

Sonja Stockert, Natalie Tara Richardson, and Markus Lienkamp. 2015. Driving in an Increasingly Automated World – Approaches to Improve the Driver-automation Interaction. Procedia Manufacturing 3(2015), 2889–2896. https://doi.org/10.1016/j.promfg.2015.07.797 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the Affiliated Conferences, AHFE 2015.

[32]

Wilson P. Tanner and John A. Swets. 1954. A Decision-Making Theory of Visual Detection. Psychological Review 61, 6 (1954), 401–409. https://doi.org/10.1037/h0058700

[33]

Tesla. 2022. Tesla Autopilot. https://www.tesla.com/autopilot Retrieved: 2022-04-21.

[34]

Jinke D. Van Der Laan, Adriaan Heino, and Dick De Waard. 1997. A simple procedure for the assessment of acceptance of advanced transport telematics. Transportation Research Part C: Emerging Technologies 5, 1(1997), 1–10. https://doi.org/10.1016/S0968-090X(96)00025-3

[35]

Marcel Walch, Mark Colley, and Michael Weber. 2019. CooperationCaptcha: On-The-Fly Object Labeling for Highly Automated Vehicles. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–6. https://doi.org/10.1145/3290607.3313022

Digital Library

[36]

Marcel Walch, David Lehr, Mark Colley, and Michael Weber. 09212019. Don’t you see them?. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, Christian P. Janssen, Stella F. Donker, Lewis L. Chuang, and Wendy Ju (Eds.). ACM, New York, NY, USA, 232–237. https://doi.org/10.1145/3349263.3351338

Digital Library

[37]

Marcel Walch, Stacey Li, Ilan Mandel, David Goedicke, Natalie Friedman, and Wendy Ju. 2020. Crosswalk Cooperation: A Phone-Integrated Driver-Vehicle Cooperation Approach to Predict the Crossing Intentions of Pedestrians in Automated Driving. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, Virtual Event DC USA, 74–77. https://doi.org/10.1145/3409251.3411727

Digital Library

[38]

Marcel Walch, Kristin Mühl, Johannes Kraus, Tanja Stoll, Martin Baumann, and Michael Weber. 2017. From Car-Driver-Handovers to Cooperative Interfaces: Visions for Driver–Vehicle Interaction in Automated Driving. In Automotive User Interfaces, Gerrit Meixner and Christian Müller (Eds.). Springer International Publishing, Cham, 273–294. https://doi.org/10.1007/978-3-319-49448-7_10

[39]

Marcel Walch, Marcel Woide, Kristin Mühl, Martin Baumann, and Michael Weber. 09212019. Cooperative Overtaking. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, New York, NY, USA, 144–155. https://doi.org/10.1145/3342197.3344531

Digital Library

[40]

Chao Wang, Matti Krüger, and Christiane B. Wiebel-Herboth. 2020. “Watch out!”: Prediction-Level Intervention for Automated Driving. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, Virtual Event DC USA, 169–180. https://doi.org/10.1145/3409120.3410652

Digital Library

[41]

Christopher Wickens. 2002. Multiple resources and performance prediction. Theoretical Issues in Ergonomic Science 3 (01 2002), 159–177. https://doi.org/10.1080/14639220210123806

[42]

Gesa Wiegand, Kai Holländer, Katharina Rupp, and Heinrich Hussmann. 09212020. The Joy of Collaborating with Highly Automated Vehicles. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, New York, NY, USA, 223–232. https://doi.org/10.1145/3409120.3410643

Digital Library

[43]

John R. Wilson and Andrew Rutherford. 1989. Mental Models: Theory and Application in Human Factors. Human Factors 31, 6 (1989), 617–634. https://doi.org/10.1177/001872088903100601

Digital Library

[44]

Philipp Wintersberger, Clemens Schartmüller, Shadan Shadeghian-Borojeni, Anna-Katharina Frison, and Andreas Riener. 0. Evaluation of Imminent Take-Over Requests With Real Automation on a Test Track. Human Factors 0, 0 (0), 00187208211051435. https://doi.org/10.1177/00187208211051435 arXiv:https://doi.org/10.1177/00187208211051435

[45]

Ingo Wolf. 2016. The Interaction Between Humans and Autonomous Agents. Springer Berlin Heidelberg, Berlin, Heidelberg, 103–124. https://doi.org/10.1007/978-3-662-48847-8_6

[46]

Markus Zimmermann, Stefan Bauer, Niklas Lutteken, Iris M. Rothkirch, and Klaus J. Bengler. 2014. Acting together by mutual control: Evaluation of a multimodal interaction concept for cooperative driving. In 2014 International Conference on Collaboration Technologies and Systems (CTS). IEEE, Minneapolis, MN, USA, 227–235. https://doi.org/10.1109/CTS.2014.6867569

Cited By

Peintner JEscher BDetjen HManger CRiener A(2024)How to Design Human-Vehicle Cooperation for Automated Driving: A Review of Use Cases, Concepts, and InterfacesMultimodal Technologies and Interaction10.3390/mti80300168:3(16)Online publication date: 26-Feb-2024
https://doi.org/10.3390/mti8030016
Stampf ASasalovici MMeinhardt LColley MGiss MRukzio E(2024)Move, Connect, Interact: Introducing a Design Space for Cross-Traffic InteractionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785808:3(1-40)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678580
Manger CPeintner JHoffmann MProbst MWennmacher RRiener A(2023)Providing Explainability in Safety-Critical Automated Driving Situations through Augmented Reality Windshield HMIsAdjunct Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3581961.3609874(174-179)Online publication date: 18-Sep-2023
https://dl.acm.org/doi/10.1145/3581961.3609874
Show More Cited By

Index Terms

“Can you rely on me?” Evaluating a Confidence HMI for Cooperative, Automated Driving
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Communication of Uncertainty Information in Cooperative, Automated Driving: A Comparative Study of Different Modalities
AutomotiveUI '23: Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Cooperation between drivers and automated vehicles requires transparent communication of the automation’s current status. This can be achieved by communicating confidence or certainty in its current perception or decision. We evaluate different sensory ...
Two Methods to Bring Drivers Back Better into the Loop after Automated Driving at SAE Level 3
AutomotiveUI '22: Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Vehicles offering conditional automated driving (SAE Level 3 [1]) are now becoming available worldwide. At this automated driving level, it is possible for drivers to engage in non-driving related activities while the vehicle takes over all lateral and ...
Truck Drivers as Stakeholders in Cooperative Driving
Ambient Intelligence
Abstract
Cooperative driving for trucks has been claimed to bring substantial benefits for society and fleet owners because of better throughput and reduced fuel consumption, but benefits for truck drivers are questionable. While most work on cooperative ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AutomotiveUI '22: Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

September 2022

371 pages

ISBN:9781450394154

DOI:10.1145/3543174

General Chairs:
Yong Gu Ji
Yonsei University, South Korea
,
Myounghoon Jeon
Virginia Tech, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 September 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AutomotiveUI '22

Sponsor:

SIGCHI

AutomotiveUI '22: 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

September 17 - 20, 2022

Seoul, Republic of Korea

Acceptance Rates

Overall Acceptance Rate 248 of 566 submissions, 44%

Upcoming Conference

AutomotiveUI '25

Sponsor:
sigchi

17th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

September 21 - 25, 2025

Brisbane , QLD , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
754
Total Downloads

Downloads (Last 12 months)370
Downloads (Last 6 weeks)54

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Peintner JEscher BDetjen HManger CRiener A(2024)How to Design Human-Vehicle Cooperation for Automated Driving: A Review of Use Cases, Concepts, and InterfacesMultimodal Technologies and Interaction10.3390/mti80300168:3(16)Online publication date: 26-Feb-2024
https://doi.org/10.3390/mti8030016
Stampf ASasalovici MMeinhardt LColley MGiss MRukzio E(2024)Move, Connect, Interact: Introducing a Design Space for Cross-Traffic InteractionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785808:3(1-40)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678580
Manger CPeintner JHoffmann MProbst MWennmacher RRiener A(2023)Providing Explainability in Safety-Critical Automated Driving Situations through Augmented Reality Windshield HMIsAdjunct Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3581961.3609874(174-179)Online publication date: 18-Sep-2023
https://dl.acm.org/doi/10.1145/3581961.3609874
Malve BPeintner JSadeghian SRiener A(2023)“Do You Want to Drive Together?” - A Use Case Analysis on Cooperative, Automated Driving.Adjunct Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3581961.3609871(209-214)Online publication date: 18-Sep-2023
https://dl.acm.org/doi/10.1145/3581961.3609871

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten