research-article

Hands-free but not Eyes-free: A Usability Evaluation of Siri while Driving

Authors:

Helene Høgh Larsen,

Alexander Nuka Scheel,

Toine Bogers, and

Birger LarsenAuthors Info & Claims

CHIIR '20: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval

March 2020

Pages 63 - 72

https://doi.org/10.1145/3343413.3377962

Published: 14 March 2020 Publication History

Abstract

Distractions while driving are a major cause of traffic accidents and chief among these is the use of mobile phones. Driver distractions typically fall into four categories-visual, cognitive, bio-mechanical, and auditory-and different technological solutions have been proposed to address these. Intelligent Personal Assistants (IPAs), such as Siri, is a recent example of such a technological solution that offers the potential for hands-free phone interaction through a voice-controlled interface. IPAs could potentially reduce visual and bio-mechanical distractions if they are usable enough to not increase a driver's cognitive load. We present the results of a controlled experiment with the aim of understanding how the use of Siri while driving compares to manual interaction in terms of usability and distractions. We also tested these two interaction types in the lab in order to understand how the main driving task influences Siri's (perceived) usability. Our study shows that Siri is not ready for every-day use in the car: interacting with Siri while driving is likely to be unsafe for most participants, especially less experienced drivers. Participants were distracted by Siri due to its over-reliance on visual feedback as well as frequent time-outs by Siri when waiting for a response from a driver occupied with the road environment. Speech recognition quality in a noisy car as well as problematic multi-lingual speech recognition in general are other issues that resulted in low usability and more cognitive distractions. While interacting with Siri may be hands-free, it does not provide an eyes-free and distraction-free experience yet.

References

[1]

M. J. Albers. 2011. Tapping as a Measure of Cognitive Load and Website Usability. In Proceedings of the 29th ACM International Conference on Design of Communication. ACM, 25--32.

Digital Library

[2]

Toine Bogers, Ammar Ali Abdelrahim Al-Basri, Claes Ostermann Rytlig, Mads Emil Bak Møller, Mette Juhl Rasmussen, Nikita Katrine Bates Michelsen, and Sara Gerling Jørgensen. 2019. A Study of Usage and Usability of Intelligent Personal Assistants in Denmark. In Proceedings of the 2019 iConference. Springer, 79--90.

[3]

Matthias Bö hmer, B. Hecht, J. Schö ning, A. Krü ger, and G. Bauer. 2011. Falling Asleep with Angry Birds, Facebook and Kindle: A Large Scale Study on Mobile Application Usage. In MobileHCI '11: Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. ACM, 47--56.

[4]

B. Cowan, N. Pantidi, D. Coyle, K. Morrissey, P. Clarke, S. Al-Shehri, D. Earley, and N. Bandeira. 2017. “What can I help you with?”: Infrequent Users' Experiences of Intelligent Personal Assistants. In MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 1--12.

[5]

Danish Road Traffic Accident Investigation Board. 2014. Why do road traffic accidents happen? Technical Report. Retrieved from http://www.hvu.dk/SiteCollectionDocuments/HVUdec14_UK_HvorforSkerUlykkerne.pdf.

[6]

Marco Dozza, Carol A. C. Flannagan, and James R. Sayer. 2015. Real-World Effects of Using a Phone while Driving on Lateral and Longitudinal Control of Vehicles . Journal of Safety Research, Vol. 55 (2015), 81--87.

[7]

Frank A. Drews, Monisha Pasupathi, and David L. Strayer. 2008. Passenger and cell phone conversations in simulated driving. Journal of Experimental Psychology: Applied, Vol. 14, 4 (2008), 392.

[8]

Marta Perez Garcia, Sarita Saffon Lopez, and Hector Donis. 2018. Everybody is Talking about Voice Activated Virtual Assistants, but how are People really Adopting and Using Them? Lessons from a Multi-Country Study. In Proceedings of the 32nd Human Computer Interaction Conference. BCS.

[9]

Ido Guy. 2016. Searching by talking: Analysis of voice queries on mobile web search. In SIGIR '16: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 35--44.

Digital Library

[10]

Joanne L. Harbluk, Y. Ian Noy, Patricia L. Trbovich, and Moshe Eizenman. 2007. An On-Road Assessment of Cognitive Distraction: Impacts on Drivers' Visual Behaviour and Braking Performance . Accident Analysis and Prevention, Vol. 39 (2007), 372--379.

[11]

S. G. Hart and L. E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research . Advances in Psychology, Vol. 52 (1988), 139--183.

[12]

Morgan Harvey and Matthew Pointon. 2017a. Perceptions of the Effect of Fragmented Attention on Mobile Web Search Tasks. In CHIIR '17: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. ACM, New York, NY, USA, 293--296.

[13]

Morgan Harvey and Matthew Pointon. 2017b. Searching on the Go: The Effects of Fragmented Attention on Mobile Web Search Tasks. In SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, USA, 155--164.

[14]

Morgan Harvey and Matthew Pointon. 2018. Noisy Signals: Understanding the Impact of Auditory Distraction on Web Search Tasks. In CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval. ACM, New York, NY, USA, 241--244.

[15]

Marika Hoedemaeker and Mark Neerincx. 2007. Attuning In-Car User Interfaces to the Momentary Cognitive Load. In International Conference on Foundations of Augmented Cognition. Springer, 286--293.

[16]

Yoko Ishigami and Raymond M. Klein. 2009. Is a Hands-Free Phone Safer than a Handheld Phone? Journal of Safety Research, Vol. 40, 2 (2009), 157--164.

[17]

Brett Kinsella. 2018. Over Half of Smartphone Owners Use Voice Assistants, Siri Leads the Pack . https://voicebot.ai/2018/04/03/over-half-of-smartphone-owners-use-voice-assistants-siri-leads-the-pack/. (2018). Last visited September 10, 2018.

[18]

Julia Kiseleva, Kyle Williams, Ahmed Hassan Awadallah, Aidan C. Crook, Imed Zitouni, and Tasos Anastasakos. 2016a. Predicting User Satisfaction with Intelligent Assistants. In SIGIR '16: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, USA, 45--54.

[19]

Julia Kiseleva, Kyle Williams, Jiepu Jiang, Ahmed Hassan Awadallah, Aidan C. Crook, Imed Zitouni, and Tasos Anastasakos. 2016b. Understanding User Satisfaction with Intelligent Assistants. In CHIIR '16: Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval. ACM, New York, NY, USA, 121--130.

[20]

Sheila G. Klauer, Feng Guo, Bruce G. Simons-Morton, Marie Claude Ouimet, Suzanne E. Lee, and Thomas A. Dingus. 2014. Distracted Driving and Risk of Road Crashes among Novice and Experienced Drivers . New England Journal of Medicine, Vol. 370, 1 (2014), 54--59.

[21]

Eric M. Lamberg and Lisa M. Muratori. 2012. Cell Phones Change the Way we Walk . Gait & Posture, Vol. 35, 4 (2012), 688--690.

[22]

Sammy Licence, Robynne Smith, Miranda P. McGuigan, and Conrad P. Earnest. 2015. Gait Pattern Alterations during Walking, Texting and Walking and Texting during Cognitively Distractive Tasks while Negotiating Common Pedestrian Obstacles . PLoS one, Vol. 10, 7 (2015), e0133281.

[23]

Gustavo López, Luis Quesada, and Luis A Guerrero. 2017. Alexa vs. Siri vs. Cortana vs. Google Assistant: A Comparison of Speech-based Natural User Interfaces. In International Conference on Applied Human Factors and Ergonomics. Springer, 241--250.

[24]

E. Luger and A. Sellen. 2016. “Like having a really bad PA”: The gulf between user expectation and experience of conversational agents. In CHI '16. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5286--5297.

[25]

Luis Nunes and Miguel Angel Recarte. 2002. Cognitive Demands of Hands-Free-Phone Conversation while Driving . Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 5, 2 (2002), 133--144.

[26]

Antti Oulasvirta, Sakari Tamminen, Virpi Roto, and Jaana Kuorelahti. 2005. Interaction in 4-second Bursts: The Fragmented Nature of Attentional Resources in Mobile HCI. In CHI '05: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 919--928.

Digital Library

[27]

Oscar Oviedo-Trespalacios. 2018. Getting Away with Texting: Behavioural Adaptation of Drivers Engaging in Visual-Manual Tasks while Driving . Transportation Research Part A: Policy and Practice, Vol. 116 (2018), 112--121.

[28]

T. A. Ranney, W. R. Garrott, and M. J. Goodman. 2001. NHTSA Driver Distraction Research: Past, Present, and Future. Technical Report 2001-06-0177. Society of Automotive Engineers. SAE Technical Paper.

[29]

Jeff Rubin and Dana Chisnell. 2008. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests 2nd ed.). Wiley.

[30]

Zhanna Sarsenbayeva, Niels van Berkel, Eduardo Velloso, Vassilis Kostakos, and Jorge Goncalves. 2018. Effect of Distinct Ambient Noise Types on Mobile Interaction . Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 2, 2, Article 82 (2018), bibinfonumpages23 pages.

Digital Library

[31]

Ahmed Seffah, Mohammad Donyaee, Rex B. Kline, and Harkirat K. Padda. 2006. Usability Measurement and Metrics: A Consolidated Model . Software Quality Journal, Vol. 14, 2 (2006), 159--178.

[32]

Amanda Spink, C. Cole, and M. Waller. 2008. Multitasking Behavior . Annual Review of Information Science and Technology, Vol. 41, 1 (2008), 93--118.

[33]

S. Sternberg. 1966. High-Speed Scanning in Human Memory . Science, Vol. 153, 3736 (1966), 652--654.

[34]

David L. Strayer, Joel M. Cooper, Jonna Turrill, James R. Coleman, and Rachel J. Hopman. 2017. The Smartphone and the Driver's Cognitive Workload: A Comparison of Apple, Google, and Microsoft's Intelligent Personal Assistants . Canadian Journal of Experimental Psychology, Vol. 71, 2 (2017), 93.

[35]

Paul J. Treffner and Rod Barrett. 2004. Hands-Free Mobile Phone Speech while Driving Degrades Coordination and Control . Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 7, 4--5 (2004), 229--246.

Cited By

Weith H(2024)Faire KI-basierte SprachassistentenFair KI-based Voice AssistantsHMD Praxis der Wirtschaftsinformatik10.1365/s40702-024-01047-661:2(537-554)Online publication date: 26-Feb-2024
https://doi.org/10.1365/s40702-024-01047-6
Chen QGong YLu Y(2023)User Experience of Digital Voice Assistant: Conceptualization and MeasurementACM Transactions on Computer-Human Interaction10.1145/362278231:1(1-35)Online publication date: 29-Nov-2023
https://dl.acm.org/doi/10.1145/3622782
Pabst ATabb TStewart KDanielescu A(2023)A Qualitative Study on the Expectations and Concerns Around Voice and Gesture Interactions in VehiclesProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596040(2155-2171)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596040
Show More Cited By

Index Terms

Hands-free but not Eyes-free: A Usability Evaluation of Siri while Driving
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
    2. Interaction paradigms
      1. Natural language interfaces
  2. Ubiquitous and mobile computing
    1. Empirical studies in ubiquitous and mobile computing
2. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
      1. Presentation of retrieval results
    2. Users and interactive retrieval
      1. Search interfaces

Recommendations

Eyes on the road, hands on the wheel: thumb-based interaction techniques for input on steering wheels
GI '07: Proceedings of Graphics Interface 2007

The increasing quantity and complexity of in-vehicle systems creates a demand for user interfaces which are suited to driving. The steering wheel is a common location for the placement of buttons to control navigation, entertainment, and environmental ...
Read More
Four Eyes See More Than Two: Shared Gaze in the Car
Human-Computer Interaction – INTERACT 2015
Abstract
Purposeful collaboration of driver and front-seat passenger can help in demanding driving situations and therefore increase safety. The characteristics of the car, as a context, limit the collaboration possibilities of the driver and front-seat ...
Read More
'Eyes free' in-car assistance: parent and child passenger collaboration during phone calls
MobileHCI '13: Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services

This paper examines routine family car journeys, looking specifically at how passengers assist during a mobile telephone call while the drivers address the competing demands of handling the vehicle, interacting with various artefacts and controls in the ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHIIR '20: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval

March 2020

596 pages

ISBN:9781450368926

DOI:10.1145/3343413

General Chairs:
Heather O'Brien
University of British Columbia, Canada
,
Luanne Freund
University of British Columbia, Canada
,
Program Chairs:
Ioannis Arapakis
Telefonica I+D, Spain
,
Orland Hoeber
University of Regina, Canada
,
Irene Lopatovska
Pratt Institute, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHIIR '20

Sponsor:

CHIIR '20: Conference on Human Information Interaction and Retrieval

March 14 - 18, 2020

Vancouver BC, Canada

Acceptance Rates

Overall Acceptance Rate 55 of 163 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
536
Total Downloads

Downloads (Last 12 months)93
Downloads (Last 6 weeks)7

Other Metrics

View Author Metrics

Citations

Cited By

Weith H(2024)Faire KI-basierte SprachassistentenFair KI-based Voice AssistantsHMD Praxis der Wirtschaftsinformatik10.1365/s40702-024-01047-661:2(537-554)Online publication date: 26-Feb-2024
https://doi.org/10.1365/s40702-024-01047-6
Chen QGong YLu Y(2023)User Experience of Digital Voice Assistant: Conceptualization and MeasurementACM Transactions on Computer-Human Interaction10.1145/362278231:1(1-35)Online publication date: 29-Nov-2023
https://dl.acm.org/doi/10.1145/3622782
Pabst ATabb TStewart KDanielescu A(2023)A Qualitative Study on the Expectations and Concerns Around Voice and Gesture Interactions in VehiclesProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596040(2155-2171)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596040
Ferreira JRodriguez FSantos ADieste OAcuna SJuristo N(2023)Impact of Usability Mechanisms: A Family of Experiments on Efficiency, Effectiveness and User SatisfactionIEEE Transactions on Software Engineering10.1109/TSE.2022.314958649:1(251-267)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TSE.2022.3149586
Tan ZDai NSu YZhang RLi YWu DLi S(2022)Human–Machine Interaction in Intelligent and Connected Vehicles: A Review of Status Quo, Issues, and OpportunitiesIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2021.312721723:9(13954-13975)Online publication date: Sep-2022
https://doi.org/10.1109/TITS.2021.3127217
Saad MZia ARaza MKundi MHaleem M(2022)A Comprehensive Analysis of Healthcare Websites Usability Features, Testing Techniques and IssuesIEEE Access10.1109/ACCESS.2022.319337810(97701-97718)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3193378
Jiang TLiu J(2022)Reflection on future directions: a systematic review of reported limitations and solutions in interactive information retrieval user studiesAslib Journal of Information Management10.1108/AJIM-05-2022-0253Online publication date: 19-Dec-2022
https://doi.org/10.1108/AJIM-05-2022-0253
da Silva TFurtado VFurtado EMendes MAlmeida VSales L(2022)How Do Illiterate People Interact with an Intelligent Voice Assistant?International Journal of Human–Computer Interaction10.1080/10447318.2022.212121940:3(584-602)Online publication date: 22-Sep-2022
https://doi.org/10.1080/10447318.2022.2121219
Sim JJi Y(2022)Investigating the Impact of Mounted Mobile Phones on Driving BehaviorInternational Journal of Human–Computer Interaction10.1080/10447318.2022.209778439:17(3431-3444)Online publication date: 25-Jul-2022
https://doi.org/10.1080/10447318.2022.2097784
Hong DCho C(2022)Factors Affecting Innovation Resistance of Smartphone AI Voice AssistantsInternational Journal of Human–Computer Interaction10.1080/10447318.2022.208089939:13(2557-2572)Online publication date: 14-Jun-2022
https://doi.org/10.1080/10447318.2022.2080899
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents