research-article

Open access

A comprehensive study of autonomous vehicle bugs

Authors:

Sumaya Almanee,

and Qi Alfred ChenAuthors Info & Claims

ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering

Pages 385 - 396

https://doi.org/10.1145/3377811.3380397

Published: 01 October 2020 Publication History

Abstract

Self-driving cars, or Autonomous Vehicles (AVs), are increasingly becoming an integral part of our daily life. About 50 corporations are actively working on AVs, including large companies such as Google, Ford, and Intel. Some AVs are already operating on public roads, with at least one unfortunate fatality recently on record. As a result, understanding bugs in AVs is critical for ensuring their security, safety, robustness, and correctness. While previous studies have focused on a variety of domains (e.g., numerical software; machine learning; and error-handling, concurrency, and performance bugs) to investigate bug characteristics, AVs have not been studied in a similar manner. Recently, two software systems for AVs, Baidu Apollo and Autoware, have emerged as frontrunners in the open-source community and have been used by large companies and governments (e.g., Lincoln, Volvo, Ford, Intel, Hitachi, LG, and the US Department of Transportation). From these two leading AV software systems, this paper describes our investigation of 16,851 commits and 499 AV bugs and introduces our classification of those bugs into 13 root causes, 20 bug symptoms, and 18 categories of software components those bugs often affect. We identify 16 major findings from our study and draw broader lessons from them to guide the research community towards future directions in software bug detection, localization, and repair.

References

[1]

August 2019. 46 Corporations Working On Autonomous Vehicles. https://www.cbinsights.com/research/autonomous-driverless-vehicles-corporations-list/.

[2]

August 2019. Apollo Cyber RT. https://github.com/ApolloAuto/apollo/tree/master/cyber.

[3]

August 2019. Autoware: Open-source software for urban autonomous driving. https://github.com/CPFL/Autoware.

[4]

August 2019. Baidu Apollo: An open autonomous driving platform. http://apollo.auto/.

[5]

August 2019. Baidu hits the gas on autonomous vehicles with Volvo and Ford deals. https://techcrunch.com/2018/11/01/baidu-volvo-ford-autonomous-driving/.

[6]

August 2019. Baidu starts mass production of autonomous buses. https://www.dw.com/en/baidu-starts-mass-production-of-autonomous-buses/a-44525629.

[7]

August 2019. Bazel. https://bazel.build/.

[8]

August 2019. CMake. https://cmake.org/.

[9]

August 2019. A Comprehensive Study of Autonomous Vehicle Bugs - Artifact Website. http://tiny.cc/cps_bug_analysis.

[10]

August 2019. Fatal Tesla Crash Exposes Gap In Automaker's Use Of Car Data. https://www.forbes.com/sites/alanohnsman/2018/04/16/tesla-autopilot-fatal-crash-waze-hazard-alerts/#7bb735fb5572.

[11]

August 2019. Github: The CARMA platform. https://github.com/usdot-fhwa-stol/CARMAPlatform.

[12]

August 2019. Self-Driving Tesla Was Involved in Fatal Crash, U.S. Says. https://www.nytimes.com/2016/07/01/business/self-driving-tesla-fatal-crash-investigation.html.

[13]

August 2019. Self-Driving Uber Car Kills Pedestrian in Arizona, Where Robots Roam. https://www.nytimes.com/2018/03/19/technology/uber-driverless-fatality.html.

[14]

August 2019. Tesla: Autopilot was on during deadly Mountain View crash. https://www.mercurynews.com/2018/03/30/tesla-autopilot-was-on-during-deadly-mountain-view-crash/.

[15]

August 2019. Tesla driver dies in first fatal crash while using autopilot mode. https://www.theguardian.com/technology/2016/jun/30/tesla-autopilot-death-self-driving-car-elon-musk.

[16]

August 2019. The 18 Companies Most Likely to Get Self-driving Dars on the Road First. https://www.businessinsider.com/the-companies-most-likely-to-get-driverless-cars-on-the-road-first-2017-4.

[17]

August 2019. There are some scary similarities between Tesla's deadly crashes linked to Autopilot. https://qz.com/783009/the-scary-similarities-between-teslas-tsla-deadly-autopilot-crashes/.

[18]

August 2019. Two Years On, A Father Is Still Fighting Tesla Over Autopilot And His Son's Fatal Crash. https://jalopnik.com/two-years-on-a-father-is-still-fighting-tesla-over-aut-1823189786.

[19]

August 2019. Uber Self-Driving Car Crash: What Really Happened. https://www.forbes.com/sites/meriameberboucha/2018/05/28/uber-self-driving-car-crash-what-really-happened.

[20]

August 2019. Udacity: Self-Driving Fundamentals: Featuring Apollo. https://www.udacity.com/course/self-driving-car-fundamentals-featuring-apollo-ud0419.

[21]

August 2019. USDOT: The CARMA platform. https://highways.dot.gov/research/research-programs/operations/CARMA.

[22]

August 2019. Waymo Launches Self-driving car Service Waymo One. https://techcrunch.com/2018/12/05/waymo-launches-self-driving-car-service-waymo-one.

[23]

August 2019. Waymo's autonomous cars have driven 8 million miles on public roads. https://www.theverge.com/2018/7/20/17595968/waymo-self-driving-cars-8-million-miles-testing.

[24]

August 2019. You can take a ride in a self-driving Lyft during CES. https://www.theverge.com/2018/1/2/16841090/lyft-aptiv-self-driving-car-ces-2018.

[25]

Karl J Åström and Björn Wittenmark. 2013. Adaptive control. Courier Corporation.

[26]

Peter Biber and Wolfgang Straßer. 2003. The normal distributions transform: A new approach to laser scan matching. In Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003)(Cat. No. 03CH37453), Vol. 3. IEEE, 2743--2748.

[27]

N. Cacho, E. A. Barbosa, J. Araujo, F. Pranto, A. Garcia, T. Cesar, E. Soares, A. Cassio, T. Filipe, and I. Garcia. 2014. How Does Exception Handling Behavior Evolve? An Exploratory Study in Java and C# Applications. In 2014 IEEE International Conference on Software Maintenance and Evolution. 31--40.

Digital Library

[28]

Yulong Cao, Chaowei Xiao, Dawei Yang, Jing Fang, Ruigang Yang, Mingyan Liu, and Bo Li. 2019. Adversarial Objects Against LiDAR-Based Autonomous Driving Systems. arXiv preprint arXiv:1907.05418 (2019).

[29]

R. Coelho, L. Almeida, G. Gousios, and A. v Deursen. 2015. Unveiling Exception Handling Bug Hazards in Android Based on GitHub and Google Code Issues. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. 134--145.

[30]

SAE On-Road Automated Vehicle Standards Committee et al. 2014. Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. SAE Standard J 3016 (2014), 1--16.

[31]

Melvin E Conway. 1968. How do committees invent. Datamation 14, 4 (1968), 28--31.

[32]

Anthony Di Franco, Hui Guo, and Cindy Rubio-Gonzalez. 2017. A Comprehensive Study of Real-world Numerical Bug Characteristics. In Proceedings of the 32Nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2017). IEEE Press, Piscataway, NJ, USA, 509--519. http://dl.acm.org/citation.cfm?id=3155562.3155627 event-place: Urbana-Champaign, IL, USA.

[33]

Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, Tadayoshi Kohno, and Dawn Song. 2018. Physical Adversarial Examples for Object Detectors. In USENIX Workshop on Offensive Technologies (WOOT).

[34]

Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. 2017. Robust physicalworld attacks on deep learning models. arXiv preprint arXiv:1707.08945 (2017).

[35]

Carlos E Garcia, David M Prett, and Manfred Morari. 1989. Model predictive control: theory and practice - a survey. Automatica 25, 3 (1989), 335--348.

Digital Library

[36]

Ali Ghanbari, Samuel Benton, and Lingming Zhang. 2019. Practical program repair via bytecode mutation. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM, 19--30.

Digital Library

[37]

Dirk Holz, Alexandru E Ichim, Federico Tombari, Radu B Rusu, and Sven Behnke. 2015. Registration with the point cloud library: A modular framework for aligning in 3-D. IEEE Robotics & Automation Magazine 22, 4 (2015), 110--124.

[38]

Md Johirul Islam, Giang Nguyen, Rangeet Pan, and Hridesh Rajan. 2019. A Comprehensive Study on Deep Learning Bug Characteristics. arXiv:1906.01388 [cs] (June 2019). http://arxiv.org/abs/1906.01388 arXiv: 1906.01388.

[39]

Guoliang Jin, Linhai Song, Xiaoming Shi, Joel Scherpelz, and Shan Lu. 2012. Understanding and Detecting Real-world Performance Bugs. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '12). ACM, New York, NY, USA, 77--88. event-place: Beijing, China.

Digital Library

[40]

Tanakorn Leesatapornwongsa, Jeffrey F. Lukman, Shan Lu, and Haryadi S. Gunawi. 2016. TaxDC: A Taxonomy of Non-Deterministic Concurrency Bugs in Datacenter Distributed Systems. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '16). ACM, New York, NY, USA, 517--530. event-place: Atlanta, Georgia, USA.

Digital Library

[41]

Shan Lu, Soyeon Park, Eunsoo Seo, and Yuanyuan Zhou. 2008. Learning from Mistakes: A Comprehensive Study on Real World Concurrency Bug Characteristics. In Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XIII). ACM, New York, NY, USA, 329--339. event-place: Seattle, WA, USA.

Digital Library

[42]

Manish Motwani, Sandhya Sankaranarayanan, René Just, and Yuriy Brun. 2018. Do automated program repair techniques repair hard and important bugs? Empirical Software Engineering 23, 5 (01 Oct 2018), 2901--2947.

Digital Library

[43]

Nvidia. 2010. CUDA Programming guide.

[44]

Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y Ng. 2009. ROS: an open-source Robot Operating System. In ICRA workshop on open source software, Vol. 3. Kobe, Japan, 5.

[45]

Carolyn B. Seaman, Forrest Shull, Myrna Regardie, Denis Elbert, Raimund L. Feldmann, Yuepu Guo, and Sally Godfrey. 2008. Defect Categorization: Making Use of a Decade of Widely Varying Historical Data. In Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '08). ACM, New York, NY, USA, 149--157.

Digital Library

[46]

Marija Selakovic and Michael Pradel. 2016. Performance Issues and Optimizations in JavaScript: An Empirical Study. In Proceedings of the 38th International Conference on Software Engineering (ICSE '16). ACM, New York, NY, USA, 61--72. event-place: Austin, Texas.

Digital Library

[47]

F. Thung, S. Wang, D. Lo, and L. Jiang. 2012. An Empirical Study of Bugs in Machine Learning Systems. In 2012 IEEE 23rd International Symposium on Software Reliability Engineering. 271--280.

Digital Library

[48]

Yuchi Tian, Kexin Pei, Suman Jana, and Baishakhi Ray. 2018. Deeptest: Automated Testing of Deep-neural-network-driven Autonomous Cars. In International Conference on Software Engineering (ICSE).

Digital Library

[49]

Yuchi Tian and Baishakhi Ray. 2017. Automatically Diagnosing and Repairing Error Handling Bugs in C. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2017). ACM, New York, NY, USA, 752--762. event-place: Paderborn, Germany.

Digital Library

[50]

Bogdan Vasilescu, Yue Yu, Huaimin Wang, Premkumar Devanbu, and Vladimir Filkov. 2015. Quality and productivity outcomes relating to continuous integration in GitHub. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 805--816.

Digital Library

[51]

Zhiyuan Wan, David Lo, Xin Xia, and Liang Cai. 2017. Bug characteristics in blockchain systems: a large-scale empirical study. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 413--424.

Digital Library

[52]

Greg Welch and Gary Bishop. 1995. An introduction to the Kalman Filter. (1995).

[53]

X. Xia, L. Bao, D. Lo, and S. Li. 2016. "Automated Debugging Considered Harmful" Considered Harmful: A User Study Revisiting the Usefulness of Spectra-Based Fault Localization Techniques with Professionals Using Real Bugs from Large Systems. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). 267--278.

[54]

Mengshi Zhang, Yuqun Zhang, Lingming Zhang, Cong Liu, and Sarfraz Khurshid. 2018. DeepRoad: GAN-based Metamorphic Testing and Input Validation Framework for Autonomous Driving Systems. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE 2018). ACM, New York, NY, USA, 132--142.

Digital Library

[55]

Yuhao Zhang, Yifan Chen, Shing-Chi Cheung, Yingfei Xiong, and Lu Zhang. 2018. An Empirical Study on TensorFlow Program Bugs. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2018). ACM, New York, NY, USA, 129--140. event-place: Amsterdam, Netherlands.

Digital Library

[56]

Yue Zhao, Hong Zhu, Ruigang Liang, Qintao Shen, Shengzhi Zhang, and Kai Chen. 2018. Seeing isn't Believing: Practical Adversarial Attack Against Object Detectors. arXiv preprint arXiv:1812.10217 (2018).

[57]

Hucheng Zhou, Jian-Guang Lou, Hongyu Zhang, Haibo Lin, Haoxiang Lin, and Tingting Qin. 2015. An empirical study on quality issues of production big data platform. In Proceedings of the 37th International Conference on Software Engineering-Volume 2. IEEE Press, 17--26.

Digital Library

Cited By

Jiang MJiang JWu TMa ZLuo XZhou Y(2024)Understanding Vulnerability Inducing Commits of the Linux KernelACM Transactions on Software Engineering and Methodology10.1145/3672452Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672452
Chen YHuai YLi SHong CGarcia J(2024)Misconfiguration Software Testing for Failure Emergence in Autonomous Driving SystemsProceedings of the ACM on Software Engineering10.1145/36607921:FSE(1913-1936)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660792
Hassan MSalvador JSantu SRahman A(2024)State Reconciliation Defects in Infrastructure as CodeProceedings of the ACM on Software Engineering10.1145/36607901:FSE(1865-1888)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660790
Show More Cited By

Recommendations

Motion planning of autonomous vehicles in a non-autonomous vehicle environment without speed lanes

Planning is one of the key problems for autonomous vehicles operating in road scenarios. Present planning algorithms operate with the assumption that traffic is organised in predefined speed lanes, which makes it impossible to allow autonomous vehicles ...
An Empirical Study on Critical Blocking Bugs
ICPC '20: Proceedings of the 28th International Conference on Program Comprehension

Blocking bugs are a severe type of bugs that prevent other bugs from being fixed. As software becomes increasingly complex and large, blocking bugs occur in many large-scale software, especially in software ecosystems. Blocking bugs may have a high ...
A Macroscopic Study on Dedicated Highway Lanes for Autonomous Vehicles
Computational Science – ICCS 2019
Abstract
The introduction of Autonomous Vehicles (AVs) will have far-reaching effects on road traffic in cities and on highways. The implementation of Automated Highway Systems (AHS), possibly with a dedicated lane only for AVs, is believed to be a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering

June 2020

1640 pages

ISBN:9781450371216

DOI:10.1145/3377811

General Chairs:
Gregg Rothermel
North Carolina State University
,
Doo-Hwan Bae
KAIST, South Korea

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

KIISE: Korean Institute of Information Scientists and Engineers
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
National Science Foundation

Conference

ICSE '20

Sponsor:

SIGSOFT

ICSE '20: 42nd International Conference on Software Engineering

June 27 - July 19, 2020

Seoul, South Korea

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

65
Total Citations
View Citations
2,699
Total Downloads

Downloads (Last 12 months)863
Downloads (Last 6 weeks)85

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jiang MJiang JWu TMa ZLuo XZhou Y(2024)Understanding Vulnerability Inducing Commits of the Linux KernelACM Transactions on Software Engineering and Methodology10.1145/3672452Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672452
Chen YHuai YLi SHong CGarcia J(2024)Misconfiguration Software Testing for Failure Emergence in Autonomous Driving SystemsProceedings of the ACM on Software Engineering10.1145/36607921:FSE(1913-1936)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660792
Hassan MSalvador JSantu SRahman A(2024)State Reconciliation Defects in Infrastructure as CodeProceedings of the ACM on Software Engineering10.1145/36607901:FSE(1865-1888)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660790
Birchler CMohammed TRani PNechita TKehrer TPanichella S(2024)How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?Proceedings of the ACM on Software Engineering10.1145/36437681:FSE(929-950)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643768
De Santana TNeto PDe Almeida EAhmed I(2024)Bug Analysis in Jupyter Notebook Projects: An Empirical StudyACM Transactions on Software Engineering and Methodology10.1145/364153933:4(1-34)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3641539
Sun HPoskitt CSun YSun JChen YRoychoudhury APaiva AAbreu RStorey M(2024)ACAV: A Framework for Automatic Causality Analysis in Autonomous Vehicle Accident RecordingsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639175(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639175
Liu DJiang HGuo SChen YQiao L(2024)What's Wrong With Low-Code Development Platforms? An Empirical Study of Low-Code Development Platform BugsIEEE Transactions on Reliability10.1109/TR.2023.329500973:1(695-709)Online publication date: Mar-2024
https://doi.org/10.1109/TR.2023.3295009
Poddar SM MP RN P(2024)A comprehensive Study on Security Threats in Autonomous Vehicles: Safeguarding the Future2024 1st International Conference on Cognitive, Green and Ubiquitous Computing (IC-CGU)10.1109/IC-CGU58078.2024.10530760(1-6)Online publication date: 1-Mar-2024
https://doi.org/10.1109/IC-CGU58078.2024.10530760
Meher JBiswas SMall R(2024)Deep learning-based software bug classificationInformation and Software Technology10.1016/j.infsof.2023.107350166:COnline publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107350
Song YXie XXu B(2024)When debugging encounters artificial intelligence: state of the art and open challengesScience China Information Sciences10.1007/s11432-022-3803-967:4Online publication date: 21-Feb-2024
https://doi.org/10.1007/s11432-022-3803-9
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents