Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., and Yoo, S. The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering 41, 5 (May 2015), 507--525.

Digital Library

Google Scholar

[3]

Brown, J., Zhou, Z.Q., and Chow, Y.-W. Metamorphic testing of navigation software: A pilot study with Google Maps. In Proceedings of the 51st Annual Hawaii International Conference on System Sciences (Big Island, HI, Jan. 3--6, 2018) 5687--5696; http://hdl.handle.net/10125/50602

Crossref

Google Scholar

[4]

Chen, T.Y., Kuo, F.-C., Liu, H., Poon, P.-L., Towey, D., Tse, T.H., and Zhou, Z.Q. Metamorphic testing: A review of challenges and opportunities. ACM Computing Surveys 51, 1 (Jan. 2018), 4:1--4:27.

Digital Library

Google Scholar

[5]

Chen, T.Y., Kuo, F.-C., Ma, W., Susilo, W., Towey, D., Voas, J., and Zhou, Z.Q. Metamorphic testing for cybersecurity. Computer 49, 6 (June 2016), 48--55.

Digital Library

Google Scholar

[6]

Chen, T.Y., Tse, T.H., and Zhou, Z.Q. Fault-based testing without the need of oracles. Information and Software Technology 45, 1 (2003), 1--9.

Digital Library

Google Scholar

[7]

Donaldson, A.F., Evrard, H., Lascu, A., and Thomson, P. Automated testing of graphics shader compilers. Proceedings of the ACM on Programming Languages 1 (2017), 93:1--93:29.

Digital Library

Google Scholar

[8]

Jarman, D.C., Zhou, Z.Q., and Chen, T.Y. Metamorphic testing for Adobe data analytics software. In Proceedings of the IEEE/ACM Second International Workshop on Metamorphic Testing, in conjunction with the 39th International Conference on Software Engineering (Buenos Aires, Argentina, May 22). IEEE, 2017. 21--27

Digital Library

Google Scholar

[9]

Kanewala, U., Pullum, L.L., Segura, S., Towey, D., and Zhou, Z.Q. Message from the workshop chairs. In Proceedings of the IEEE/ACM First International Workshop on Metamorphic Testing, in conjunction with the 38th International Conference on Software Engineering (Austin, TX, May 16). ACM Press, New York, 2016.

Google Scholar

[10]

Le, V., Afshari, M., and Su, Z. Compiler validation via equivalence modulo inputs. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (Edinburgh, U.K., June 9--11). ACM Press, New York, 2014, 216--226.

Digital Library

Google Scholar

[11]

Lee, D. Sensor firm Velodyne 'baffled' by Uber self-driving death. BBC News (Mar. 23, 2018); http://www.bbc.com/news/technology-43523286

Google Scholar

[12]

Levin, S. Uber crash shows 'catastrophic failure' of self-driving technology, experts say. The Guardian (Mar. 23, 2018); https://www.theguardian.com/technology/2018/mar/22/self-driving-car-uber-death-woman-failure-fatal-crash-arizona

Google Scholar

[13]

Lindvall, M., Ganesan, D., Árdal, R., and Wiegand, R.E. Metamorphic model-based testing applied on NASA DAT --- An experience report. In Proceedings of the 37th IEEE/ACM International Conference on Software Engineering (Firenze, Italy, May 16--24). IEEE, 2015, 129--138.

Digital Library

Google Scholar

[14]

Lindvall, M., Porter, A., Magnusson, G., and Schulze, C. Metamorphic model-based testing of autonomous systems. In Proceedings of the Second IEEE/ACM International Workshop on Metamorphic Testing, in conjunction with the 39th International Conference on Software Engineering (Buenos Aires, Argentina, May 22). IEEE, 2017, 35--41.

Digital Library

Google Scholar

[15]

Ohnsman, A. LiDAR maker Velodyne 'baffled' by self-driving Uber's failure to avoid pedestrian. Forbes (Mar. 23, 2018); https://www.forbes.com/sites/alanohnsman/2018/03/23/lidar-maker-velodyne-baffled-by-self-driving-ubers-failure-to-avoid-pedestrian

Google Scholar

[16]

Posky, M. LiDAR supplier defends hardware, blames Uber for fatal crash. The Truth About Cars (Mar. 23, 2018); http://www.thetruthaboutcars.com/2018/03/lidar-supplier-blames-uber/

Google Scholar

[17]

Regehr, J. Finding Compiler Bugs by Removing Dead Code. Blog, June 20, 2014; http://blog.regehr.org/archives/1161

Google Scholar

[18]

Segura, S., Fraser, G., Sanchez, A.B., and Ruiz-Cortés, A. A survey on metamorphic testing. IEEE Transactions on Software Engineering 42, 9 (Sept. 2016), 805--824.

Crossref

Google Scholar

[19]

Segura, S. and Zhou, Z.Q. Metamorphic testing: Introduction and applications. ACM SIGSOFT webinar, Sept. 27, 2017; https://event.on24.com/wcc/r/1451736/8B5B5925E82FC9807CF83C84834A6F3D

Google Scholar

[20]

Segura, S. and Zhou, Z.Q. Metamorphic testing 20 years later: A hands-on introduction. In Proceedings of the 40th IEEE/ACM International Conference on Software Engineering (Gothenburg, Sweden, May 27--June 3, 2018). ACM Press, New York, 2018.

Digital Library

Google Scholar

[21]

Tian, Y., Pei, K., Jana, S., and Ray, B. DeepTest: Automated testing of deep neural network-driven autonomous cars. In Proceedings of the 40th IEEE/ACM International Conference on Software Engineering (Gothenburg, Sweden, May 27--June 3, 2018). ACM Press, New York, 2018.

Digital Library

Google Scholar

[22]

Vassilev, A. and Celi, C. Avoiding cyberspace catastrophes through smarter testing. Computer 47, 10 (Oct. 2014), 102--106.

Digital Library

Google Scholar

[23]

Velodyne, Velodyne's HDL-64E: A High-Definition LiDAR Sensor for 3-D Applications, White Paper, 2007; https://www.velodynelidar.com/

Google Scholar

[24]

Zhou, Z.Q., Towey, D., Poon, P.-L., and Tse, T.H. Introduction to the special issue on test oracles. Journal of Systems and Software 136 (Feb. 2018), 187

Crossref

Google Scholar

[25]

Zhou, Z.Q., Xiang, S., and Chen, T.Y. Metamorphic testing for software quality assessment: A study of search engines. IEEE Transactions on Software Engineering 42, 3 (Mar. 2016), 264--284.

Digital Library

Google Scholar

Cited By

View all

Tang SZhang ZZhou JLei LZhou YXue YFilkov VRay BZhou M(2024)LeGEND: A Top-Down Approach to Scenario Generation of Autonomous Driving Systems Assisted by Large Language ModelsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695520(1497-1508)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695520
Crespo-Rodriguez VNeelofar Aleti ASaadatmand MLonetti FBudnik CLi JGuerriero A(2024)PAFOT: A Position-Based Approach for Finding Optimal Tests of Autonomous VehiclesProceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024)10.1145/3644032.3644457(159-170)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3644032.3644457
Duque-Torres ARoychoudhury APaiva AAbreu RStorey M(2024)Selecting and Constraining Metamorphic RelationsProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3639781(212-216)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639478.3639781
Show More Cited By

Index Terms

Metamorphic testing of driverless cars

Recommendations

Metamorphic Testing: A Review of Challenges and Opportunities

Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and ...
Metamorphic Fuzz Testing of Autonomous Vehicles
ICSEW'20: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops

Driving simulation is the primary approach for testing the software components of autonomous vehicles. This paper presents an automated testing method, termed metamorphic fuzz testing (MFT), in the context of simulation testing of autonomous driving. ...
Feedback-Directed Metamorphic Testing
Over the past decade, metamorphic testing has gained rapidly increasing attention from both academia and industry, particularly thanks to its high efficacy on revealing real-life software faults in a wide variety of application domains. On the basis of a ...

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 62, Issue 3

March 2019

109 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/3314328

Editor:
Andrew A. Chien
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 February 2019

Published in CACM Volume 62, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Popular
Refereed

Funding Sources

Australian Research Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

147
Total Citations
View Citations
4,341
Total Downloads

Downloads (Last 12 months)406
Downloads (Last 6 weeks)65

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Tang SZhang ZZhou JLei LZhou YXue YFilkov VRay BZhou M(2024)LeGEND: A Top-Down Approach to Scenario Generation of Autonomous Driving Systems Assisted by Large Language ModelsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695520(1497-1508)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695520
Crespo-Rodriguez VNeelofar Aleti ASaadatmand MLonetti FBudnik CLi JGuerriero A(2024)PAFOT: A Position-Based Approach for Finding Optimal Tests of Autonomous VehiclesProceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024)10.1145/3644032.3644457(159-170)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3644032.3644457
Duque-Torres ARoychoudhury APaiva AAbreu RStorey M(2024)Selecting and Constraining Metamorphic RelationsProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3639781(212-216)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639478.3639781
Rehman FUmbreen SRehman M(2024)MetaCDP: Metamorphic Testing for Quality Assurance of Containerized Data Pipelines2024 IEEE Cloud Summit10.1109/Cloud-Summit61220.2024.00029(135-142)Online publication date: 27-Jun-2024
https://doi.org/10.1109/Cloud-Summit61220.2024.00029
Zhang YTowey DPike MZhou ZChen T(2024)Enhancing ADS Testing: An Open Educational Resource for Metamorphic Testing2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00029(138-147)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00029
Nazir SFranke Kleist VSurendra NPeace A(2024)Informating and Automating in Systems Design: Extending the Zuboffian View of Technology Ready-to-Use vs. Technology Unintuitive-to-UseJournal of Global Information Technology Management10.1080/1097198X.2024.237255427:3(200-221)Online publication date: 4-Jul-2024
https://doi.org/10.1080/1097198X.2024.2372554
Harwood S(2024)A cybersystemic view of autonomous weapon systems (AWS)Technological Forecasting and Social Change10.1016/j.techfore.2024.123514205(123514)Online publication date: Aug-2024
https://doi.org/10.1016/j.techfore.2024.123514
Guo AFeng YCheng YChen Z(2024)Semantic-guided fuzzing for virtual testing of autonomous driving systemsJournal of Systems and Software10.1016/j.jss.2024.112017212:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112017
Wang HShao WSun CYang KCao DLi J(2024)A Survey on an Emerging Safety Challenge for Autonomous Vehicles: Safety of the Intended FunctionalityEngineering10.1016/j.eng.2023.10.01133(17-34)Online publication date: Feb-2024
https://doi.org/10.1016/j.eng.2023.10.011
Lin SChen FXi LWang GXi RSun YZhu H(2024)TM-fuzzer: fuzzing autonomous driving systems through traffic managementAutomated Software Engineering10.1007/s10515-024-00461-w31:2Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10515-024-00461-w
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Metamorphic Testing: A Review of Challenges and Opportunities

Metamorphic Fuzz Testing of Autonomous Vehicles

Feedback-Directed Metamorphic Testing

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Digital Edition

Magazine Site

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations