research-article

Embedded out-of-distribution detection on an autonomous robot platform

Authors:

Daniel Jun Xian Ng,

Zahra Rahiminasab,

Arvind EaswaranAuthors Info & Claims

Destion '21: Proceedings of the Workshop on Design Automation for CPS and IoT

Pages 13 - 18

https://doi.org/10.1145/3445034.3460509

Published: 18 May 2021 Publication History

Abstract

Machine learning (ML) is actively finding its way into modern cyber-physical systems (CPS), many of which are safety-critical real-time systems. It is well known that ML outputs are not reliable when testing data are novel with regards to model training and validation data, i.e., out-of-distribution (OOD) test data. We implement an unsupervised deep neural network-based OOD detector on a real-time embedded autonomous Duckiebot and evaluate detection performance. Our OOD detector produces a success rate of 87.5% for emergency stopping a Duckiebot on a braking test bed we designed. We also provide case analysis on computing resource challenges specific to the Robot Operating System (ROS) middleware on the Duckiebot.

References

[1]

P. Almási, R. Moni, and B. Gyires-Tóth. 2020. Robust Reinforcement Learning-based Autonomous Driving Agent for Simulation and Real World. In 2020 International Joint Conference on Neural Networks (IJCNN). 1--8.

[2]

Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K Varshney, and Dawn Song. 2020. Anomalous Instance Detection in Deep Learning: A Survey. arXiv preprint arXiv:2003.06979 (2020).

[3]

Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. 2020. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11621--11631.

[4]

The Duckietown Foundation. [n.d.]. The Duckiebot Manual - Unit C-4 - Assembling the Duckiebot DB18. https://docs.duckietown.org/daffy/opmanual_duckiebot/out/assembling_duckiebot_db18.html

[5]

The Duckietown Foundation. [n.d.]. The Duckiebot Manual - Unit C-8 - Duckiebot SD Card Initialization. https://docs.duckietown.org/daffy/opmanual_duckiebot/out/setup_duckiebot.html

[6]

Pablo González-Nalda, Ismael Etxeberria-Agiriano, Isidro Calvo, and Mari Carmen Otero. 2017. A modular CPS architecture design based on ROS and Docker. International Journal on Interactive Design and Manufacturing (IJIDeM) 11, 4 (2017), 949--955.

[7]

Xiaozhe Gu and Arvind Easwaran. 2019. Towards safe machine learning for cps: infer uncertainty from training data. In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems. 249--258.

Digital Library

[8]

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2016. beta-vae: Learning basic visual concepts with a constrained variational framework. (2016).

[9]

A. Kalapos, C. Gór, R. Moni, and I. Harmati. 2020. Sim-to-real reinforcement learning applied to end-to-end vehicle control. In 2020 23rd International Symposium on Measurement and Control in Robotics (ISMCR). 1--6.

[10]

Shinpei Kato, Eijiro Takeuchi, Yoshio Ishiguro, Yoshiki Ninomiya, Kazuya Takeda, and Tsuyoshi Hamada. 2015. An open approach to autonomous vehicles. IEEE Micro 35, 6 (2015), 60--68.

Digital Library

[11]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).

[12]

Anis Koubaa. 2017. Robot Operating System (ROS): The Complete Reference (Volume 2) (1st ed.). Springer Publishing Company, Incorporated.

[13]

Dirk Merkel. 2014. Docker: Lightweight Linux Containers for Consistent Development and Deployment. Linux J. 2014, 239, Article 2 (March 2014).

Digital Library

[14]

L. Paull, J. Tani, H. Ahn, J. Alonso-Mora, L. Carlone, M. Cap, Y. F. Chen, C. Choi, J. Dusek, Y. Fang, D. Hoehener, S. Liu, M. Novitzky, I. F. Okuyama, J. Pazis, G. Rosman, V. Varricchio, H. Wang, D. Yershov, H. Zhao, M. Benjamin, C. Carr, M. Zuber, S. Karaman, E. Frazzoli, D. Del Vecchio, D. Rus, J. How, J. Leonard, and A. Censi. 2017. Duckietown: An open, inexpensive and flexible platform for autonomy education and research. In 2017 IEEE International Conference on Robotics and Automation (ICRA). 1497--1504.

Digital Library

[15]

Shreyas Ramakrishna, Zahra Rahiminasab, Arvind Easwaran, and Abhishek Dubey. 2020. Efficient Multi-Class Out-of-Distribution Reasoning for Perception Based Networks: Work-in-Progress. In 2020 International Conference on Embedded Software (EMSOFT). IEEE, 40--42.

[16]

W. Shi, G. Huang, S. Song, Z. Wang, T. Lin, and C. Wu. 2020. Self-Supervised Discovering of Interpretable Features for Reinforcement Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020), 1--1.

[17]

David Tian. [n.d.]. DeepPiCar - Part 4: Autonomous Lane Navigation via OpenCV. https://towardsdatascience.com/deeppicar-part-4-lane-following-via-opencv-737dd9e47c96

[18]

M. Tim, M. Szemenyei, and R. Moni. 2020. Simulation to Real Domain Adaptation for Lane Segmentation. In 2020 23rd International Symposium on Measurement and Control in Robotics (ISMCR). 1--6.

[19]

Kush R Varshney and Homa Alemzadeh. 2017. On the safety of machine learning: Cyber-physical systems, decision sciences, and data products. Big data 5, 3 (2017), 246--255.

[20]

Sachin Vernekar, Ashish Gaurav, Vahdat Abdelzad, Taylor Denouden, Rick Salay, and Krzysztof Czarnecki. 2019. Out-of-distribution detection in classifiers via generation. arXiv preprint arXiv:1910.04241 (2019).

Cited By

Saadati MBalu AChiranjeevi SJubery TSingh ASarkar SSingh AGanapathysubramanian B(2024)Out-of-Distribution Detection Algorithms for Robust Insect ClassificationPlant Phenomics10.34133/plantphenomics.01706Online publication date: 30-Apr-2024
https://doi.org/10.34133/plantphenomics.0170
Das SYuhas MKoh REaswaran A(2024)Interpretable Latent Space for Meteorological Out-of-Distribution Detection via Weak SupervisionACM Transactions on Cyber-Physical Systems10.1145/36512248:2(1-26)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3651224
Joshi HKumar S(2024)Artificial Intelligence Failures in Autonomous Vehicles: Causes, Implications, and PreventionComputer10.1109/MC.2024.344943557:11(18-30)Online publication date: Nov-2024
https://doi.org/10.1109/MC.2024.3449435
Show More Cited By

Embedded out-of-distribution detection on an autonomous robot platform
1. Computer systems organization

Recommendations

An Embedded Computing Platform for Robot
SUTC '08: Proceedings of the 2008 IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (sutc 2008)

As the robotic industry is growing boomingly, the functionalities and system's architecture of robots are more and more complex. The development of robotic application system becomes a time-consuming and difficult task. In this paper, we propose an ...
Embedded Control System Design for Autonomous Navigation Mobile Robot
APCIP '09: Proceedings of the 2009 Asia-Pacific Conference on Information Processing - Volume 01

This paper presents Embedded Based Control System Design for Autonomous Navigation Mobile Robot. A hierarchical control systems that based on multi-processor is given, including the main control system based on the S3C2440, subordinate control system ...
Analysis and Implementation of an Embedded System Platform Based on FreeRTOS and Cortex-M3
ICCIS 2017: Proceedings of the 2017 2nd International Conference on Communication and Information Systems

FreeRTOS is an open-source and portable real-time operating system which is widely used in embedded applications. As embedded system becoming more complex, porting real-time operating system kernel to the hardware platform can shorten the design cycle ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Destion '21: Proceedings of the Workshop on Design Automation for CPS and IoT

May 2021

41 pages

ISBN:9781450383165

DOI:10.1145/3445034

Conference Chair:
Qi Zhu
Northwestern University
,
Program Chairs:
Abhishek Dubey
Vanderbilt University
,
Alessandro Pinto
Raytheon Technologies

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGBED: ACM Special Interest Group on Embedded Systems

In-Cooperation

IEEE Signal Processing Society
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

MoE, Singapore

Conference

CPS-IoT Week '21

Sponsor:

SIGBED

CPS-IoT Week '21: Cyber-Physical Systems and Internet of Things Week 2021

May 18, 2021

Tennessee, Nashville

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
162
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)3

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Saadati MBalu AChiranjeevi SJubery TSingh ASarkar SSingh AGanapathysubramanian B(2024)Out-of-Distribution Detection Algorithms for Robust Insect ClassificationPlant Phenomics10.34133/plantphenomics.01706Online publication date: 30-Apr-2024
https://doi.org/10.34133/plantphenomics.0170
Das SYuhas MKoh REaswaran A(2024)Interpretable Latent Space for Meteorological Out-of-Distribution Detection via Weak SupervisionACM Transactions on Cyber-Physical Systems10.1145/36512248:2(1-26)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3651224
Joshi HKumar S(2024)Artificial Intelligence Failures in Autonomous Vehicles: Causes, Implications, and PreventionComputer10.1109/MC.2024.344943557:11(18-30)Online publication date: Nov-2024
https://doi.org/10.1109/MC.2024.3449435
Cui SJha SChen ZKalbarczvk ZIyer R(2024)iPrism: Characterize and Mitigate Risk by Quantifying Change in Escape Routes2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN58291.2024.00027(142-155)Online publication date: 24-Jun-2024
https://doi.org/10.1109/DSN58291.2024.00027
Akdeniz FAtay MVural ŞSavaş BBecerikli Y(2024)A Review of a Research in Autonomous Vehicles with Embedded SystemsInnovations in Smart Cities Applications Volume 710.1007/978-3-031-53824-7_21(229-239)Online publication date: 20-Feb-2024
https://doi.org/10.1007/978-3-031-53824-7_21
Escobar-Naranjo JCaiza GAyala PJordan EGarcia CGarcia M(2023)Autonomous Navigation of Robots: Optimization with DQNApplied Sciences10.3390/app1312720213:12(7202)Online publication date: 16-Jun-2023
https://doi.org/10.3390/app13127202
Kristoffersson Lind STriebel RNardi LKrueger V(2023)Out-of-Distribution Detection for Adaptive Computer VisionImage Analysis10.1007/978-3-031-31438-4_21(311-325)Online publication date: 27-Apr-2023
https://doi.org/10.1007/978-3-031-31438-4_21
Cui PWang J(2022)Out-of-Distribution (OOD) Detection Based on Deep Learning: A ReviewElectronics10.3390/electronics1121350011:21(3500)Online publication date: 28-Oct-2022
https://doi.org/10.3390/electronics11213500
Yuhas MXian Ng DEaswaran A(2022)Design Methodology for Deep Out-of-Distribution Detectors in Real-Time Cyber-Physical Systems2022 IEEE 28th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA)10.1109/RTCSA55878.2022.00025(180-185)Online publication date: Aug-2022
https://doi.org/10.1109/RTCSA55878.2022.00025

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents