research-article

Open access

EcoFusion: energy-aware adaptive sensor fusion for efficient autonomous vehicle perception

Authors:

Arnav Vaibhav Malawade,

Trier Mortlock,

Mohammad Abdullah Al FaruqueAuthors Info & Claims

DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference

Pages 481 - 486

https://doi.org/10.1145/3489517.3530489

Published: 23 August 2022 Publication History

Abstract

Autonomous vehicles use multiple sensors, large deep-learning models, and powerful hardware platforms to perceive the environment and navigate safely. In many contexts, some sensing modalities negatively impact perception while increasing energy consumption. We propose EcoFusion: an energy-aware sensor fusion approach that uses context to adapt the fusion method and reduce energy consumption without affecting perception performance. EcoFusion performs up to 9.5% better at object detection than existing fusion methods with approximately 60% less energy and 58% lower latency on the industry-standard Nvidia Drive PX2 hardware platform. We also propose several context-identification strategies, implement a joint optimization between energy and performance, and present scenario-specific results.

References

[1]

Sam Abuelsamid. 2020. Nvidia Cranks Up And Turns Down Its Drive AGX Orin Computers. Forbes (Jun 2020). https://www.forbes.com/sites/samabuelsamid/2020/05/14/nvidia-cranks-up-and-turns-down-its-drive-agx-orin-computers

[2]

Sabur Baidya et al. 2020. Vehicular and edge computing for emerging connected and autonomous vehicle applications. In DAC '20. IEEE, 1--6.

[3]

Dieter Balemans et al. 2020. Resource efficient sensor fusion by knowledge-based network pruning. Internet of Things 11 (2020), 100231.

[4]

Jared A Baxter et al. 2018. Review of electrical architectures and power requirements for automated vehicles. In ITEC '18. IEEE, 944--949.

[5]

Ivan Beretta et al. 2012. Design exploration of energy-performance trade-offs for wireless sensor networks. In DAC '12. 1043--1048.

[6]

Justin M Bradley et al. 2015. Optimization and control of cyber-physical vehicle systems. Sensors 15, 9 (2015), 23020--23049.

[7]

Changhao Chen et al. 2019. Selective sensor fusion for neural visual-inertial odometry. In CVPR '19. 10542--10551.

[8]

Mark Everingham et al. 2010. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision 88, 2 (2010), 303--338.

[9]

Vineet Gokhale et al. 2021. FEEL: fast, energy efficient localization for autonomous indoor vehicles. In ICC '21. IEEE.

[10]

Kaiming He et al. 2016. Deep residual learning for image recognition. In CVPR '16. 770--778.

[11]

Fred Lambert. 2016. All new Teslas are equipped with NVIDIA's new Drive PX 2 AI platform for self-driving - Electrek. https://electrek.co/2016/10/21/all-new-teslas-are-equipped-with-nvidias-new-drive-px-2-ai-platform-for-self-driving.

[12]

Sanghoon Lee et al. 2020. Accuracy---power controllable lidar sensor system with 3D object recognition for autonomous vehicle. Sensors 20, 19 (2020), 5706.

[13]

Velodyne Lidar. 2021. Velodyne HDL-32e Datasheet. https://velodynelidar.com/products/hdl-32e/

[14]

Shih-Chieh Lin et al. 2018. The architectural implications of autonomous driving: Constraints and acceleration. In ASPLOS'18.

[15]

Arnav Vaibhav Malawade, Trier Mortlock, and Mohammad Abdullah Al Faruque. 2022. HydraFusion: Context-Aware Selective Sensor Fusion for Robust and Efficient Autonomous Vehicle Perception. In ICCPS '22.

[16]

Arnav Malawade et al. 2021. SAGE: A Split-Architecture Methodology for Efficient End-to-End Autonomous Vehicle Control. ACM TECS 20, 5s (2021).

[17]

Ravi Teja Mullapudi et al. 2018. Hydranets: Specialized dynamic architectures for efficient inference. In CVPR '18. 8080--8089.

[18]

Navtech Radar. 2021. Navtech CTS Series. https://navtechradar.com/clearway-technical-specifications/compact-sensors

[19]

Shaoqing Ren et al. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015), 91--99.

[20]

K. Samal et al. 2020. Attention-Based Activation Pruning to Reduce Data Movement in Real-Time AI: A Case-Study on Local Motion Planning in Autonomous Vehicles. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 10, 3 (2020), 306--319.

[21]

Shreyas Sen. 2016. Context-aware energy-efficient communication for IoT sensor nodes. In DAC '16. IEEE, 1--6.

Digital Library

[22]

Marcel Sheeny et al. 2020. RADIATE: A radar dataset for automotive perception. arXiv preprint arXiv:2010.09076 3, 4 ( 2020).

[23]

Roman Solovyev et al. 2021. Weighted boxes fusion: Ensembling boxes from different object detection models. Image and Vision Computing 107 (2021), 104117.

[24]

Stereolabs. 2021. ZED Camera and SDK Overview. https://cdn.stereolabs.com/assets/datasheets/zed-camera-datasheet.pdf

[25]

Hokchhay Tann et al. 2016. Runtime configurable deep neural networks for energy-accuracy trade-off. In 2016 CODES + ISSS. IEEE, 1--10.

[26]

Korosh Vatanparvar et al. 2015. Battery lifetime-aware automotive climate control for electric vehicles. In DAC '15. IEEE, 1--6.

[27]

Boyu Zhang et al. 2018. Exploring energy and accuracy tradeoff in structure simplification of trained deep neural networks. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 8, 4 (2018), 836--848.

Cited By

Charroud AEl Moutaouakil KPalade VYahyaouy AOnyekpe UEyo E(2024)Localization and Mapping for Self-Driving Vehicles: A SurveyMachines10.3390/machines1202011812:2(118)Online publication date: 7-Feb-2024
https://doi.org/10.3390/machines12020118
Ignatious HEl-Sayed HBouktif S(2024)IFGAN—A Novel Image Fusion Model to Fuse 3D Point Cloud Sensory DataJournal of Sensor and Actuator Networks10.3390/jsan1301001513:1(15)Online publication date: 7-Feb-2024
https://doi.org/10.3390/jsan13010015
Mortlock TMalawade ATsujio KAl Faruque M(2024)CASTNet: A Context-Aware, Spatio-Temporal Dynamic Motion Prediction Ensemble for Autonomous DrivingACM Transactions on Cyber-Physical Systems10.1145/36486228:2(1-20)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3648622
Show More Cited By

Recommendations

Factors influencing continuance intention to use social network sites

Factors influencing continuance intention to use Facebook are examined.Satisfaction and perceived usefulness influence continuance intention.Enjoyment and subjective norms influence continuance intention.Habit mediates the relationship between ...
Factors affecting consumer stickiness to continue using mobile applications

According to a recent survey, more than half of all such apps are rarely been downloaded. This study thus examines consumer stickiness to continue to use mobile apps, using the post-acceptance model of information system continuance and technology ...
A study of mobile banking loyalty in Iran

This study investigates an exploratory study of mobile banking usage.Attitude is explained by both perceived usefulness and ease of use.Compatibility was found to be the most important driver.Perceived usefulness did not mediate the relationship between ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference

July 2022

1462 pages

ISBN:9781450391429

DOI:10.1145/3489517

General Chair:
Rob Oshana
NXP

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGDA: ACM Special Interest Group on Design Automation
IEEE CEDA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 August 2022

Check for updates

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

DAC '22

Sponsor:

SIGDA

DAC '22: 59th ACM/IEEE Design Automation Conference

July 10 - 14, 2022

California, San Francisco

Acceptance Rates

Overall Acceptance Rate 1,770 of 5,499 submissions, 32%

Upcoming Conference

DAC '25

Sponsor:
sigda

62nd ACM/IEEE Design Automation Conference

June 22 - 26, 2025

San Francisco , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
513
Total Downloads

Downloads (Last 12 months)300
Downloads (Last 6 weeks)26

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Charroud AEl Moutaouakil KPalade VYahyaouy AOnyekpe UEyo E(2024)Localization and Mapping for Self-Driving Vehicles: A SurveyMachines10.3390/machines1202011812:2(118)Online publication date: 7-Feb-2024
https://doi.org/10.3390/machines12020118
Ignatious HEl-Sayed HBouktif S(2024)IFGAN—A Novel Image Fusion Model to Fuse 3D Point Cloud Sensory DataJournal of Sensor and Actuator Networks10.3390/jsan1301001513:1(15)Online publication date: 7-Feb-2024
https://doi.org/10.3390/jsan13010015
Mortlock TMalawade ATsujio KAl Faruque M(2024)CASTNet: A Context-Aware, Spatio-Temporal Dynamic Motion Prediction Ensemble for Autonomous DrivingACM Transactions on Cyber-Physical Systems10.1145/36486228:2(1-20)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3648622
Naqvi SJamil HIqbal NKhan SKhan MQayyum FKim D(2024)Evolving Electric Mobility Energy Efficiency: In-Depth Analysis of Integrated Electronic Control Unit Development in Electric VehiclesIEEE Access10.1109/ACCESS.2024.335659812(15957-15983)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3356598
O'Hara CYairi T(2024)Graph-based meta-learning for context-aware sensor management in nonlinear safety-critical environmentsAdvanced Robotics10.1080/01691864.2024.232708338:6(368-385)Online publication date: 12-Mar-2024
https://doi.org/10.1080/01691864.2024.2327083
Kim SJung YMin HKim TJung J(2024)Adaptive sensor management for UGV monitoring based on risk mapsRobotics and Autonomous Systems10.1016/j.robot.2023.104605172:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.robot.2023.104605
Ignatious HEl-Sayed HKhan MKulkarni P(2023)A Generic Framework for Enhancing Autonomous Driving Accuracy through Multimodal Data FusionApplied Sciences10.3390/app13191074913:19(10749)Online publication date: 27-Sep-2023
https://doi.org/10.3390/app131910749
Ma JLi LXu C(2023)AutoRS: Environment-Dependent Real-Time Scheduling for End-to-End Autonomous DrivingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.332397534:12(3238-3252)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TPDS.2023.3323975
Ronco ABaumann NGiordano MMagno M(2023)Towards Robust Velocity and Position Estimation of Opponents for Autonomous Racing Using Low-Power Radar2023 9th International Workshop on Advances in Sensors and Interfaces (IWASI)10.1109/IWASI58316.2023.10164312(21-26)Online publication date: 8-Jun-2023
https://doi.org/10.1109/IWASI58316.2023.10164312
Zhang YMalawade AZhang XLi YSeong DAl Faruque MHuang S(2023)CARMA: Context-Aware Runtime Reconfiguration for Energy-Efficient Sensor Fusion2023 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED)10.1109/ISLPED58423.2023.10244517(1-6)Online publication date: 7-Aug-2023
https://doi.org/10.1109/ISLPED58423.2023.10244517
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents