research-article

Public Access

An Energy-aware Online Learning Framework for Resource Management in Heterogeneous Platforms

Authors:

Sumit K. Mandal,

Janardhan Rao Doppa,

Partha Pratim Pande,

Umit Y. OgrasAuthors Info & Claims

ACM Transactions on Design Automation of Electronic Systems (TODAES), Volume 25, Issue 3

Article No.: 28, Pages 1 - 26

https://doi.org/10.1145/3386359

Published: 13 May 2020 Publication History

All formats PDF

Abstract

Mobile platforms must satisfy the contradictory requirements of fast response time and minimum energy consumption as a function of dynamically changing applications. To address this need, systems-on-chip (SoC) that are at the heart of these devices provide a variety of control knobs, such as the number of active cores and their voltage/frequency levels. Controlling these knobs optimally at runtime is challenging for two reasons. First, the large configuration space prohibits exhaustive solutions. Second, control policies designed offline are at best sub-optimal, since many potential new applications are unknown at design-time. We address these challenges by proposing an online imitation learning approach. Our key idea is to construct an offline policy and adapt it online to new applications to optimize a given metric (e.g., energy). The proposed methodology leverages the supervision enabled by power-performance models learned at runtime. We demonstrate its effectiveness on a commercial mobile platform with 16 diverse benchmarks. Our approach successfully adapts the control policy to an unknown application after executing less than 25% of its instructions.

References

[1]

Ali Aalsaud et al. 2016. Power–aware performance adaptation of concurrent applications in heterogeneous many-core systems. In Proceedings of the International Symposium on Low Power Electronics and Design. 368--373.

[2]

Frank Bellosa. 2000. The benefits of event: Driven energy accounting in power-sensitive systems. In Proceedings of the 9th Workshop on ACM SIGOPS European Workshop: Beyond the PC: New Challenges for the Operating System. 37--42.

Digital Library

[3]

Ganapati Bhat, Sumit K. Mandal, Ujjwal Gupta, and Umit Y. Ogras. 2018. Online learning for adaptive optimization of heterogeneous socs. In Proceedings of the International Conference on Computer-Aided Design. ACM, 61.

[4]

Ganapati Bhat, Gaurav Singla, Ali K. Unver, and Umit Y. Ogras. 2017. Algorithmic optimization of thermal and power management for heterogeneous mobile platforms. IEEE Trans. VLSI Syst. 26, 3 (2017), 544--557.

Digital Library

[5]

Christian Bienia et al. 2008. The PARSEC benchmark suite: Characterization and architectural implications. In Proceedings of the International Conference on Parallel Architecture and Compilation Techniques. 72--81.

[6]

David Brooks, Robert P. Dick, Russ Joseph, and Li Shang. 2007. Power, thermal, and reliability modeling in nanometer-scale microprocessors. IEEE Micro 27, 3 (2007), 49--62.

Digital Library

[7]

Zhuo Chen and Diana Marculescu. 2015. Distributed reinforcement learning for power limited many-core system performance optimization. In Proceedings of the Design Automation in Test in Europe Conference (DATE’15). 1521--1526.

[8]

Ryan Cochran et al. [n.d.]. Pack 8 Cap: Adaptive DVFS and thread packing under power caps. In Proceedings of the 2011 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 175--185.

[9]

Gaurav Dhiman et al. 2009. System-level power management using online learning. IEEE Trans. Comput.-Aid. Des. Integr. Circ. Syst. 28, 5 (2009), 676--689.

Digital Library

[10]

Benedikt Dietrich and Samarjit Chakraborty. 2014. Lightweight graphics instrumentation for game state-specific power management in android. Multimedia Syst. 20, 5 (2014), 563--578.

Digital Library

[11]

Benedikt Dietrich, Swaroop Nunna, Dip Goswami, Samarjit Chakraborty, and Matthias Gries. 2010. LMS-based low-complexity game workload prediction for DVFS. In Proceedings of the 2010 IEEE International Conference on Computer Design. 417--424.

[12]

Yang Ge and Qinru Qiu. 2011. Dynamic thermal management for multimedia applications using machine learning. In Proceedings of the 48th Design Automation Conference. 95--100.

Digital Library

[13]

Ian J. Goodfellow et al. 2013. An empirical investigation of catastrophic forgetting in gradient-based neural networks. In Proceedings of International Conference on Learning Representations (ICLR’14).

[14]

Ujjwal Gupta et al. 2017. DyPO: Dynamic pareto-optimal configuration selection for heterogeneous MpSoCs. ACM Trans. Embedd. Comput. Syst. 16, 5s (2017), 123.

[15]

Ujjwal Gupta et al. 2019. A deep q-learning approach for dynamic management of heterogeneous processors. IEEE Comput. Arch. Lett. 18, 1 (2019), 14–17.

Digital Library

[16]

Ujjwal Gupta, Manoj Babu, Raid Ayoub, Michael Kishinevsky, Francesco Paterna, Suat Gumussoy, and Umit Y. Ogras. 2018. An online learning methodology for performance modeling of graphics processors. IEEE Trans. Comput. 67, 12 (2018), 1677--1691.

Digital Library

[17]

Ujjwal Gupta, Manoj Babu, Raid Ayoub, Michael Kishinevsky, Francesco Paterna, and Umit Y. Ogras. 2018. STAFF: Online learning with stabilized adaptive forgetting factor and feature selection algorithm. In Proceedings of the 55th Annual Design Automation Conference. 1--6.

[18]

Matthew R. Guthaus et al. 2001. Mibench: A free, commercially representative embedded benchmark suite. In Proceedings of the International Workshop on Workload Characterization. 3--14.

[19]

Hardkernel. 2014. ODROID-XU3. Retrieved November 24, 2018 https://wiki.odroid.com/old_product/odroid-xu3/odroid-xu3.

[20]

Robert Hecht-Nielsen. 1992. Theory of the backpropagation neural network. In Neural Networks for Perception. Elsevier, 65--93.

Digital Library

[21]

David Kadjo, Raid Ayoub, Michael Kishinevsky, and Paul V. Gratz. 2015. A control-theoretic approach for energy efficient CPU-GPU subsystem in mobile platforms. In Proceedings of the 52nd Annual Design Automation Conference. ACM, 62.

[22]

Ryan Gary Kim et al. 2017. Imitation learning for dynamic VFI control in large-scale manycore systems. IEEE Trans. VLSI Syst. 25, 9 (2017), 2458--2471.

Digital Library

[23]

Chris Lattner and Vikram Adve. 2004. LLVM: A compilation framework for lifelong program analysis 8 transformation. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization. 75.

[24]

Sumit K. Mandal et al. 2019. Dynamic resource management of heterogeneous mobile platforms via imitation learning. IEEE Trans. VLSI Syst. 27, 12 (2019), 2842–2854.

[25]

Jose F. Martinez and Engin Ipek. 2009. Dynamic multicore resource management: A machine learning approach. IEEE Micro 29, 5 (2009).

[26]

Jerry M. Mendel. 1995. Lessons in Estimation Theory for Signal Processing, Communications, and Control. Pearson Education.

[27]

Volodymyr Mnih et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.

[28]

Philip J. Mucci, Shirley Browne, Christine Deane, and George Ho. 1999. PAPI: A portable interface to hardware performance counters. In Proceedings of the Department of Defense HPCMP Users Group Conference, Vol. 710.

[29]

Venkatesh Pallipadi and Alexey Starikovskiy. 2006. The ondemand governor. In Proceedings of the Linux Symposium, Vol. 2. 215--230.

[30]

Jurn-Gyu Park, Nikil Dutt, and Sung-Soo Lim. 2017. ML-Gov: A machine learning enhanced integrated CPU-GPU DVFS governor for mobile gaming. In Proceedings of the Symposium on Embedded Systems for Real-Time Multimedia. 12--21.

Digital Library

[31]

Anuj Pathania et al. 2015. Power-performance modelling of mobile gaming workloads on heterogeneous MPSoCs. In Proceedings of the Design Automation Conference. 201.

[32]

Anuj Pathania, Qing Jiao, Alok Prakash, and Tulika Mitra. 2014. Integrated CPU-GPU power management for 3D mobile games. In Proceedings of the Design Automation Conference. 1--6.

Digital Library

[33]

Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 627--635.

[34]

Stefan Schaal. 1999. Is imitation learning the route to humanoid robots? Trends Cogn. Sci. 3, 6 (1999), 233--242.

[35]

Rishad A. Shafik, Sheng Yang, Anup Das, Luis A. Maeda-Nunez, Geoff V. Merrett, and Bashir M. Al-Hashimi. 2015. Learning transfer-based adaptive energy minimization in embedded systems. IEEE Trans. Comput.-Aid. Des. Integr. Circ. Syst. 35, 6 (2015), 877--890.

Digital Library

[36]

Amit Singh, Karunakar Reddy Basireddy, Alok Prakash, Geoff Merrett, and Bashir M. Al-Hashimi. 2020. Collaborative adaptation for energy-efficient heterogeneous mobile SoCs. IEEE Trans. Comput. 69, 2 (2020), 185--197.

[37]

Statista. [n.d.]. Mobile App Usage—Statistics 8 Facts. Retrieved November 24, 2018 from https://www.statista.com/topics/1002/mobile-app-usage/.

[38]

Hameedah Sultan, Gayathri Ananthanarayanan, and Smruti R. Sarangi. 2014. Processor power estimation techniques: A survey. Int. J. High Perf. Syst. Arch. 5, 2 (2014), 93--114.

Digital Library

[39]

Wen Sun, Arun Venkatraman, Geoffrey J. Gordon, Byron Boots, and J. Andrew Bagnell. 2017. Deeply AggreVaTeD: Differentiable imitation learning for sequential prediction. In Proceedings of the 34th International Conference Machine Learning, Vol. 70. 3309--3318.

[40]

Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press.

Digital Library

[41]

Shelby Thomas et al. 2014. CortexSuite: A synthetic brain benchmark suite. In Proceedings of the IEEE International Symposium on Workload Characterization (IISWC’14). 76--79.

[42]

Zhongyuan Tian, Zhe Wang, Haoran Li, Peng Yang, Rafael Kioji Vivas Maeda, and Jiang Xu. 2018. Multi-device collaborative management through knowledge sharing. In Proceedings of the 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC’18). 22--27.

[43]

F. M. M. ul Islam and Man Lin. 2017. Hybrid DVFS scheduling for real-time systems based on reinforcement learning. IEEE Syst. J. 11, 2 (2017), 931--940.

[44]

N. Vallina-Rodriguez and Jon Crowcroft. 2012. Energy management techniques in modern mobile handsets. IEEE Comm. Surv. Tutor. 15, 1 (2012), 1--20.

[45]

Jae-Yeon Won et al. 2014. Up by their bootstraps: Online learning in artificial neural networks for CMP uncore power management. In Proceedings of the 2014 IEEE 20th International Symposium on HPCA. 308--319.

[46]

Wu Ye, Narayanan Vijaykrishnan, Mahmut Kandemir, and Mary Jane Irwin. 2000. The design and use of simplepower: A cycle-accurate energy estimation tool. In Proceedings of the 37th Annual Design Automation Conference. ACM, 340--345.

Digital Library

[47]

Qingchen Zhang et al. 2017. Energy-efficient scheduling for real-time systems based on deep q-learning model. IEEE Trans. Sust. Comput. 4, 1 (2017), 132--141.

[48]

Qingchen Zhang, Man Lin, Laurence T. Yang, Zhikui Chen, Samee U. Khan, and Peng Li. 2018. A double deep Q-learning model for energy-efficient edge scheduling. IEEE Trans. Serv. Comput. (2018).

Cited By

Cao KCui YLi LZhou JHu S(2024)CPU-GPU Cooperative QoS Optimization of Personalized Digital Healthcare Using Machine Learning and Swarm IntelligenceIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2022.320750921:4(521-533)Online publication date: Jul-2024
https://doi.org/10.1109/TCBB.2022.3207509
Lu YHu CGong YCheng D(2024)Improving User Experience via Reinforcement Learning-Based Resource Management on Mobile DevicesAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5581-3_31(383-395)Online publication date: 1-Aug-2024
https://doi.org/10.1007/978-981-97-5581-3_31
Xiao BYang JQi X(2023)Imitation Learning-Based Performance-Power Trade-Off Uncore Frequency Scaling Policy for Multicore SystemSensors10.3390/s2303144923:3(1449)Online publication date: 28-Jan-2023
https://doi.org/10.3390/s23031449
Show More Cited By

Index Terms

An Energy-aware Online Learning Framework for Resource Management in Heterogeneous Platforms
1. Computing methodologies
2. Hardware
  1. Very large scale integration design
    1. On-chip resource management

Recommendations

Online Inverse Reinforcement Learning Under Occlusion
AAMAS '19: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems

Inverse reinforcement learning (IRL) is the problem of learning the preferences of an agent from observing its behavior on a task. While this problem is witnessing sustained attention, the related problem of online IRL - where the observations are ...
Online learning of timeout policies for dynamic power management
Regular Papers

Dynamic power management (DPM) refers to strategies which selectively change the operational states of a device during runtime to reduce the power consumption based on the past usage pattern, the current workload, and the given performance constraint. ...
A Reinforcement Learning Framework for Dynamic Power Management of a Portable, Multi-camera Traffic Monitoring System
GREENCOM '12: Proceedings of the 2012 IEEE International Conference on Green Computing and Communications

Dynamic Power Management (DPM) refers to a set of strategies that achieves efficient power consumption by selectively turning off (or reducing the performance of) a system components when they are idle or are serving light workloads. This paper presents ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Design Automation of Electronic Systems

ACM Transactions on Design Automation of Electronic Systems Volume 25, Issue 3

May 2020

179 pages

ISSN:1084-4309

EISSN:1557-7309

DOI:10.1145/3386183

Editor:
Naehyuck Chang
Korea Advanced Institute of Science and Technology, Korea

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 13 May 2020

Online AM: 07 May 2020

Accepted: 01 February 2020

Revised: 01 February 2020

Received: 01 September 2019

Published in TODAES Volume 25, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
1,056
Total Downloads

Downloads (Last 12 months)289
Downloads (Last 6 weeks)35

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cao KCui YLi LZhou JHu S(2024)CPU-GPU Cooperative QoS Optimization of Personalized Digital Healthcare Using Machine Learning and Swarm IntelligenceIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2022.320750921:4(521-533)Online publication date: Jul-2024
https://doi.org/10.1109/TCBB.2022.3207509
Lu YHu CGong YCheng D(2024)Improving User Experience via Reinforcement Learning-Based Resource Management on Mobile DevicesAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5581-3_31(383-395)Online publication date: 1-Aug-2024
https://doi.org/10.1007/978-981-97-5581-3_31
Xiao BYang JQi X(2023)Imitation Learning-Based Performance-Power Trade-Off Uncore Frequency Scaling Policy for Multicore SystemSensors10.3390/s2303144923:3(1449)Online publication date: 28-Jan-2023
https://doi.org/10.3390/s23031449
Esper KWildermann STeich J(2023)Automatic Synthesis of FSMs for Enforcing Non-functional Requirements on MPSoCs Using Multi-objective Evolutionary AlgorithmsACM Transactions on Design Automation of Electronic Systems10.1145/361783228:6(1-20)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3617832
Kwon JLee YKal HKim MKim YRo W(2023)McCore: A Holistic Management of High-Performance Heterogeneous MulticoresProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3614295(1044-1058)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3614295
Spieck JSixdenier PEsper KWildermann STeich JZhu Qvon Hanxleden RStephen EBrandt J(2023)Hybrid Genetic Reinforcement Learning for Generating Run-Time Requirement EnforcersProceedings of the 21st ACM-IEEE International Conference on Formal Methods and Models for System Design10.1145/3610579.3611091(23-35)Online publication date: 21-Sep-2023
https://dl.acm.org/doi/10.1145/3610579.3611091
Narang GDeshwal AAyoub RKishinevsky MRao Doppa JPande P(2023)Dynamic Power Management in Large Manycore Systems: A Learning-to-Search FrameworkACM Transactions on Design Automation of Electronic Systems10.1145/360350128:5(1-21)Online publication date: 8-Sep-2023
https://dl.acm.org/doi/10.1145/3603501
Chen TZhang GYu BLi BSchlichtmann U(2023)Machine Learning in Advanced IC Design: A Methodological SurveyIEEE Design & Test10.1109/MDAT.2022.321679940:1(17-33)Online publication date: Feb-2023
https://doi.org/10.1109/MDAT.2022.3216799
Karzhaubayeva MAmangeldi APark J(2023)CNN Workloads Characterization and Integrated CPU–GPU DVFS Governors on Embedded SystemsIEEE Embedded Systems Letters10.1109/LES.2023.329933515:4(202-205)Online publication date: Dec-2023
https://doi.org/10.1109/LES.2023.3299335
Narang GAyoub RKishinevsky MDoppa JPande P(2023)Uncertainty-Aware Online Learning for Dynamic Power Management in Large Manycore Systems2023 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED)10.1109/ISLPED58423.2023.10244486(1-6)Online publication date: 7-Aug-2023
https://doi.org/10.1109/ISLPED58423.2023.10244486
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents