Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article
Public Access

An Energy-aware Online Learning Framework for Resource Management in Heterogeneous Platforms

Published: 13 May 2020 Publication History

Abstract

Mobile platforms must satisfy the contradictory requirements of fast response time and minimum energy consumption as a function of dynamically changing applications. To address this need, systems-on-chip (SoC) that are at the heart of these devices provide a variety of control knobs, such as the number of active cores and their voltage/frequency levels. Controlling these knobs optimally at runtime is challenging for two reasons. First, the large configuration space prohibits exhaustive solutions. Second, control policies designed offline are at best sub-optimal, since many potential new applications are unknown at design-time. We address these challenges by proposing an online imitation learning approach. Our key idea is to construct an offline policy and adapt it online to new applications to optimize a given metric (e.g., energy). The proposed methodology leverages the supervision enabled by power-performance models learned at runtime. We demonstrate its effectiveness on a commercial mobile platform with 16 diverse benchmarks. Our approach successfully adapts the control policy to an unknown application after executing less than 25% of its instructions.

References

[1]
Ali Aalsaud et al. 2016. Power–aware performance adaptation of concurrent applications in heterogeneous many-core systems. In Proceedings of the International Symposium on Low Power Electronics and Design. 368--373.
[2]
Frank Bellosa. 2000. The benefits of event: Driven energy accounting in power-sensitive systems. In Proceedings of the 9th Workshop on ACM SIGOPS European Workshop: Beyond the PC: New Challenges for the Operating System. 37--42.
[3]
Ganapati Bhat, Sumit K. Mandal, Ujjwal Gupta, and Umit Y. Ogras. 2018. Online learning for adaptive optimization of heterogeneous socs. In Proceedings of the International Conference on Computer-Aided Design. ACM, 61.
[4]
Ganapati Bhat, Gaurav Singla, Ali K. Unver, and Umit Y. Ogras. 2017. Algorithmic optimization of thermal and power management for heterogeneous mobile platforms. IEEE Trans. VLSI Syst. 26, 3 (2017), 544--557.
[5]
Christian Bienia et al. 2008. The PARSEC benchmark suite: Characterization and architectural implications. In Proceedings of the International Conference on Parallel Architecture and Compilation Techniques. 72--81.
[6]
David Brooks, Robert P. Dick, Russ Joseph, and Li Shang. 2007. Power, thermal, and reliability modeling in nanometer-scale microprocessors. IEEE Micro 27, 3 (2007), 49--62.
[7]
Zhuo Chen and Diana Marculescu. 2015. Distributed reinforcement learning for power limited many-core system performance optimization. In Proceedings of the Design Automation in Test in Europe Conference (DATE’15). 1521--1526.
[8]
Ryan Cochran et al. [n.d.]. Pack 8 Cap: Adaptive DVFS and thread packing under power caps. In Proceedings of the 2011 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 175--185.
[9]
Gaurav Dhiman et al. 2009. System-level power management using online learning. IEEE Trans. Comput.-Aid. Des. Integr. Circ. Syst. 28, 5 (2009), 676--689.
[10]
Benedikt Dietrich and Samarjit Chakraborty. 2014. Lightweight graphics instrumentation for game state-specific power management in android. Multimedia Syst. 20, 5 (2014), 563--578.
[11]
Benedikt Dietrich, Swaroop Nunna, Dip Goswami, Samarjit Chakraborty, and Matthias Gries. 2010. LMS-based low-complexity game workload prediction for DVFS. In Proceedings of the 2010 IEEE International Conference on Computer Design. 417--424.
[12]
Yang Ge and Qinru Qiu. 2011. Dynamic thermal management for multimedia applications using machine learning. In Proceedings of the 48th Design Automation Conference. 95--100.
[13]
Ian J. Goodfellow et al. 2013. An empirical investigation of catastrophic forgetting in gradient-based neural networks. In Proceedings of International Conference on Learning Representations (ICLR’14).
[14]
Ujjwal Gupta et al. 2017. DyPO: Dynamic pareto-optimal configuration selection for heterogeneous MpSoCs. ACM Trans. Embedd. Comput. Syst. 16, 5s (2017), 123.
[15]
Ujjwal Gupta et al. 2019. A deep q-learning approach for dynamic management of heterogeneous processors. IEEE Comput. Arch. Lett. 18, 1 (2019), 14–17.
[16]
Ujjwal Gupta, Manoj Babu, Raid Ayoub, Michael Kishinevsky, Francesco Paterna, Suat Gumussoy, and Umit Y. Ogras. 2018. An online learning methodology for performance modeling of graphics processors. IEEE Trans. Comput. 67, 12 (2018), 1677--1691.
[17]
Ujjwal Gupta, Manoj Babu, Raid Ayoub, Michael Kishinevsky, Francesco Paterna, and Umit Y. Ogras. 2018. STAFF: Online learning with stabilized adaptive forgetting factor and feature selection algorithm. In Proceedings of the 55th Annual Design Automation Conference. 1--6.
[18]
Matthew R. Guthaus et al. 2001. Mibench: A free, commercially representative embedded benchmark suite. In Proceedings of the International Workshop on Workload Characterization. 3--14.
[19]
Hardkernel. 2014. ODROID-XU3. Retrieved November 24, 2018 https://wiki.odroid.com/old_product/odroid-xu3/odroid-xu3.
[20]
Robert Hecht-Nielsen. 1992. Theory of the backpropagation neural network. In Neural Networks for Perception. Elsevier, 65--93.
[21]
David Kadjo, Raid Ayoub, Michael Kishinevsky, and Paul V. Gratz. 2015. A control-theoretic approach for energy efficient CPU-GPU subsystem in mobile platforms. In Proceedings of the 52nd Annual Design Automation Conference. ACM, 62.
[22]
Ryan Gary Kim et al. 2017. Imitation learning for dynamic VFI control in large-scale manycore systems. IEEE Trans. VLSI Syst. 25, 9 (2017), 2458--2471.
[23]
Chris Lattner and Vikram Adve. 2004. LLVM: A compilation framework for lifelong program analysis 8 transformation. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization. 75.
[24]
Sumit K. Mandal et al. 2019. Dynamic resource management of heterogeneous mobile platforms via imitation learning. IEEE Trans. VLSI Syst. 27, 12 (2019), 2842–2854.
[25]
Jose F. Martinez and Engin Ipek. 2009. Dynamic multicore resource management: A machine learning approach. IEEE Micro 29, 5 (2009).
[26]
Jerry M. Mendel. 1995. Lessons in Estimation Theory for Signal Processing, Communications, and Control. Pearson Education.
[27]
Volodymyr Mnih et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.
[28]
Philip J. Mucci, Shirley Browne, Christine Deane, and George Ho. 1999. PAPI: A portable interface to hardware performance counters. In Proceedings of the Department of Defense HPCMP Users Group Conference, Vol. 710.
[29]
Venkatesh Pallipadi and Alexey Starikovskiy. 2006. The ondemand governor. In Proceedings of the Linux Symposium, Vol. 2. 215--230.
[30]
Jurn-Gyu Park, Nikil Dutt, and Sung-Soo Lim. 2017. ML-Gov: A machine learning enhanced integrated CPU-GPU DVFS governor for mobile gaming. In Proceedings of the Symposium on Embedded Systems for Real-Time Multimedia. 12--21.
[31]
Anuj Pathania et al. 2015. Power-performance modelling of mobile gaming workloads on heterogeneous MPSoCs. In Proceedings of the Design Automation Conference. 201.
[32]
Anuj Pathania, Qing Jiao, Alok Prakash, and Tulika Mitra. 2014. Integrated CPU-GPU power management for 3D mobile games. In Proceedings of the Design Automation Conference. 1--6.
[33]
Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 627--635.
[34]
Stefan Schaal. 1999. Is imitation learning the route to humanoid robots? Trends Cogn. Sci. 3, 6 (1999), 233--242.
[35]
Rishad A. Shafik, Sheng Yang, Anup Das, Luis A. Maeda-Nunez, Geoff V. Merrett, and Bashir M. Al-Hashimi. 2015. Learning transfer-based adaptive energy minimization in embedded systems. IEEE Trans. Comput.-Aid. Des. Integr. Circ. Syst. 35, 6 (2015), 877--890.
[36]
Amit Singh, Karunakar Reddy Basireddy, Alok Prakash, Geoff Merrett, and Bashir M. Al-Hashimi. 2020. Collaborative adaptation for energy-efficient heterogeneous mobile SoCs. IEEE Trans. Comput. 69, 2 (2020), 185--197.
[37]
Statista. [n.d.]. Mobile App Usage—Statistics 8 Facts. Retrieved November 24, 2018 from https://www.statista.com/topics/1002/mobile-app-usage/.
[38]
Hameedah Sultan, Gayathri Ananthanarayanan, and Smruti R. Sarangi. 2014. Processor power estimation techniques: A survey. Int. J. High Perf. Syst. Arch. 5, 2 (2014), 93--114.
[39]
Wen Sun, Arun Venkatraman, Geoffrey J. Gordon, Byron Boots, and J. Andrew Bagnell. 2017. Deeply AggreVaTeD: Differentiable imitation learning for sequential prediction. In Proceedings of the 34th International Conference Machine Learning, Vol. 70. 3309--3318.
[40]
Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press.
[41]
Shelby Thomas et al. 2014. CortexSuite: A synthetic brain benchmark suite. In Proceedings of the IEEE International Symposium on Workload Characterization (IISWC’14). 76--79.
[42]
Zhongyuan Tian, Zhe Wang, Haoran Li, Peng Yang, Rafael Kioji Vivas Maeda, and Jiang Xu. 2018. Multi-device collaborative management through knowledge sharing. In Proceedings of the 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC’18). 22--27.
[43]
F. M. M. ul Islam and Man Lin. 2017. Hybrid DVFS scheduling for real-time systems based on reinforcement learning. IEEE Syst. J. 11, 2 (2017), 931--940.
[44]
N. Vallina-Rodriguez and Jon Crowcroft. 2012. Energy management techniques in modern mobile handsets. IEEE Comm. Surv. Tutor. 15, 1 (2012), 1--20.
[45]
Jae-Yeon Won et al. 2014. Up by their bootstraps: Online learning in artificial neural networks for CMP uncore power management. In Proceedings of the 2014 IEEE 20th International Symposium on HPCA. 308--319.
[46]
Wu Ye, Narayanan Vijaykrishnan, Mahmut Kandemir, and Mary Jane Irwin. 2000. The design and use of simplepower: A cycle-accurate energy estimation tool. In Proceedings of the 37th Annual Design Automation Conference. ACM, 340--345.
[47]
Qingchen Zhang et al. 2017. Energy-efficient scheduling for real-time systems based on deep q-learning model. IEEE Trans. Sust. Comput. 4, 1 (2017), 132--141.
[48]
Qingchen Zhang, Man Lin, Laurence T. Yang, Zhikui Chen, Samee U. Khan, and Peng Li. 2018. A double deep Q-learning model for energy-efficient edge scheduling. IEEE Trans. Serv. Comput. (2018).

Cited By

View all
  • (2024)QoS-Aware Power Management via Scheduling and Governing Co-Optimization on Mobile DevicesIEEE Transactions on Mobile Computing10.1109/TMC.2024.343826723:12(13654-13669)Online publication date: Dec-2024
  • (2024)CPU-GPU Cooperative QoS Optimization of Personalized Digital Healthcare Using Machine Learning and Swarm IntelligenceIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2022.320750921:4(521-533)Online publication date: Jul-2024
  • (2024)Improving User Experience via Reinforcement Learning-Based Resource Management on Mobile DevicesAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5581-3_31(383-395)Online publication date: 5-Aug-2024
  • Show More Cited By

Index Terms

  1. An Energy-aware Online Learning Framework for Resource Management in Heterogeneous Platforms

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Design Automation of Electronic Systems
      ACM Transactions on Design Automation of Electronic Systems  Volume 25, Issue 3
      May 2020
      179 pages
      ISSN:1084-4309
      EISSN:1557-7309
      DOI:10.1145/3386183
      • Editor:
      • Naehyuck Chang
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Journal Family

      Publication History

      Published: 13 May 2020
      Online AM: 07 May 2020
      Accepted: 01 February 2020
      Revised: 01 February 2020
      Received: 01 September 2019
      Published in TODAES Volume 25, Issue 3

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Dynamic power management
      2. imitation learning
      3. online learning
      4. reinforcement learning

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Funding Sources

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)287
      • Downloads (Last 6 weeks)30
      Reflects downloads up to 23 Dec 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)QoS-Aware Power Management via Scheduling and Governing Co-Optimization on Mobile DevicesIEEE Transactions on Mobile Computing10.1109/TMC.2024.343826723:12(13654-13669)Online publication date: Dec-2024
      • (2024)CPU-GPU Cooperative QoS Optimization of Personalized Digital Healthcare Using Machine Learning and Swarm IntelligenceIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2022.320750921:4(521-533)Online publication date: Jul-2024
      • (2024)Improving User Experience via Reinforcement Learning-Based Resource Management on Mobile DevicesAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5581-3_31(383-395)Online publication date: 5-Aug-2024
      • (2023)Imitation Learning-Based Performance-Power Trade-Off Uncore Frequency Scaling Policy for Multicore SystemSensors10.3390/s2303144923:3(1449)Online publication date: 28-Jan-2023
      • (2023)Automatic Synthesis of FSMs for Enforcing Non-functional Requirements on MPSoCs Using Multi-objective Evolutionary AlgorithmsACM Transactions on Design Automation of Electronic Systems10.1145/361783228:6(1-20)Online publication date: 16-Oct-2023
      • (2023)McCore: A Holistic Management of High-Performance Heterogeneous MulticoresProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3614295(1044-1058)Online publication date: 28-Oct-2023
      • (2023)Hybrid Genetic Reinforcement Learning for Generating Run-Time Requirement EnforcersProceedings of the 21st ACM-IEEE International Conference on Formal Methods and Models for System Design10.1145/3610579.3611091(23-35)Online publication date: 21-Sep-2023
      • (2023)Dynamic Power Management in Large Manycore Systems: A Learning-to-Search FrameworkACM Transactions on Design Automation of Electronic Systems10.1145/360350128:5(1-21)Online publication date: 8-Sep-2023
      • (2023)Machine Learning in Advanced IC Design: A Methodological SurveyIEEE Design & Test10.1109/MDAT.2022.321679940:1(17-33)Online publication date: Feb-2023
      • (2023)CNN Workloads Characterization and Integrated CPU–GPU DVFS Governors on Embedded SystemsIEEE Embedded Systems Letters10.1109/LES.2023.329933515:4(202-205)Online publication date: Dec-2023
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Login options

      Full Access

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media