research-article

Online learning of timeout policies for dynamic power management

Authors:

Umair Ali Khan,

Bernhard RinnerAuthors Info & Claims

ACM Transactions on Embedded Computing Systems (TECS), Volume 13, Issue 4

Article No.: 96, Pages 1 - 25

https://doi.org/10.1145/2529992

Published: 10 March 2014 Publication History

Abstract

Dynamic power management (DPM) refers to strategies which selectively change the operational states of a device during runtime to reduce the power consumption based on the past usage pattern, the current workload, and the given performance constraint. The power management problem becomes more challenging when the workload exhibits nonstationary behavior which may degrade the performance of any single or static DPM policy.

This article presents a reinforcement learning (RL)-based DPM technique for optimal selection of timeout values in the different device states. Each timeout period determines how long the device will remain in a particular state before the transition decision is taken. The timeout selection is based on workload estimates derived from a Multilayer Artificial Neural Network (ML-ANN) and an objective function given by weighted performance and power parameters. Our DPM approach is further able to adapt the power-performance weights online to meet user-specified power and performance constraints, respectively. We have completely implemented our DPM algorithm on our embedded traffic surveillance platform and performed long-term experiments using real traffic data to demonstrate the effectiveness of the DPM. Our results show that the proposed learning algorithm not only adequately explores the power-performance trade-off with nonstationary workload but can also successfully perform online adjustment of the trade-off parameter in order to meet the user-specified constraint.

References

[1]

ACPI. 2011. Advanced Configuration and Power Interface Specification (ACPI). ACPI Specification Document 5. http://www.acpi.info.

[2]

F. Barrero, S. Toral, M. Vargas, F. Cortés, and J. Milla. 2010. Internet in the development of future road-traffic control systems. Int. Res. 20, 2, 154--168.

[3]

L. Benini, A. Bogliolo, and G. De Micheli. 1999. Policy optimization for dynamic power management. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 18, 6, 813--833.

Digital Library

[4]

L. Benini, A. Bogliolo, and G. De Micheli. 2000. A survey of design techniques for system-level dynamic power management. IEEE Trans. VLSI Syst. 8, 3, 299--316.

Digital Library

[5]

K. Bhatti, C. Belleudy, and M. Auguin. 2010. Power management in real time embedded systems through online and adaptive interplay of dpm and dvfs policies. In Proceedings of the IEEE/IFIP 8th International Conference on Embedded and Ubiquitous Computing (EUC). 184--191.

Digital Library

[6]

H. Bischof, M. Godec, C. Leistner, B. Rinner, and A. Starzacher. 2010. Autonomous audio-supported learning of visual classifiers for traffic monitoring. IEEE Intell. Syst. 25, 3, 15--23.

Digital Library

[7]

P. Bogdan and R. Marculescu. 2011a. Cyberphysical systems: Workload modeling and design optimization. IEEE Trans. Des. Test Comput. 28, 4, 78--87.

Digital Library

[8]

P. Bogdan and R. Marculescu. 2011b. Non-stationary traffic analysis and its implications on multicore platform design. IEEE Trans. Comput.-Aid. Des. Integr. Circuits Syst. 30, 4, 508--519.

Digital Library

[9]

P. Bogdan and R. Marculescu. 2011c. Towards a science of cyber-physical systems design. In Proceedings of the IEEE/ACM International Conference on Cyber-Physical Systems (ICCPS). 99--108.

Digital Library

[10]

P. Bogdan, R. Marculescu, S. Jain, and R. T. Gavila. 2012. An optimal control approach to power management for multi-voltage and frequency islands multiprocessor platforms under highly variable workloads. In Proceedings of the IEEE/ACM 6th International Symposium on Networks on Chip (NoCS). 35--42.

Digital Library

[11]

E. Y. Chung, L. Benini, and G. De Micheli. 1999. Dynamic power management using adaptive learning tree. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. 274--279.

Digital Library

[12]

R. David, P. Bogdan, and R. Marculescu. 2012. Dynamic power management for multicores: Case study using the intel SCC. In Proceedings of the IEEE/IFIP 20th International Conference on VLSI and System-on-Chip (VLSI-SoC). 147--152.

[13]

G. Dhiman and T. S. Rosing. 2006. Dynamic power management using machine learning. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. 747--754.

Digital Library

[14]

F. Douglis, P. Krishnan, and B. Bershad. 1995. Adaptive disk spin-down policies for mobile computers. Comput. Syst. 8, 4.

[15]

J. Durand, S. Girard, and V. Ciriza. 2012. Optimization of power consumption and device availability based on point process modelling of the request sequence. J. Royal Stat. Soci. 3.

[16]

C. Eui-Young, L. Benini, A. Bogliolo, L. Yung-Hsiang, and G. De-Micheli. 2002. Dynamic power management for nonstationary service requests. IEEE Trans. Comput. 51, 11, 1345--1361.

Digital Library

[17]

A. Fallahi and E. Hossain. 2007. QoS provisioning in wireless video sensor networks: A dynamic power management framework. IEEE Trans. Wirel. Commun. 14, 6, 40--49.

Digital Library

[18]

K. Fei, T. Pin, and Q. Shi. 2006. Genetic algorithm based idle length prediction scheme for dynamic power management. In Proceedings of the IMACS Multiconference on Computational Engineering in Systems Applications. 1437--1443.

[19]

M. Ghasemazar, H. Goudarzi, and M. Pedram. 2012. Robust optimization of a chip multiprocessor's performance under power and thermal constraints. In Proceedings of the IEEE 30th Conference on Computer Design (ICCD). IEEE, 108--114.

Digital Library

[20]

C. H. Hwang and A. C. H. Wu. 2000. A predictive system shutdown method for energy saving of event-driven computation. ACM Trans. Des. Autom. Electron. Syst. 5, 2, 226--241.

Digital Library

[21]

J. Hwisung and M. Pedram. 2010. Supervised learning based power management for multicore processors. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 29, 9, 1395--1408.

Digital Library

[22]

S. Irani, S. Shukla, and R. Gupta. 2003. Online strategies for dynamic power management in systems with multiple power-saving states. ACM Trans. Embed. Comput. Syst. 2, 3, 325--346.

Digital Library

[23]

N. Kandasamy and S. Abdelwahed. 2004. A control-theoretic approach to power management in embedded processors. In Proceedings of Informatics 9, 1--12.

[24]

U. A. Khan. 2013. Online learning of timeout policies for dynamic power management. Ph.D. dissertation. Alpen-Adria Universität Klagenfurt, Austria.

[25]

U. A. Khan, M. Godec, M. Quaritsch, M. Hennecke, H. Bischof, and B. Rinner. 2012. MobiTrick--Mobile traffic checker. In Proceedings of the ITS World Congress.

[26]

U. A. Khan, M. Quaritsch, and B. Rinner. 2011. Design of a heterogeneous, energy-aware, stereo-vision based sensing platform for traffic surveillance. In Proceedings of the 9th Workshop on Intelligent Solutions in Embedded Systems. 47--52.

[27]

U. A. Khan and B. Rinner. 2012a. Dynamic power management for portable, multi-camera traffic monitoring. In Proceedings of the IEEE Real Time and Embedded Technology and Applications Symposium.

[28]

U. A. Khan and B. Rinner. 2012b. A reinforcement learning framework for dynamic power management of a portable, multi-camera traffic monitoring system. In Proceedings of the IEEE Conference on Green Computing and Communications.

Digital Library

[29]

Y. H. Lu and G. De Micheli. 2001. Comparing system-level power management policies. IEEE Trans. Des. Test Comput. 18, 2, 10--19.

Digital Library

[30]

S. Mannor, B. Kveton, S. Siddiqi, and C. H. Yu. 2006. Machine learning for adaptive power management. Auton. Comput. 10, 4, 299--312.

[31]

C. Moser, J. Chen, and L. Thiele. 2010. Dynamic power management in environmentally powered systems. In Proceedings of the Design Automation Conference (ASP-DAC). IEEE, 81--88.

Digital Library

[32]

S. Natarajan and P. Tadepalli. 2005. Dynamic preferences in multi-criteria reinforcement learning. In Proceedings of the International Conference on Machine Learning. 601--608.

Digital Library

[33]

U. Y. Ogras, R. Marculescu, D. Marculescu, and E. G. Jung. 2009. Design and management of voltage-frequency island partitioned networks-on-chip. IEEE Trans. VLSI Syst. 17, 3, 330--341.

Digital Library

[34]

C. M. Olsen and C. Narayanaswarni. 2006. PowerNap: An efficient power management scheme for mobile devices. IEEE Trans. Mobile Comput. 5, 7, 816--828.

Digital Library

[35]

A. Paul. 2013. Dynamic power management for ubiquitous network devices. Adv. Sci. Lett. 19, 7, 2046--2049.

[36]

T. Phit and K. Abe. 2006. Packet inter-arrival time estimation using neural network models. In Proceedings of the Internet Conference.

[37]

F. Pletzer, R. Tusch, L. Böszörmenyi, and B. Rinner. 2012. Robust traffic state estimation on smart cameras. In Proceedings of the IEEE Conference on Advanced Video and Signal-Based Surveillance. 434--439.

Digital Library

[38]

V. L. Prabha and E. C. Monie. 2007. Hardware architecture of reinforcement learning scheme for dynamic power management in embedded systems. EURASIP J. Embed. Syst. 07, 1, 1--6.

Digital Library

[39]

Q. Qiu and M. Pedram. 1999. Dynamic power management based on continuous-time Markov decision processes. In Proceedings of the 36th Annual ACM/IEEE Design Automation Conference. 555--561.

Digital Library

[40]

Z. Ren, B. H. Krogh, and R. Marculescu. 2005. Hierarchical adaptive dynamic power management. IEEE Trans. Comput. 54, 4, 409--420.

Digital Library

[41]

H. C. Shih and K. Wang. 2012. An adaptive hybrid dynamic power management algorithm for mobile devices. Comput. Netw. 56, 2, 548--565.

Digital Library

[42]

T. Simunic, L. Benini, P. Glynn, and G. De Micheli. 2000. Dynamic power management for portable systems. In Proceedings of the 6th International Conference on Mobile Computing and Networking. 11--19.

Digital Library

[43]

T. Simunic, L. Benini, P. Glynn, and G. De Micheli. 2001. Event-driven power management. IEEE Trans. Comput.-Aid. Des. Integ. Circuits Syst. 20, 7, 840--857.

Digital Library

[44]

M. B. Srivastava, A. P. Chandrakasan, and R. W. Brodersen. 1996. Predictive system shutdown and other architectural techniques for energy efficient programmable computation. IEEE Trans. Integ. VLSI Syst. 4, 1, 42--55.

Digital Library

[45]

R. S. Sutton. 1990. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the International Conference on Machine Learning. 216--224.

Digital Library

[46]

Y. Tan, W. Liu, and Q. Qiu. 2009. Adaptive power management using reinforcement learning. In Proceedings of the International Conference on Computer-Aided Design. 461--467.

Digital Library

[47]

Y. Wang, Q. Xie, A. Ammari, and M. Pedram. 2011b. Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classification. In Proceedings of the Design Automation Conference. 41--46.

Digital Library

[48]

Z. Wang, H. Wang, X. Chen, and J. Lin. 2011a. Cyber physical systems. J. Chinese Comput. Syst. 32, 5, 881--886.

[49]

C. J. C. H. Watkins and P. Dayan. 2011. Q-Learning. Machine Learn. 8, 3--4, 279--292.

Digital Library

[50]

H. Young, K. Sung, and C. Ki-Seok. 2010. A predictive dynamic power management technique for embedded mobile devices. IEEE Trans. Consumer Electron. 56, 2, 713--719.

Digital Library

[51]

A. Zafra, E. L. Gibaja, and S. Ventura. 2011. Multiple instance learning with multiple objective genetic programming for Web mining. Appl. Soft Comput. 11, 1, 93--102.

Digital Library

Cited By

Fahim APapalexakis EKrishnamurthy SK. Roy Chowdhury AKaplan LAbdelzaher T(2023)AcTrak: Controlling a Steerable Surveillance Camera using Reinforcement LearningACM Transactions on Cyber-Physical Systems10.1145/35853167:2(1-27)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3585316
Pandey PRahmati MBajwa WPompili D(2023)Real-Time In-Network Image Compression via Distributed Dictionary LearningIEEE Transactions on Mobile Computing10.1109/TMC.2021.307206622:1(472-486)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TMC.2021.3072066
Agyeman RRinner B(2022)Resource-efficient Pervasive Smart Camera Networks2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops53856.2022.9767290(503-508)Online publication date: 21-Mar-2022
https://doi.org/10.1109/PerComWorkshops53856.2022.9767290
Show More Cited By

Index Terms

Online learning of timeout policies for dynamic power management

Recommendations

A Reinforcement Learning Framework for Dynamic Power Management of a Portable, Multi-camera Traffic Monitoring System
GREENCOM '12: Proceedings of the 2012 IEEE International Conference on Green Computing and Communications

Dynamic Power Management (DPM) refers to a set of strategies that achieves efficient power consumption by selectively turning off (or reducing the performance of) a system components when they are idle or are serving light workloads. This paper presents ...
Reinforcement learning based dynamic power management with a hybrid power supply
ICCD '12: Proceedings of the 2012 IEEE 30th International Conference on Computer Design (ICCD 2012)

Dynamic power management (DPM) in battery-powered mobile systems attempts to achieve higher energy efficiency by selectively setting idle components to a sleep state. However, re-activating these components at a later time consumes a large amount of ...
System-level power management using online learning

In this paper, we propose a novel online-learning algorithm for system-level power management. We formulate both dynamic power management (DPM) and dynamic voltage-frequency scaling problems as one of workload characterization and selection and solve ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Embedded Computing Systems

ACM Transactions on Embedded Computing Systems Volume 13, Issue 4

Regular Papers

November 2014

647 pages

ISSN:1539-9087

EISSN:1558-3465

DOI:10.1145/2592905

Editor:
Sandeep K. Shukla
Virginia Tech, USA

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 10 March 2014

Accepted: 01 August 2013

Revised: 01 June 2013

Received: 01 February 2013

Published in TECS Volume 13, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Österreichische Forschungsförderungsgesellschaft

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
455
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fahim APapalexakis EKrishnamurthy SK. Roy Chowdhury AKaplan LAbdelzaher T(2023)AcTrak: Controlling a Steerable Surveillance Camera using Reinforcement LearningACM Transactions on Cyber-Physical Systems10.1145/35853167:2(1-27)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3585316
Pandey PRahmati MBajwa WPompili D(2023)Real-Time In-Network Image Compression via Distributed Dictionary LearningIEEE Transactions on Mobile Computing10.1109/TMC.2021.307206622:1(472-486)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TMC.2021.3072066
Agyeman RRinner B(2022)Resource-efficient Pervasive Smart Camera Networks2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops53856.2022.9767290(503-508)Online publication date: 21-Mar-2022
https://doi.org/10.1109/PerComWorkshops53856.2022.9767290
Majzoub SSaleh RTaouil MHamdioui SBamakhrama M(2022)Rapid Design-Space Exploration for Low-Power Manycores Under Process Variation Utilizing Machine LearningIEEE Access10.1109/ACCESS.2022.318714010(70187-70203)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3187140
Rodriguez-Zurrunero RAraujo A(2021)Adaptive frequency scaling strategy to improve energy efficiency in a tick-less Operating System for resource-constrained embedded devicesFuture Generation Computer Systems10.1016/j.future.2021.05.038124:C(230-242)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1016/j.future.2021.05.038
Costero LIranfar AZapater MIgual FOlcoz KAtienza D(2020)Resource Management for Power-Constrained HEVC Transcoding Using Reinforcement LearningIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2020.300473531:12(2834-2850)Online publication date: 1-Dec-2020
https://doi.org/10.1109/TPDS.2020.3004735
Huang HLin MYang LZhang Q(2020) Autonomous Power Management With Double- Q Reinforcement Learning Method IEEE Transactions on Industrial Informatics10.1109/TII.2019.295393216:3(1938-1946)Online publication date: Mar-2020
https://doi.org/10.1109/TII.2019.2953932
Costero LIranfar AZapater MIgual FOlcoz KAtienza D(2019)MAMUT: Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-User Video Transcoding2019 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE.2019.8715256(558-563)Online publication date: Mar-2019
https://doi.org/10.23919/DATE.2019.8715256
Moazzemi KMaity BYi SRahmani ADutt N(2019)HESSLE-FREEACM Transactions on Embedded Computing Systems10.1145/335820318:5s(1-19)Online publication date: 8-Oct-2019
https://dl.acm.org/doi/10.1145/3358203
Marz SZanden BGao W(2019)Reducing Event Latency and Power Consumption in Mobile Devices by Using a Kernel-Level Display ServerIEEE Transactions on Mobile Computing10.1109/TMC.2018.285780918:5(1174-1187)Online publication date: 1-May-2019
https://doi.org/10.1109/TMC.2018.2857809
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents