research-article

Towards energy-aware scheduling in data centers using machine learning

Authors:

Josep Ll. Berral,

Ricard Gavaldà, and

Jordi TorresAuthors Info & Claims

e-Energy '10: Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking

April 2010

Pages 215 - 224

https://doi.org/10.1145/1791314.1791349

Published: 13 April 2010 Publication History

Abstract

As energy-related costs have become a major economical factor for IT infrastructures and data-centers, companies and the research community are being challenged to find better and more efficient power-aware resource management strategies. There is a growing interest in "Green" IT and there is still a big gap in this area to be covered.

In order to obtain an energy-efficient data center, we propose a framework that provides an intelligent consolidation methodology using different techniques such as turning on/off machines, power-aware consolidation algorithms, and machine learning techniques to deal with uncertain information while maximizing performance. For the machine learning approach, we use models learned from previous system behaviors in order to predict power consumption levels, CPU loads, and SLA timings, and improve scheduling decisions. Our framework is vertical, because it considers from watt consumption to workload features, and cross-disciplinary, as it uses a wide variety of techniques.

We evaluate these techniques with a framework that covers the whole control cycle of a real scenario, using a simulation with representative heterogeneous workloads, and we measure the quality of the results according to a set of metrics focused toward our goals, besides traditional policies. The results obtained indicate that our approach is close to the optimal placement and behaves better when the level of uncertainty increases.

References

[1]

Ask.com. http://www.ask.com.

[2]

XtreemOS European Project, 2006--2010. http://www.xtreemos.eu.

[3]

Green Grid Consortium, 2009. http://www.thegreengrid.org.

[4]

Omnet, 2009. http://www.omnet.org.

[5]

The Grid Workloads Archive, 2009. http://gwa.ewi.tudelft.nl.

[6]

K. Appleby, S. Fakhouri, L. Fong, G. Goldszmidt, M. Kalantar, S. Krishnakumar, D. Pazel, J. Pershing, and B. Rochwerger. Oceano-SLA based management of a computing utility. In 7th IFIP/IEEE International Symposium on Integrated Network Management, volume 5. Citeseer, 2001.

[7]

L. A. Barroso and U. Hölzle. The case for energy-proportional computing. Computer, 40(12):33--37, 2007.

Digital Library

[8]

R. Bianchini and R. Rajaniony. Power and energy management for server systems. Computer(Long Beach, CA), 37(11):68--76, 2004.

Digital Library

[9]

J. Chase, D. Anderson, P. Thakar, A. Vahdat, and R. Doyle. Managing energy and server resources in hosting centers. 18th ACM symposium on Operating systems principles (SIGOPS), 35(5):103--116, 2001.

Digital Library

[10]

Y. Chen, A. Das, W. Qin, A. Sivasubramaniam, Q. Wang, and N. Gautam. Managing server energy and operational costs in hosting centers. ACM SIGMETRICS Performance Evaluation Review, 33(1):303--314, 2005.

Digital Library

[11]

B.-G. Chun, G. Iannaccone, G. Iannaccone, R. Katz, L. Gunho, and L. Niccolini. An Energy Case for Hybrid Datacenters. HotPower, 10 2009.

Digital Library

[12]

T. Cortes, C. Franke, Y. Jégou, T. Kielmann, D. Laforenza, B. Matthews, C. Morin, L. P. Prieto, and A. Reinefeld. XtreemOS: a Vision for a Grid Operating System, 2008.

[13]

T. Cortes and R. Nou. AEM prototype, D3.3.6, XtreemOS deliverable, 2008.

[14]

R. Das, G. Tesauro, J. O. Kephart, D. W. Levine, C. Lefurgy, and H. Chan. Autonomic multi-agent management of power and performance in data centers, 2008.

[15]

D. Filani, J. He, S. Gao, M. Rajappa, A. Kumar, R. Shah, and R. Nagappan. Dynamic Data Center Power Management: Trends, Issues and Solutions. Intel Technology Journal, 2008.

[16]

I. Goiri, J. Guitart, and J. Torres. Elastic Management of Tasks in Virtualized Environments. In Proccedings of the XX Jornadas de Paralelismo 2009, pages 671--676, 2009.

[17]

V. Hamscher, U. Schwiegelshohn, A. Streit, and R. Yahyapour. Evaluation of job-scheduling strategies for grid computing. Lecture Notes in Computer Science, pages 191--202, 2000.

Digital Library

[18]

J. Kephart, H. Chan, R. Das, D. Levine, G. Tesauro, F. Rawson, and C. Lefurgy. Coordinating multiple autonomic managers to achieve specified power-performance tradeoffs. In Autonomic Computing, 2007. ICAC'07., pages 24--24, 2007.

Digital Library

[19]

B. Khargharia, S. Hariri, and M. Yousif. Autonomic power and performance management for computing systems. Cluster Computing, 11(2):167--181, 2008.

Digital Library

[20]

C. Lefurgy, K. Rajamani, F. Rawson, W. Felter, M. Kistler, and T. Keller. Energy Management for Commercial Servers. Computer, 36(12):39--48, 2003.

Digital Library

[21]

L. Liu, H. Wang, X. Liu, X. Jin, W. He, Q. Wang, and Y. Chen. GreenCloud: a new architecture for green data center. In 6th international conference industry session on Autonomic computing and communications, pages 29--38. ACM New York, NY, USA, 2009.

Digital Library

[22]

R. Nou. Energy Efficiency: A case study. Technical Report UPC-DAC-RR-CAP-2009-14, Technical University of Catalonia (UPC) - Computer Architecture Department, 2009.

[23]

R. Nou, S. Kounev, F. Julià, and J. Torres. Autonomic QoS control in enterprise Grid environments using online simulation. J. Syst. Softw., 82(3):486--502, 2009.

Digital Library

[24]

V. Petrucci, O. Loques, and D. Mossé. A dynamic configuration model for power-efficient virtualized server clusters. In 11th Brazillian Workshop on Real-Time and Embedded Systems (WTR), 2009.

[25]

V. Petrucci, O. Loques, B. Niteroi, and D. Mossé. Dynamic configuration support for power-aware virtualized server clusters. In WiP Session of the 21th Euromicro Conference on Real-Time Systems. Dublin, Ireland, 2009.

[26]

E. Pinheiro, R. Bianchini, E. Carrera, and T. Heath. Load balancing and unbalancing for power and performance in cluster-based systems. In Workshop on Compilers and Operating Systems for Low Power, volume 180, pages 182--195. Citeseer, 2001.

[27]

S. Ranjan, J. Rolia, H. Fu, and E. Knightly. Qos-driven server migration for internet data centers. In 10th International Workshop on Quality of Service (IWQoS 2002), pages 3--12. Citeseer, 2002.

[28]

S. Rivoire, M. Shah, P. Ranganathan, and C. Kozyrakis. JouleSort: a balanced energy-efficiency benchmark. In 2007 ACM SIGMOD international conference on Management of data, page 376, 2007.

Digital Library

[29]

K. Shen, H. Tang, T. Yang, and L. Chu. Integrated resource management for cluster-based internet services. SIGOPS OS Rev., 36(SI):225--238, 2002.

Digital Library

[30]

G. Tesauro. Reinforcement learning in autonomic computing: A manifesto and case studies. IEEE Internet Computing, 11(1):22--30, 2007.

Digital Library

[31]

G. Tesauro, R. Das, H. Chan, J. Kephart, D. Levine, F. Rawson, and C. Lefurgy. Managing power consumption and performance of computing systems using reinforcement learning. Advances in Neural Information Processing Systems, 20, 2007.

[32]

D. Vengerov. A reinforcement learning approach to dynamic resource allocation. Eng. Appl. Artif. Intell., 20(3):383--390, 2007.

Digital Library

[33]

A. Verma, P. Ahuja, and A. Neogi. Power-aware dynamic placement of hpc applications. In ICS '08: International Conference on Supercomputing, pages 175--184, New York, NY, USA, 2008. ACM.

Digital Library

[34]

A. Verma, G. Dasgupta, T. Kumar, N. Pradipta, and D. R. Kothari. Server workload analysis for power minimization using consolidation, 2009.

[35]

I. H. Witten and E. Frank. Data mining: practical machine learning tools and techniques with java implementations. SIGMOD Rec., 31(1):76--77, 2002.

Digital Library

Cited By

Selvakumar SSubramanian P(2024)Intelligent and metaheuristic task scheduling for cloud using black widow optimization algorithmSerbian Journal of Electrical Engineering10.2298/SJEE2401053S21:1(53-71)Online publication date: 2024
https://doi.org/10.2298/SJEE2401053S
Shahnejat Bushehri AAmirnia ABelkhiri AKeivanpour Sde Magalhães FNicolescu G(2024)Deep Learning-Driven Anomaly Detection for Green IoT Edge NetworksIEEE Transactions on Green Communications and Networking10.1109/TGCN.2023.33353428:1(498-513)Online publication date: Mar-2024
https://doi.org/10.1109/TGCN.2023.3335342
Singh VKaushik V(2023)A Brief Overview of Cyber Security Advances and Techniques Along With a Glimpse on Quantum CryptographyExploring Cyber Criminals and Data Privacy Measures10.4018/978-1-6684-8422-7.ch003(40-64)Online publication date: 30-Jun-2023
https://doi.org/10.4018/978-1-6684-8422-7.ch003
Show More Cited By

Index Terms

Towards energy-aware scheduling in data centers using machine learning

Recommendations

Rack aware scheduling in HPC data centers: an energy conservation strategy

Energy consumption in high performance computing data centers has become a long standing issue. With rising costs of operating the data center, various techniques need to be employed to reduce the overall energy consumption. Currently, among others ...
Read More
Minimizing SLA violation and power consumption in Cloud data centers using adaptive energy-aware algorithms
Abstract
In this paper, we address the problem of reducing Cloud datacenter high energy consumption with minimal Service Level Agreement (SLA) violation. Although there are many energy-aware resource management solutions for Cloud datacenters, existing ...
Highlights
- Addressed the problem of reducing Cloud datacenter high energy consumption with minimal Service Level Agreement (SLA) violation.
- We propose two novel adaptive energy-aware algorithms for maximizing energy efficiency and minimizing SLA ...
Read More
Energy Aware Resource Scheduling Algorithm for Data Center Using Reinforcement Learning
ICICTA '12: Proceedings of the 2012 Fifth International Conference on Intelligent Computation Technology and Automation

More and more attention is paid for energy consumption aware and power control for data center with the emergency of energy crisis. The use of virtualization technology makes it possible for dynamic configuration of data center resources. The N:1 ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

e-Energy '10: Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking

April 2010

239 pages

ISBN:9781450300421

DOI:10.1145/1791314

General Chairs:
Randy Katz
UC Berkeley
,
David Hutchison
Lancaster University, U.K.

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

IFIP TC6
University of Passau

In-Cooperation

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 April 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

e-Energy '10

Sponsor:

e-Energy '10: 1st Int'l Conf. on Energy-Efficient Computing and Networking

April 13 - 15, 2010

Passau, Germany

Acceptance Rates

Overall Acceptance Rate 160 of 446 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

169
Total Citations
View Citations
2,678
Total Downloads

Downloads (Last 12 months)123
Downloads (Last 6 weeks)17

Other Metrics

View Author Metrics

Citations

Cited By

Selvakumar SSubramanian P(2024)Intelligent and metaheuristic task scheduling for cloud using black widow optimization algorithmSerbian Journal of Electrical Engineering10.2298/SJEE2401053S21:1(53-71)Online publication date: 2024
https://doi.org/10.2298/SJEE2401053S
Shahnejat Bushehri AAmirnia ABelkhiri AKeivanpour Sde Magalhães FNicolescu G(2024)Deep Learning-Driven Anomaly Detection for Green IoT Edge NetworksIEEE Transactions on Green Communications and Networking10.1109/TGCN.2023.33353428:1(498-513)Online publication date: Mar-2024
https://doi.org/10.1109/TGCN.2023.3335342
Singh VKaushik V(2023)A Brief Overview of Cyber Security Advances and Techniques Along With a Glimpse on Quantum CryptographyExploring Cyber Criminals and Data Privacy Measures10.4018/978-1-6684-8422-7.ch003(40-64)Online publication date: 30-Jun-2023
https://doi.org/10.4018/978-1-6684-8422-7.ch003
Gupta SCampos Zeballos Jdel Río Castro GTomičić AAndrés Morales SMahfouz MOsemwegie IPhemia Comlan Sessi VSchmitz MMahmoud NInyaregh M(2023)Operationalizing Digitainability: Encouraging Mindfulness to Harness the Power of Digitalization for Sustainable DevelopmentSustainability10.3390/su1508684415:8(6844)Online publication date: 18-Apr-2023
https://doi.org/10.3390/su15086844
Kocot BCzarnul PProficz J(2023)Energy-Aware Scheduling for High-Performance Computing Systems: A SurveyEnergies10.3390/en1602089016:2(890)Online publication date: 12-Jan-2023
https://doi.org/10.3390/en16020890
Bajpai GSingh PAgarwal A(2023)A Comprehensive Review on Autonomous Consolidation of Virtual Machine for Energy and Resource ManagementProceedings of the 5th International Conference on Information Management & Machine Intelligence10.1145/3647444.3647901(1-7)Online publication date: 23-Nov-2023
https://dl.acm.org/doi/10.1145/3647444.3647901
Howind SSauter T(2023)Modeling Energy Consumption of Industrial Processes with Seq2Seq Machine Learning2023 IEEE 32nd International Symposium on Industrial Electronics (ISIE)10.1109/ISIE51358.2023.10228118(1-4)Online publication date: 19-Jun-2023
https://doi.org/10.1109/ISIE51358.2023.10228118
Nayak PSingh RKushwaha SBevara PKumar VMedara R(2023)Efficient task scheduling on the cloud using artificial neural network and particle swarm optimizationConcurrency and Computation: Practice and Experience10.1002/cpe.795436:6Online publication date: 20-Nov-2023
https://doi.org/10.1002/cpe.7954
Patra SGoswami V(2022)Performance Enhancement of Cloud Datacenters Through Replicated Database ServerJournal of Information Technology Research10.4018/JITR.29994815:1(1-23)Online publication date: 1-Jan-2022
https://doi.org/10.4018/JITR.299948
Hijji MAhmad BAlam GAlwakeel AAlwakeel MAbdulaziz Alharbi LAljarf AKhan M(2022)Cloud Servers: Resource Optimization Using Different Energy Saving TechniquesSensors10.3390/s2221838422:21(8384)Online publication date: 1-Nov-2022
https://doi.org/10.3390/s22218384
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents