research-article

AutoScale: Dynamic, Robust Capacity Management for Multi-Tier Data Centers

Authors:

Mor Harchol-Balter,

Ram Raghunathan,

Michael A. KozuchAuthors Info & Claims

ACM Transactions on Computer Systems (TOCS), Volume 30, Issue 4

Article No.: 14, Pages 1 - 26

https://doi.org/10.1145/2382553.2382556

Published: 01 November 2012 Publication History

Abstract

Energy costs for data centers continue to rise, already exceeding $15 billion yearly. Sadly much of this power is wasted. Servers are only busy 10--30% of the time on average, but they are often left on, while idle, utilizing 60% or more of peak power when in the idle state.

We introduce a dynamic capacity management policy, AutoScale, that greatly reduces the number of servers needed in data centers driven by unpredictable, time-varying load, while meeting response time SLAs. AutoScale scales the data center capacity, adding or removing servers as needed. AutoScale has two key features: (i) it autonomically maintains just the right amount of spare capacity to handle bursts in the request rate; and (ii) it is robust not just to changes in the request rate of real-world traces, but also request size and server efficiency.

We evaluate our dynamic capacity management approach via implementation on a 38-server multi-tier data center, serving a web site of the type seen in Facebook or Amazon, with a key-value store workload. We demonstrate that AutoScale vastly improves upon existing dynamic capacity management policies with respect to meeting SLAs and robustness.

References

[1]

Amazon Inc. 2008. Amazon Elastic Compute Cloud (Amazon EC2).

[2]

Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., Konwinski, A., Lee, G., Patterson, D. A., Rabkin, A., Stoica, I., and Zaharia, M. 2009. Above the clouds: A Berkeley view of cloud computing. Tech. rep. UCB/EECS-2009-28, EECS Department, University of California, Berkeley.

[3]

Barroso, L. A. and Hölzle, U. 2007. The case for energy-proportional computing. Computer 40, 12, 33--37.

Digital Library

[4]

Bobroff, N., Kochut, A., and Beaty, K. 2007. Dynamic placement of virtual machines for managing SLA violations. In Proceedings of the 10th IFIP/IEEE International Symposium on Integrated Network Management (IM’07). 119--128.

[5]

Bodík, P., Griffith, R., Sutton, C., Fox, A., Jordan, M., and Patterson, D. 2009. Statistical machine learning makes automatic control practical for internet datacenters. In Proceedings of the 2009 Conference on Hot Topics in Cloud Computing (HotCloud’09).

Digital Library

[6]

Castellanos, M., Casati, F., Shan, M.-C., and Dayal, U. 2005. iBOM: A platform for intelligent business operation management. In Proceedings of the 21st International Conference on Data Engineering (ICDE’05). 1084--1095.

Digital Library

[7]

Chase, J. S., Anderson, D. C., Thakar, P. N., and Vahdat, A. M. 2001. Managing energy and server resources in hosting centers. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP’01). 103--116.

Digital Library

[8]

Chen, G., He, W., Liu, J., Nath, S., Rigas, L., Xiao, L., and Zhao, F. 2008. Energy-aware server provisioning and load dispatching for connection-intensive internet services. In Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation (NSDI’08). 337--350.

Digital Library

[9]

Chen, Y., Das, A., Qin, W., Sivasubramaniam, A., Wang, Q., and Gautam, N. 2005. Managing server energy and operational costs in hosting centers. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS’05). 303--314.

Digital Library

[10]

DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., and Vogels, W. 2007. Dynamo: Amazon’s highly available key-value store. In Proceedings of 21st ACM SIGOPS Symposium on Operating Systems Principles (SOSP’07). 205--220.

Digital Library

[11]

Elnozahy, E., Kistler, M., and Rajamony, R. 2002. Energy-efficient server clusters. In Proceedings of the 2nd Workshop on Power-Aware Computing Systems (WPACS’02). 179--196.

Digital Library

[12]

Facebook. 2011. Personal communication with Facebook.

[13]

Fan, X., Weber, W.-D., and Barroso, L. A. 2007. Power provisioning for a warehouse-sized computer. In Proceedings of the 34th Annual International Symposium on Computer Architecture (ISCA’07). 13--23.

Digital Library

[14]

Gandhi, A., Chen, Y., Gmach, D., Arlitt, M., and Marwah, M. 2011a. Minimizing data center SLA violations and power consumption via hybrid resource provisioning. In Proceedings of the 2nd International Green Computing Conference (IGCC’11).

Digital Library

[15]

Gandhi, A., Harchol-Balter, M., and Kozuch, M. A. 2011b. The case for sleep states in servers. In Proceedings of the 4th Workshop on Power-Aware Computing and Systems (HotPower’11).

Digital Library

[16]

Gandhi, N., Tilbury, D., Diao, Y., Hellerstein, J., and Parekh, S. 2002. MIMO control of an Apache web server: Modeling and controller design. In Proceedings of the 2002 American Control Conference (ACC’02 Series, vol. 6). 4922--4927.

[17]

Gmach, D., Krompass, S., Scholz, A., Wimmer, M., and Kemper, A. 2008. Adaptive quality of service management for enterprise services. ACM Trans. Web 2, 1, 1--46.

Digital Library

[18]

Grunwald, D., Morrey III, C. B., Levis, P., Neufeld, M., and Farkas, K. I. 2000. Policies for dynamic clock scheduling. In Proceedings of the 4th Conference on Symposium of Operating System Design and Implementation (OSDI’00).

Digital Library

[19]

Hoffmann, H., Sidiroglou, S., Carbin, M., Misailovic, S., Agarwal, A., and Rinard, M. 2011. Dynamic knobs for responsive power-aware computing. In Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). 199--212.

Digital Library

[20]

Horvath, T. and Skadron, K. 2008. Multi-mode energy management for multi-tier server clusters. In Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques (PACT’08). 270--279.

Digital Library

[21]

ita. 1998. The Internet Traffic Archives: WorldCup98. http://ita.ee.lbl.gov/html/contrib/WorldCup.html.

[22]

Iyer, S. and Druschel, P. 2001. Anticipatory scheduling: A disk scheduling framework to overcome deceptive idleness in synchronous I/O. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP’01). 117--130.

Digital Library

[23]

Kim, J. and Rosing, T. S. 2006. Power-aware resource management techniques for low-power embedded systems. In Handbook of Real-Time and Embedded Systems. Taylor-Francis Group LLC.

[24]

Kivity, A. 2007. KVM: The Linux virtual machine monitor. In Proceedings of the 2007 Ottawa Linux Symposium (OLS’07). 225--230.

[25]

Kleinrock, L. 1975. Queueing Systems, Volume I: Theory. Wiley-Interscience.

[26]

Krioukov, A., Mohan, P., Alspaugh, S., Keys, L., Culler, D., and Katz, R. 2010. NapSAC: Design and implementation of a power-proportional web cluster. In Proceedings of the 1st ACM SIGCOMM Workshop on Green Networking (Green Networking’10). 15--22.

Digital Library

[27]

Leite, J. C., Kusic, D. M., and Mossé, D. 2010. Stochastic approximation control of power and tardiness in a three-tier web-hosting cluster. In Proceeding of the 7th International Conference on Autonomic Computing (ICAC’10). 41--50.

Digital Library

[28]

Li, B. and Nahrstedt, K. 1999. A control-based middleware framework for quality of service adaptations. IEEE J. Sel. Areas Commun. 17, 1632--1650.

Digital Library

[29]

Lim, S.-H., Sharma, B., Tak, B. C., and Das, C. R. 2011. A dynamic energy management scheme for multi-tier data centers. In Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS’11). 257--266.

Digital Library

[30]

Lu, C., Lu, Y., Abdelzaher, T., Stankovic, J., and Son, S. 2006. Feedback control architecture and design methodology for service delay guarantees in web servers. IEEE Trans. Paral. Distrib. Syst. 17, 9, 1014--1027.

Digital Library

[31]

Lu, Y.-H., Chung, E.-Y., Šimunić, T., Benini, L., and De Micheli, G. 2000. Quantitative comparison of power management algorithms. In Proceedings of the Conference on Design, Automation and Test in Europe (DATE’00). 20--26.

Digital Library

[32]

Meisner, D., Gold, B. T., and Wenisch, T. F. 2009. PowerNap: Eliminating server idle power. In Proceeding of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’09). 205--216.

Digital Library

[33]

Meisner, D., Sadler, C. M., Barroso, L. A., Weber, W.-D., and Wenisch, T. F. 2011. Power management of online data-intensive services. In Proceedings of the 38th Annual International Symposium on Computer Architecture (ISCA’11). 319--330.

Digital Library

[34]

Mosberger, D. and Jin, T. 1998. httperf---A tool for measuring web server performance. ACM Sigmetrics: Perf. Eval. Rev. 26, 3, 31--37.

Digital Library

[35]

Nathuji, R., Kansal, A., and Ghaffarkhah, A. 2010. Q-clouds: Managing performance interference effects for QoS-aware clouds. In Proceedings of the 5th European Conference on Computer Systems, (EuroSys’10). 237--250.

Digital Library

[36]

Newman, M. E. J. 2005. Power laws, Pareto distributions and Zipf’s law. Contemp. Phys. 46, 323--351.

[37]

nlanr. 1995. National Laboratory for Applied Network Research. Anonymized access logs. ftp://ftp.ircache.net/Traces/.

[38]

Pering, T., Burd, T., and Brodersen, R. 1998. The simulation and evaluation of dynamic voltage scaling algorithms. In Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED’98). 76--81.

Digital Library

[39]

Qin, W., and Wang, Q. 2007. Modeling and control design for performance management of web servers via an IPV approach. IEEE Trans. Control Syst. Tech. 15, 2, 259--275.

[40]

sap. 2011. SAP application trace from anonymous source.

[41]

Snyder, B. 2010. Server virtualization has stalled, despite the hype. http://www.infoworld.com/print/146901.

[42]

Urgaonkar, B. and Chandra, A. 2005. Dynamic provisioning of multi-tier internet applications. In Proceedings of the 2nd International Conference on Automatic Computing (ICAC’05). 217--228.

Digital Library

[43]

Urgaonkar, B., Pacifici, G., Shenoy, P., Spreitzer, M., and Tantawi, A. 2005. An analytical model for multi-tier internet services and its applications. In Proceedings of the 2005 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS’05). 291--302.

Digital Library

[44]

Verma, A., Dasgupta, G., Nayak, T. K., De, P., and Kothari, R. 2009. Server workload analysis for power minimization using consolidation. In Proceedings of the 2009 Conference on USENIX Annual Technical Conference (USENIX’09).

Digital Library

[45]

Wang, X. and Chen, M. 2008. Cluster-level feedback power control for performance optimization. In Proceeding of the 14th IEEE International Symposium on High-Performance Computer Architecture (HPCA’08). 101--110.

[46]

Wood, T., Shenoy, P. J., Venkataramani, A., and Yousif, M. S. 2007. Black-box and gray-box strategies for virtual machine migration. In Proceedings of the 4th USENIX Conference on Networked Systems Design and Implementation (NSDI’07). 229--242.

Digital Library

Cited By

Le-Anh TPhung-Duc T(2025)Energy-performance tradeoffs in server farms with batch services and setup timesPerformance Evaluation10.1016/j.peva.2025.102468168(102468)Online publication date: Jun-2025
https://doi.org/10.1016/j.peva.2025.102468
Rutten DMukherjee D(2024)A New Approach to Capacity Scaling Augmented with Unreliable Machine Learning PredictionsMathematics of Operations Research10.1287/moor.2023.136449:1(476-508)Online publication date: Feb-2024
https://doi.org/10.1287/moor.2023.1364
Sharma SJukan AMalik AGumaste A(2024)A Network Calculus Model for SFC Realization and Traffic Bounds Estimation in Data CentersACM Transactions on Internet Technology10.1145/370044024:4(1-32)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3700440
Show More Cited By

Index Terms

AutoScale: Dynamic, Robust Capacity Management for Multi-Tier Data Centers
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
2. General and reference
  1. Cross-computing tools and techniques
    1. Reliability

Recommendations

Engineering simulated evolution for integrated power optimization in data centers

Cloud computing has evolved as the next-generation platform for hosting applications ranging from engineering to sciences, and from social networking to media content delivery. The numerous data centers, employed to provide cloud services, consume large ...
Optimal resource provisioning for cloud computing environment

The paper presents an efficient cloud resource provisioning approach. The Software as a Service (SaaS) provider leases resources from cloud providers and also leases software as services to SaaS users. The SaaS providers aim at minimizing the payment of ...
Resource provisioning with budget constraints for adaptive applications in cloud environments
HPDC '10: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

Utility computing was a vision stated more than 40 years ago. It refers to the idea that computing resources and services can be delivered, utilized, and paid for as utilities such as water or electricity. The recent emergence of cloud computing is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer Systems

ACM Transactions on Computer Systems Volume 30, Issue 4

November 2012

136 pages

ISSN:0734-2071

EISSN:1557-7333

DOI:10.1145/2382553

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2012

Accepted: 01 September 2012

Revised: 01 August 2012

Received: 01 April 2012

Published in TOCS Volume 30, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

229
Total Citations
View Citations
1,906
Total Downloads

Downloads (Last 12 months)80
Downloads (Last 6 weeks)6

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Le-Anh TPhung-Duc T(2025)Energy-performance tradeoffs in server farms with batch services and setup timesPerformance Evaluation10.1016/j.peva.2025.102468168(102468)Online publication date: Jun-2025
https://doi.org/10.1016/j.peva.2025.102468
Rutten DMukherjee D(2024)A New Approach to Capacity Scaling Augmented with Unreliable Machine Learning PredictionsMathematics of Operations Research10.1287/moor.2023.136449:1(476-508)Online publication date: Feb-2024
https://doi.org/10.1287/moor.2023.1364
Sharma SJukan AMalik AGumaste A(2024)A Network Calculus Model for SFC Realization and Traffic Bounds Estimation in Data CentersACM Transactions on Internet Technology10.1145/370044024:4(1-32)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3700440
Hasan RZhu TUrgaonkar B(2024)AutoBurst: Autoscaling Burstable Instances for Cost-effective Latency SLOsProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698530(243-258)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698530
Gu XWang QYan QLiu JPu CQuek TGao DZhou JCardenas A(2024)Sync-Millibottleneck Attack on Microservices Cloud ArchitectureProceedings of the 19th ACM Asia Conference on Computer and Communications Security10.1145/3634737.3644991(799-813)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3634737.3644991
Wang JBerger DKazhamiaka FIrvene CZhang CChoukse EFrost KFonseca RWarrier BBansal CStern JBianchini RSriraman A(2024)Designing Cloud Servers for Lower Carbon2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00041(452-470)Online publication date: 29-Jun-2024
https://doi.org/10.1109/ISCA59077.2024.00041
Gu XWang QLiu JWei J(2024)Grunt Attack: Exploiting Execution Dependencies in Microservices2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN58291.2024.00025(115-128)Online publication date: 24-Jun-2024
https://doi.org/10.1109/DSN58291.2024.00025
Hossen MSochat VSarkar AIslam MMilroy D(2024)Enabling Workload-Driven Elasticity in MPI-based Ensembles2024 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER59578.2024.00029(250-262)Online publication date: 24-Sep-2024
https://doi.org/10.1109/CLUSTER59578.2024.00029
Liang YRuan NYi LSu X(2024)An approach to workload generation for modern data centers: A view from Alibaba traceBenchCouncil Transactions on Benchmarks, Standards and Evaluations10.1016/j.tbench.2024.1001644:1(100164)Online publication date: Mar-2024
https://doi.org/10.1016/j.tbench.2024.100164
Cotroneo DNatella RRosiello S(2024)DRACO: Distributed Resource-aware Admission Control for large-scale, multi-tier systemsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2024.104935192(104935)Online publication date: Oct-2024
https://doi.org/10.1016/j.jpdc.2024.104935
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents