research-article

Agile dynamic provisioning of multi-tier Internet applications

Authors:

Bhuvan Urgaonkar,

Prashant Shenoy,

Abhishek Chandra,

Timothy WoodAuthors Info & Claims

ACM Transactions on Autonomous and Adaptive Systems (TAAS), Volume 3, Issue 1

Article No.: 1, Pages 1 - 39

https://doi.org/10.1145/1342171.1342172

Published: 27 March 2008 Publication History

Abstract

Dynamic capacity provisioning is a useful technique for handling the multi-time-scale variations seen in Internet workloads. In this article, we propose a novel dynamic provisioning technique for multi-tier Internet applications that employs (1) a flexible queuing model to determine how much of the resources to allocate to each tier of the application, and (2) a combination of predictive and reactive methods that determine when to provision these resources, both at large and small time scales. We propose a novel data center architecture based on virtual machine monitors to reduce provisioning overheads. Our experiments on a forty-machine Xen/Linux-based hosting platform demonstrate the responsiveness of our technique in handling dynamic workloads. In one scenario where a flash crowd caused the workload of a three-tier application to double, our technique was able to double the application capacity within five minutes, thus maintaining response-time targets. Our technique also reduced the overhead of switching servers across applications from several minutes to less than a second, while meeting the performance targets of residual sessions.

References

[1]

Abdelzaher, T. and Bhatti, N. 1999. Web content adaptation to improve server overload behavior. In Proceedings of the World Wide Web Conference (WWW8). Tornoto.

Digital Library

[2]

Abdelzaher, T., Shin, K. G., and Bhatti, N. 2002. Performance guarantees for Web server end-systems: A control-theoretical approach. IEEE Trans. Para. Distrib. Syst. 13, 1 (Jan.).

Digital Library

[3]

AMAZON. 2000. The Holiday Shopping Season, So Far. http://www.fool.com/news/2000/wmt001127.htm.

[4]

Appleby, K., Fakhouri, S., Fong, L., Goldzmidt, M. K. G., Krishnakumar, S., Pazel, D., Pershing, J., and Rochwerger, B. 2001. Oceano---SLA-based management of a computing utility. In Proceedings of the IFIP/IEEE Symposium on Integrated Network Management.

[5]

Arlitt, M. and Jin, T. 1999. Workload characterization of the 1998 World Cup Web site. Tech. Rep. HPL-1999-35R1, HP Labs.

[6]

Aron, M., Druschel, P., and Zwaenepoel, W. 2000. Cluster reserves: A mechanism for resource management in cluster-based network servers. In Proceedings of the ACM SIGMETRICS Conference. Santa Clara, CA.

Digital Library

[7]

Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebuer, R., Pratt, I., and Warfield, A. 2003. Xen and the art of virtulization. In Proceedings of the Nineteenth Symposium on Operating Systems Principles (SOSP).

Digital Library

[8]

Benani, M. and Menasce, D. 2005. Resource allocation for autonomic data centers using analytic performance models. In Proceedings of the IEEE International Conference on Autonomic Computing, Seattle (ICAC-05). WA.

Digital Library

[9]

Chandra, A., Gong, W., and Shenoy, P. 2003. Dynamic resource allocation for shared data centers using online measurements. In Proceedings of the Eleventh International Workshop on Quality of Service (IWQoS 2003). Monterey, CA.

Digital Library

[10]

Chase, J. and Doyle, R. 2001. Balance of power: Energy management for server clusters. In Proceedings of the Eighth Workshop on Hot Topics in Operating Systems (HotOS-VIII). Elmau, Germany.

Digital Library

[11]

Chase, J., Grit, L., Irwin, D., Moore, J., and Sprenkle, S. 2003. Dynamic virtual clusters in a grid site manager. In Twelfth International Symposium on High Performance Distributed Computing (HPDC-12).

Digital Library

[12]

Cherkasova, L. and Phaal, P. 1999. Session-based admission control: A mechanism for improving performance of commercial Web sites. In Proceedings of the Seventh International Workshop on Quality of Service, IEEE/IFIP Event. London.

[13]

Cohen, I., Chase, J., Goldszmidt, M., Kelly, T., and Symons, J. 2004. Correlating instrumentation data to system states: A building block for automated diagnosis and control. In Proceedings of the Sixth USENIX Symposium in Operating Systems Design and Implementation (OSDI 2004). San Francisco, CA.

Digital Library

[14]

Doyle, R., Chase, J., Asad, O., Jin, W., and Vahdat, A. 2003. Model-based resource provisioning in a Web service utility. In Proceedings of the Fourth USITS.

Digital Library

[15]

DYNASERVER. 2005. Dynaserver project. http://compsci.rice.edu/CS/Systems/DynaServer/.

[16]

Elnikety, S., Nahum, E., Tracey, J., and Zwaenepoel, W. 2004. A method for transparent admission control and request scheduling in e-commerce Web sites. In Proceedings of the Thirteenth International Conference on the World Wide Web. New York, NY. 276--286.

Digital Library

[17]

Fox, A., Gribble, S., Chawathe, Y., Brewer, E., and Gauthier, P. 1997. Cluster-based scalable network services. In Proceedings of the Sixteenth Symposium on Operating Systems Principles (SOSP'97).

Digital Library

[18]

Franks, R. G. 1999. Performance analysis of distributed server systems. Ph.D. Dissertation, Carleton University.

Digital Library

[19]

Goldberg, R. 1974. Survey of virtual machine research. IEEE Comput. 34--45.

[20]

Hellerstein, J., Zhang, F., and Shahabuddin, P. 1999. An approach to predictive detection for service management. In Proceedings of the IEEE International Conference on Systems and Network Management.

[21]

Iyer, R., Tewari, V., and Kant, K. 2000. Overload control mechanisms for Web servers. In Workshop on Performance and QoS of Next Generation Networks.

[22]

Jamjoom, H., Reumann, J., and Shin, K. 2000. QGuard: Protecting Internet servers from overload. Tech. rep., CSE-TR-427-00, Department of Computer Science, University of Michigan.

[23]

Kamra, A., Misra, V., and Nahum, E. 2004. Yaksha: A controller for managing the performance of 3-tiered Websites. In Proceedings of the Twelfth IWQoS.

[24]

Kanodia, V. and Knightly, E. 2000. Multi-class latency-bounded Web servers. In Proceedings of International Workshop on Quality of Service (IWQoS'00).

[25]

Kleinrock, L. 1976. Queueing Systems, Volume 2: Computer Applications. John Wiley and Sons, Inc.

[26]

Kounev, S. and Buchmann, A. 2003. Performance modeling and evaluation of large-scale J2EE applications. In Proceedings of the Computer Measurement Group's 2003 International Conference (CMG 2003). Dallas, TX.

[27]

Knightly, E. and Shroff, N. 1999. Admission control for statistical QoS: Theory and practice. In IEEE Netw. 13, 2, 20--29.

Digital Library

[28]

KTCPVS. 2005. Kernel TCP Virtual Server. http://www.linuxvirtualserver.org/software/ktcpvs/ktcpvs.html.

[29]

Levy, R., Nagarajarao, J., Pacifici, G., Spreitzer, M., Tantawi, A., and Youssef, A. 2003. Performance management for cluster based Web services. In IFIP/IEEE Eighth International Symposium on Integrated Network Management. vol. 246, 247--261.

[30]

Li, S. and Jamin, S. 2000. A Measurement-based admission-controlled Web server. In Proceedings of the Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2000). Tel Aviv, Israel.

[31]

Liu, T.-K., Kumaran, S., and Luo, Z. 2001. Layered queueing models for enterprise Java Beans applications. Tech. rep., IBM. (June).

[32]

Menasce, D. 2003. Web server software architectures. In IEEE Internet Comput. vol. 7.

Digital Library

[33]

ORACLE9I. 2005. Oracle9i. http://www.oracle.com/technology/products/oracle9i.

[34]

Pai, V., Aron, M., Banga, G., Svendsen, M., Druschel, P., Zwanepoel, W., and Nahum, E. 1998. Locality-aware request distribution in cluster-based network servers. In Proceedings of the Eighth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-VIII). San Jose, CA.

Digital Library

[35]

Ranjan, S., Rolia, J., Fu, H., and Knightly, E. 2002. QoS-driven server migration for Internet data centers. In Proceedings of the Tenth International Workshop on Quality of Service. Miami, FL.

[36]

Rolia, J. and Sevcik, K. 1995. The method of layers. IEEE Trans. Softw. Eng. 21, 8, 689--700.

Digital Library

[37]

Rolia, J., Zhu, X., Arlitt, M., and Andrzejak, A. 2002. Statistical service assurances for applications in utility grid environments. Tech. rep. HPL-2002-155, HP Labs.

[38]

Saito, Y., Bershad, B., and Levy, H. 1999. Manageability, availability and performance in Porcupine: A highly scalable, cluster-based mail service. In Proceedings of the Seventeenth Symposium on Operating Systems Principles (SOSP'99).

Digital Library

[39]

SAR. 2005. Sysstat Package. http://freshmeat.net/projects/sysstat.

[40]

Schroeder, B. and Harchol-Balter, M. 2003. Web servers under overload: How scheduling can help. In Proceedings of the Eighteenth International Teletraffic Congress.

[41]

Shen, K., Tang, H., Yang, T., and Chu, L. 2002. Integrated resource management for cluster-based Internet services. In Proceedings of the Fifth USENIX Symposium on Operating Systems Design and Implementation (OSDI). Boston, MA.

Digital Library

[42]

Slothouber, L. 1996. A model of Web server performance. In Proceedings of the Fifth International World Wide Web Conference.

[43]

Urgaonkar, B., Pacifici, G., Shenoy, P., Spreitzer, M., and Tantawi, A. 2005. An analytical model for multi-tier Internet services and its applications. In Proceedings of the ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS 2005). Banff, Canada.

Digital Library

[44]

Urgaonkar, B. and Shenoy, P. 2004a. Cataclysm: Handling extreme overloads in Internet services. In Proceedings of the Twenty-Third Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing (PODC 2004). St. John's, Newfoundland, Canada.

Digital Library

[45]

Urgaonkar, B. and Shenoy, P. 2004b. Sharc: Managing CPU and network bandwidth in shared clusters. In IEEE Trans. Para. Distrib. Syst. 15, 1, 2--17.

Digital Library

[46]

Urgaonkar, B., Shenoy, P., and Roscoe, T. 2002. Resource overbooking and application profiling in shared hosting platforms. In Proceedings of the Fifth USENIX Symposium on Operating Systems Design and Implementation (OSDI 2002). Boston, MA.

Digital Library

[47]

Verma, A. and Ghosal, S. 2003. On admission control for profit maximization of networked service providers. In Proceedings of the 12th International World Wide Web Conference (WWW2003). Budapest, Hungary.

Digital Library

[48]

Villela, D., Pradhan, P., and Rubenstein, D. 2004. Provisioning servers in the application tier for e-commerce systems. In Proceedings of the Twelfth IWQoS.

[49]

Voigt, T., Tewari, R., Freimuth, D., and Mehra, A. 2001. Kernel mechanisms for service differrentiation in overloaded Web servers. In Proceedings of USENIX Annual Technical Conference.

Digital Library

[50]

Welsh, M. and Culler, D. 2003. Adaptive overload control for busy Internet servers. In Proceedings of the Fourth USENIX Conference on Internet Technologies and Systems (USITS'03).

Digital Library

[51]

Welsh, M., Culler, D., and Brewer, E. 2001. SEDA: An architecture for well-conditioned, scalable Internet services. In Proceedings of the 18th Symposium on Operating Systems Principles (SOSP'01).

Digital Library

[52]

Woodside, C. and Raghunath, G. 1995. General bypass architecture for high-performance distributed algorithms. In Proceedings of the Sixth IFIP Conference on Performance of Computer Networks. Istanbul, Turkey.

Digital Library

[53]

WSLA. http://www.research.ibm.com/wsla. Web service level agreements (wsla) project.

[54]

Xu, J., Oufimtsev, A., Woodside, M., and Murphy, L. 2006. Performance modeling and prediction of enterprise JavaBeans with layered queuing network templates. SIGSOFT Softw. Eng. Notes 31, 2.

Digital Library

Cited By

Sajal SZhu TUrgaonkar BSen S(2024)TraceUpscaler: Upscaling Traces to Evaluate Systems at High LoadProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629581(942-961)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3629581
Zhao GWang JXu HZhao YYang XHuang H(2024)Joint Request Updating and Elastic Resource Provisioning With QoS Guarantee in CloudsIEEE/ACM Transactions on Networking10.1109/TNET.2023.327688132:1(110-126)Online publication date: Feb-2024
https://doi.org/10.1109/TNET.2023.3276881
Jiang YXue JHu KChen TWu T(2024)Saver: a proactive microservice resource scheduling strategy based on STGCNCluster Computing10.1007/s10586-024-04615-zOnline publication date: 1-Jul-2024
https://doi.org/10.1007/s10586-024-04615-z
Show More Cited By

Index Terms

Agile dynamic provisioning of multi-tier Internet applications
1. Software and its engineering
  1. Software creation and management
    1. Software development process management
      1. Software development methods

Recommendations

Performance modeling and analysis of virtualized multi-tier applications under dynamic workloads

Virtual machine technology facilitates implementation of modern Internet services, especially multi-tier applications. Server virtualization aims to reduce the cost of service provisioning and improve fault tolerance, portability and security of ...
Sandpiper: Black-box and gray-box resource management for virtual machines

Virtualization can provide significant benefits in data centers by enabling dynamic virtual machine resizing and migration to eliminate hotspots. We present Sandpiper, a system that automates the task of monitoring and detecting hotspots, determining a ...
An analytical model for multi-tier internet services and its applications
SIGMETRICS '05: Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems

Since many Internet applications employ a multi-tier architecture, in this paper, we focus on the problem of analytically modeling the behavior of such applications. We present a model based on a network of queues, where the queues represent different ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Autonomous and Adaptive Systems

ACM Transactions on Autonomous and Adaptive Systems Volume 3, Issue 1

March 2008

118 pages

ISSN:1556-4665

EISSN:1556-4703

DOI:10.1145/1342171

Issue’s Table of Contents

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2008

Accepted: 01 December 2007

Revised: 01 April 2007

Received: 01 October 2005

Published in TAAS Volume 3, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

295
Total Citations
View Citations
2,118
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sajal SZhu TUrgaonkar BSen S(2024)TraceUpscaler: Upscaling Traces to Evaluate Systems at High LoadProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629581(942-961)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3629581
Zhao GWang JXu HZhao YYang XHuang H(2024)Joint Request Updating and Elastic Resource Provisioning With QoS Guarantee in CloudsIEEE/ACM Transactions on Networking10.1109/TNET.2023.327688132:1(110-126)Online publication date: Feb-2024
https://doi.org/10.1109/TNET.2023.3276881
Jiang YXue JHu KChen TWu T(2024)Saver: a proactive microservice resource scheduling strategy based on STGCNCluster Computing10.1007/s10586-024-04615-zOnline publication date: 1-Jul-2024
https://doi.org/10.1007/s10586-024-04615-z
Gessert FWingerath WRitter NGessert FWingerath WRitter N(2024)Polyglotte Persistenz im DatenmanagementSchnelles und skalierbares Cloud-Datenmanagement10.1007/978-3-031-54388-3_7(161-188)Online publication date: 3-May-2024
https://doi.org/10.1007/978-3-031-54388-3_7
Bacchiani LBravetti MGiallorenzo SMauro JZavattaro G(2024)Integrated Timed Architectural Modeling/Execution LanguageActive Object Languages: Current Research Trends10.1007/978-3-031-51060-1_7(169-198)Online publication date: 29-Jan-2024
https://doi.org/10.1007/978-3-031-51060-1_7
Cui LShi TLu RZhang T(2023)Autoscaling in Mobile Edge Computing Based on Multi-Agent Reinforcement LearningProceedings of the 2023 9th International Conference on Communication and Information Processing10.1145/3638884.3638966(520-527)Online publication date: 14-Dec-2023
https://dl.acm.org/doi/10.1145/3638884.3638966
Wei WWang CGan RSu HLiu J(2023)Adaptive Container Orchestration Mechanism on Electric Power Supercomputing Clouds2023 4th International Symposium on Computer Engineering and Intelligent Communications (ISCEIC)10.1109/ISCEIC59030.2023.10271208(100-103)Online publication date: 18-Aug-2023
https://doi.org/10.1109/ISCEIC59030.2023.10271208
Tong GMeng CSong SPan MYu Y(2023)GMA: Graph Multi-agent Microservice Autoscaling Algorithm in Edge-Cloud Environment2023 IEEE International Conference on Web Services (ICWS)10.1109/ICWS60048.2023.00058(393-404)Online publication date: Jul-2023
https://doi.org/10.1109/ICWS60048.2023.00058
Hegde AKulkarni SPrasad A(2023)COUNSEL: Cloud Resource Configuration Management using Deep Reinforcement Learning2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid57682.2023.00035(286-298)Online publication date: May-2023
https://doi.org/10.1109/CCGrid57682.2023.00035
Meng CSong STong HPan MYu Y(2023)DeepScaler: Holistic Autoscaling for Microservices Based on Spatiotemporal GNN with Adaptive Graph Learning2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00038(53-65)Online publication date: 11-Sep-2023
https://doi.org/10.1109/ASE56229.2023.00038
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents