short-paper

Predicting cloud resource utilization

Authors:

Michael Borkowski,

Stefan Schulte,

Christoph HochreinerAuthors Info & Claims

UCC '16: Proceedings of the 9th International Conference on Utility and Cloud Computing

Pages 37 - 42

https://doi.org/10.1145/2996890.2996907

Published: 06 December 2016 Publication History

Abstract

A major challenge in Cloud computing is resource provisioning for computational tasks. Not surprisingly, previous work has established a number of solutions to provide Cloud resources in an efficient manner. However, in order to realize a holistic resource provisioning model, a prediction of the future resource consumption of upcoming computational tasks is necessary. Nevertheless, the topic of prediction of Cloud resource utilization is still in its infancy stage.

In this paper, we present an approach for predicting Cloud resource utilization on a per-task and per-resource level. For this, we apply machine learning-based prediction models. Based on extensive evaluation, we show that we can reduce the prediction error by 20% in a typical case, and improvements above 89% are among the best cases.

References

[1]

L. Agostinho, G. Feliciano, L. Olivi, E. Cardozo, and E. Guimarães. A Bio-inspired Approach to Provisioning of Virtual Resources in Federated Clouds. In 9th Int. Conf. on Dependable, Autonomic and Secure Comp., pages 598--604. IEEE, 2011.

Digital Library

[2]

T. W. Anderson. An introduction to multivariate statistical analysis. Wiley, 1962.

[3]

C. M. Bishop. Neural networks for pattern recognition, volume 92. Oxford university press, 1995.

Digital Library

[4]

C. M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Secaucus, NJ, USA, 2006.

Digital Library

[5]

L. Bottou. Large-Scale Machine Learning with Stochastic Gradient Descent. In Int. Conf. on Comp. Statistics (COMPSTAT), pages 177--186, 2010.

[6]

E. Caron, F. Desprez, and A. Muresan. Forecasting for grid and cloud computing on-demand resources based on pattern matching. In Int. Conf. on Cloud Comp. Techn. and Science (CloudCom), pages 456--463, 2010.

Digital Library

[7]

X. Dutreilh, S. Kirgizov, O. Melekhova, J. Malenfant, N. Rivierre, and I. Truck. Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In Int. Conf. on Autonomic and Autonomous Systems (ICAS), pages 67--74, 2011.

[8]

G. Galante and L. C. E. de Bona. A Survey on Cloud Computing Elasticity. In Int. Conf. on Utility and Cloud Comp., pages 263--270. IEEE, 2012.

Digital Library

[9]

A. Gandhi, Y. Chen, D. Gmach, M. Arlitt, and M. Marwah. Hybrid resource provisioning for minimizing data center SLA violations and power consumption. Sustainable Comp.: Informatics and Systems, 2(2):91--104, 2012.

[10]

X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforward neural networks. In Int. Conf. on Artificial Intelligence and Statistics (AISTATS), volume 9, pages 249--256, 2010.

[11]

R. Hecht-Nielsen. Theory of the Backpropagation Neural Network. Int. Joint Conf. On Neural Networks (IJCNN), 1:593--605, 1989.

[12]

C. Hochreiner, M. Vögler, S. Schulte, and S. Dustdar. Elastic Stream Processing for the Internet of Things. In Int. Conf. on Cloud Comp. (CLOUD), pages 100--107. IEEE, 2016.

[13]

P. Hoenisch, C. Hochreiner, D. Schuller, S. Schulte, J. Mendling, and S. Dustdar. Cost-Efficient Scheduling of Elastic Processes in Hybrid Clouds. In Int. Conf. on Cloud Comp. (CLOUD), pages 17--24, 2015.

Digital Library

[14]

P. Hoenisch, D. Schuller, S. Schulte, C. Hochreiner, and S. Dustdar. Optimization of Complex Elastic Processes. Tran. on Services Comp., 2016.

[15]

P. Hoenisch, I. Weber, S. Schulte, L. Zhu, and A. Fekete. Four-fold Auto-Scaling on a Contemporary Deployment Platform using Docker Containers. In Int. Conf. on Service Oriented Comp. (ICSOC), volume 9435 of Lecture Notes on Computer Science, pages 316--323. Springer, Heidelberg Berlin, 2015.

[16]

S. Islam, J. Keung, K. Lee, and A. Liu. Empirical prediction models for adaptive resource provisioning in the cloud. Future Gen. Comp. Sys., 28(1):155--162, 2012.

Digital Library

[17]

R. Jain. The art of computer systems performance analysis: techniques for experimental design, measurement, simulation, and modeling. Wiley, 1990.

[18]

P. Langley. Elements of machine learning. Morgan Kaufmann, 1996.

Digital Library

[19]

P. Leitner, W. Hummer, B. Satzger, C. Inzinger, and S. Dustdar. Cost-Efficient and Application SLA-Aware Client Side Request Scheduling in an Infrastructure-as-a-Service Cloud. In Int. Conf. on Cloud Comp. (CLOUD), pages 213--220. IEEE, 2012.

Digital Library

[20]

D. A. Menascé. TPC-W: A benchmark for e-commerce. Internet Comp., 6(3):83--87, 2002.

Digital Library

[21]

C. Napoli, G. Pappalardo, and E. Tramontana. A hybrid neuro-wavelet predictor for qos control and stability. In Cong. of the Italian Association for Artificial Intelligence, pages 527--538. Springer, 2013.

Digital Library

[22]

Y. Nesterov. A method of solving a convex programming problem with convergence rate o (1/k2). In Soviet Mathematics Doklady, volume 27, pages 372--376, 1983.

[23]

Y. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course, volume 87. Springer Science & Business Media, 2003.

Digital Library

[24]

R. Rojas. Neural networks: a systematic introduction. Springer Science & Business Media, 2013.

[25]

M. Sarkar, T. Mondal, S. Roy, and N. Mukherjee. Resource requirement prediction using clone detection technique. Future Gen. Comp. Sys., 29(4):936--952, 2013.

Digital Library

[26]

S. Singh and I. Chana. QoS-Aware Autonomic Resource Management in Cloud Computing: A Systematic Review. Comp. Surveys (CSUR), 48(3), 2016.

Digital Library

[27]

J. Sola and J. Sevilla. Importance of input data normalization for the application of neural networks to complex industrial problems. Trans. on Nuclear Science, 44(3/3):1464--1468, 1997.

[28]

I. Sutskever, J. Martens, G. Dahl, and G. Hinton. On the importance of initialization and momentum in deep learning. In Int. Conf. on Machine Learning (ICML), pages 1139--1147, 2013.

[29]

C. Szabo and T. Kroeger. Evolving Multi-objective Strategies for Task Allocation of Scientific Workflows on Public Clouds. In Cong. on Evolutionary Comp. (CEC), pages 1--8. IEEE, 2012.

[30]

C. Viswanath and C. Valliyammai. Cpu load prediction using anfis for grid computing. In Int. Conf. on Advances in Engineering, Science and Management (ICAESM), pages 343--348. IEEE, 2012.

[31]

L. Wu, S. K. Garg, and R. Buyya. SLA-Based Resource Allocation for Software as a Service Provider (SaaS) in Cloud Computing Environments. In Int. Symp. on Cluster, Cloud and Grid Comp. (CCGrid), pages 195--204. IEEE, 2011.

Digital Library

[32]

Z.-H. Zhan, X.-F. Liu, Y.-J. Gong, J. Zhang, H. S.-H. Chung, and Y. Li. Cloud Computing Resource Scheduling and a Survey of Its Evolutionary Approaches. Comp. Surveys (CSUR), 47(4), 2015.

Digital Library

Cited By

Hanczewski SStasiak MWeissenberg M(2024)An Analytical Model of IaaS Architecture for Determining Resource UtilizationSensors10.3390/s2409275824:9(2758)Online publication date: 26-Apr-2024
https://doi.org/10.3390/s24092758
Hanczewski SStasiak MWeissenberg M(2024)High-Accuracy Analytical Model for Heterogeneous Cloud Systems with Limited Availability of Physical Machine Resources Based on Markov ChainElectronics10.3390/electronics1311216113:11(2161)Online publication date: 1-Jun-2024
https://doi.org/10.3390/electronics13112161
Khan HShahzad M(2024)Characterizing Power Usage in Zero Reserved Power Data Centers to Enable Planned Maintenance2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619758(59-67)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619758
Show More Cited By

Index Terms

Predicting cloud resource utilization
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information systems applications
    1. Computing platforms
    2. Enterprise information systems

Recommendations

Resource usage optimization in Mobile Cloud Computing

We proposed and tested Mobile Cloud Computing architectures designed in order to lower resource usage on the cloud side. Proposed architectures focus on methods of task distribution and auto scaling in the remote cloud rather than on code offloading ...
Cloud resource provisioning: survey, status and future research directions

Cloud resource provisioning is a challenging job that may be compromised due to unavailability of the expected resources. Quality of Service (QoS) requirements of workloads derives the provisioning of appropriate resources to cloud workloads. Discovery ...
Toward Using Representation Learning for Cloud Resource Usage Forecasting
AI4Sys '24: Proceedings of the 2024 Workshop on AI For Systems

Cloud providers face issues with cost efficiency due to the inefficient utilization of resources within their clusters. To improve efficiency, cloud resource management systems use techniques such as autoscaling and overcommiting resources across users, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

UCC '16: Proceedings of the 9th International Conference on Utility and Cloud Computing

December 2016

549 pages

ISBN:9781450346160

DOI:10.1145/2996890

General Chairs:
Changjun Jiang
Tongji University, China
,
Omer Rana
Cardiff University, UK
,
Nick Antonopoulos
University of Derby, UK

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Commission of the European Union

Conference

UCC '16

UCC '16: 9th International Conference on Utility and Cloud Computing

December 6 - 9, 2016

Shanghai, China

Acceptance Rates

Overall Acceptance Rate 38 of 125 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
435
Total Downloads

Downloads (Last 12 months)57
Downloads (Last 6 weeks)1

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hanczewski SStasiak MWeissenberg M(2024)An Analytical Model of IaaS Architecture for Determining Resource UtilizationSensors10.3390/s2409275824:9(2758)Online publication date: 26-Apr-2024
https://doi.org/10.3390/s24092758
Hanczewski SStasiak MWeissenberg M(2024)High-Accuracy Analytical Model for Heterogeneous Cloud Systems with Limited Availability of Physical Machine Resources Based on Markov ChainElectronics10.3390/electronics1311216113:11(2161)Online publication date: 1-Jun-2024
https://doi.org/10.3390/electronics13112161
Khan HShahzad M(2024)Characterizing Power Usage in Zero Reserved Power Data Centers to Enable Planned Maintenance2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619758(59-67)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619758
Yang BJin KZou L(2024)The Parallel DCNN-LSTM Model for Cloud Computing Prediction Based on an Improved SG Filter2024 IEEE 2nd International Conference on Image Processing and Computer Applications (ICIPCA)10.1109/ICIPCA61593.2024.10709288(987-992)Online publication date: 28-Jun-2024
https://doi.org/10.1109/ICIPCA61593.2024.10709288
Tang XLiu FWang BXu DJiang JWu QChen C(2024)Workflow scheduling based on asynchronous advantage actor-critic algorithm in multi-cloud environmentExpert Systems with Applications10.1016/j.eswa.2024.125245(125245)Online publication date: Aug-2024
https://doi.org/10.1016/j.eswa.2024.125245
Ullah IAdhikari DSu XPalmieri FWu CChoi C(2024)Integration of data science with the intelligent IoT (IIoT): current challenges and future perspectivesDigital Communications and Networks10.1016/j.dcan.2024.02.007Online publication date: Mar-2024
https://doi.org/10.1016/j.dcan.2024.02.007
Yahya MShukla PDwivedi ARaza Khan AKumar NKhan RPamucar D(2024)Optimizing cloud resource utilization in the digital economy: An integrated Pythagorean fuzzy-based decision-making approachAdvanced Engineering Informatics10.1016/j.aei.2024.10265762(102657)Online publication date: Oct-2024
https://doi.org/10.1016/j.aei.2024.102657
Park SBahn H(2024)Combining genetic algorithms and bayesian neural networks for resource usage prediction in multi-tenant container environmentsCluster Computing10.1007/s10586-024-04832-628:2Online publication date: 26-Nov-2024
https://doi.org/10.1007/s10586-024-04832-6
Garg SAhuja RSingh RPerl I(2024)An effective deep learning architecture leveraging BIRCH clustering for resource usage prediction of heterogeneous machines in cloud data centerCluster Computing10.1007/s10586-023-04258-627:5(5699-5719)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10586-023-04258-6
Stupar IHuljenic D(2023)Model-based cloud service deployment optimisation method for minimisation of application service operational costJournal of Cloud Computing10.1186/s13677-023-00389-812:1Online publication date: 18-Feb-2023
https://doi.org/10.1186/s13677-023-00389-8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten