Elastic Resource Provisioning to Expand the Capacity of Cluster in Hybrid Computing Infrastructure
Pages 780 - 785
Abstract
Several fields of science have traditionally demanded large-scale workflow support, which requires thousands of CPU cores or more. In this paper, we investigate ways to support these scientific workflows in a heterogeneous environment in which cluster computing resources are integrating with cloud computing resources. Specifically, we first propose an architecture that utilizes cloud resources to address load balancing issues. For that, the proposed architecture measures the status of job queue on the front-end node, and then dynamically creates virtual machines from cloud pools based on the measured results to expand computing resource of the cluster. Next, we present experiment results of computational performance in hybrid infrastructure where the virtual and physical nodes are mixed.
References
[1]
E. Deelman, D. Gannon, M. Shields, and I. Taylor, "Workflows and e-science: An overview of workflow system features and capabilities," Future Generation Computer Systems, vol. 25, no. 5, pp. 528-540, 2009.
[2]
Y. Gil, E. Deelman, M. Ellisman, T. Fahringer, G. Fox, D. Gannon, C. Goble, M. Livny, L. Moreau, and J. Myers, "Examining the challenges of scientific workflows," IEEE Computer, vol. 40, no. 12, pp. 24-32, Dec 2007.
[3]
J. Yu and R. Buyya, "A taxonomy of scientific workflow systems for grid computing," ACM Sigmod Record, vol. 34, no. 3, pp. 44-49, 2005.
[4]
G. V. Mc Evoy and B. Schulze, "Using clouds to address grid limitations," in Proc. of the 6th international workshop on Middleware for grid computing, 2008, p. 11.
[5]
"PLSI: Partnership & leadership for the nationwide supercomputing infrastructure." [Online]. Available: http://www.plsi.or.kr.
[6]
A. Gupta, L. V. Kale, F. Gioachin, V. March, C. H. Suen, B.-S. Lee, P. Faraboschi, R. Kaufmann, and D. Milojicic, "The who, what, why and how of high performance computing applications in the cloud," HP Labs, 2013., Tech. Rep., 2013.
[7]
A. Iosup, S. Ostermann, M. N. Yigitbasi, R. Prodan, T. Fahringer, and D. H. Epema, "Performance analysis of cloud computing services for many-tasks scientific computing," IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 6, pp. 931-945, 2011.
[8]
A. J. Younge, R. Henschel, J. T. Brown, G. von Laszewski, J. Qiu, and G. C. Fox, "Analysis of virtualization technologies for high performance computing environments," in Proc. of IEEE International Conference on Cloud Computing (CLOUD), 2011, pp. 9-16.
[9]
R. S. Montero, R. Moreno-Vozmediano, and I. M. Llorente, "An elasticity model for high throughput computing clusters," Journal of Parallel and Distributed Computing, vol. 71, no. 6, pp. 750-757, 2011.
[10]
E. Walker, "Benchmarking amazon ec2 for high-performance scientific computing," in LOGIN, vol. 33, 2008, pp. 18-23.
[11]
Q. He, S. Zhou, B. Kobler, D. Duffy, and T. McGlynn, "Case study for running hpc applications in public clouds," in Proc. of the 19th ACM International Symposium on High Performance Distributed Computing, ser. HPDC'10. New York, NY, USA: ACM, 2010, pp. 395-401.
[12]
K. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H. J. Wasserman, and N. Wright, "Performance analysis of high performance computing applications on the amazon web services cloud," in Proc. of IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), 2010, Nov 2010, pp. 159-168.
[13]
"NERSC: National energy research scientific computing center." [Online]. Available: https://www.nersc.gov.
[14]
P. Ruth, P. McGachey, and D. Xu, "Viocluster: Virtualization for dynamic computational domains," in Proc. of IEEE International Cluster Computing, 2005, Sept 2005, pp. 1-10.
[15]
M. A. Murphy, B. Kagey, M. Fenn, and S. Goasguen, "Dynamic provisioning of virtual organization clusters," in Proc. of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, ser. CCGRID'09, Washington, DC, USA, 2009, pp. 364-371.
[16]
P. Marshall, K. Keahey, and T. Freeman, "Elastic site: Using clouds to elastically extend site resources," in Proc. of the 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, ser. CCGRID'10, Washington, DC, USA, 2010, pp. 43-52.
Index Terms
- Elastic Resource Provisioning to Expand the Capacity of Cluster in Hybrid Computing Infrastructure
Recommendations
Optimal resource provisioning for cloud computing environment
The paper presents an efficient cloud resource provisioning approach. The Software as a Service (SaaS) provider leases resources from cloud providers and also leases software as services to SaaS users. The SaaS providers aim at minimizing the payment of ...
The Aneka platform and QoS-driven resource provisioning for elastic applications on hybrid Clouds
Cloud computing alters the way traditional software systems are built and run by introducing a utility-based model for delivering IT infrastructure, platforms, applications, and services. The consolidation of this new paradigm in both enterprises and ...
Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints
With the popularization and development of cloud computing, lots of scientific computing applications are conducted in cloud environments. However, current application scenario of scientific computing is also becoming increasingly dynamic and ...
Comments
Information & Contributors
Information
Published In
December 2014
1035 pages
ISBN:9781479978816
Sponsors
Publisher
IEEE Computer Society
United States
Publication History
Published: 08 December 2014
Check for updates
Author Tags
Qualifiers
- Article
Acceptance Rates
Overall Acceptance Rate 38 of 125 submissions, 30%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 24Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 16 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in