research-article

Hierarchical scheduling for diverse datacenter workloads

Authors:

Arka A. Bhattacharya,

Ion StoicaAuthors Info & Claims

SOCC '13: Proceedings of the 4th annual Symposium on Cloud Computing

Article No.: 4, Pages 1 - 15

https://doi.org/10.1145/2523616.2523637

Published: 01 October 2013 Publication History

Abstract

There has been a recent industrial effort to develop multi-resource hierarchical schedulers. However, the existing implementations have some shortcomings in that they might leave resources unallocated or starve certain jobs. This is because the multi-resource setting introduces new challenges for hierarchical scheduling policies. We provide an algorithm, which we implement in Hadoop, that generalizes the most commonly used multi-resource scheduler, DRF [1], to support hierarchies. Our evaluation shows that our proposed algorithm, H-DRF, avoids the starvation and resource inefficiencies of the existing open-source schedulers and outperforms slot scheduling.

References

[1]

A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, I. Stoica, and S. Shenker. Dominant resource fairness: Fair allocation of multiple resource types. In NSDI, 2011.

Digital Library

[2]

Michael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar, and Andrew Goldberg. Quincy: Fair scheduling for distributed computing clusters. In SOSP, 2009.

Digital Library

[3]

Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling. In EuroSys, 2010.

Digital Library

[4]

T. A. Henzinger, A. V. Singh, V. Singh, T. Wies, and D. Zufferey. Static scheduling in clouds. In HotCloud, June 2011.

Digital Library

[5]

Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, and Ion Stoica. Improving MapReduce Performance in Heterogeneous Environments. In Proc. OSDI, December 2008.

Digital Library

[6]

Hadoop Capacity Scheduler. http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html.

[7]

Malte Schwarzkopf, Andy Konwinski, Michael Abd-El-Malek, and John Wilkes. Omega: flexible, scalable schedulers for large compute clusters. In Proceedings of the 8th ACM European Conference on Computer Systems, pages 351--364. ACM, 2013.

Digital Library

[8]

Alexey Tumanov, James Cipar, Gregory R Ganger, and Michael A Kozuch. alsched: Algebraic scheduling of mixed workloads in heterogeneous clouds. In Proceedings of the Third ACM Symposium on Cloud Computing. ACM, 2012.

Digital Library

[9]

Hadoop Fair Scheduler. http://hadoop.apache.org/common/docs/r0.20.2/fair_scheduler.html.

[10]

Carlee Joe-Wong, Soumya Sen, Tian Lan, and Mung Chiang. Multi-resource allocation: Fairness-efficiency tradeoffs in a unifying framework. In INFOCOM, pages 1206--1214, 2012.

[11]

Avital Gutman and Noam Nisan. Fair Allocation Without Trade. In AAMAS, June 2012.

Digital Library

[12]

David C. Parkes, Ariel D. Procaccia, and Nisarg Shah. Beyond Dominant Resource Fairness: Extensions, Limitations, and Indivisibilities. In ACM EC, 2012.

Digital Library

[13]

Ali Ghodsi, Vyas Sekar, Matei Zaharia, and Ion Stoica. Multi-resource fair queueing for packet processing. In SIGCOMM, 2012.

[14]

Charles Reiss, Alexey Tumanov, Gregory R. Ganger, Randy H. Katz, and Michael A. Kozuch. Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In ACM Symposium on Cloud Computing (SoCC), San Jose, CA, USA, October 2012.

Digital Library

[15]

Bikash Sharma, Ramya Prabhakar, Seung-Hwan Lim, Mahmut T. Kandemir, and Chita R. Das. Mrorchestrator: A fine-grained resource orchestration framework for mapreduce clusters. In IEEE CLOUD, pages 1--8, 2012.

Digital Library

[16]

R. Boutaba, L. Cheng, and Q. Zhang. On cloud computational models and the heterogeneity challenge. J. Internet Services and Applications, 3(1): 77--86, 2012.

[17]

B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. H. Katz, S. Shenker, and I. Stoica. Mesos: A platform for fine-grained resource sharing in the data center. In NSDI, 2011.

Digital Library

[18]

YARN DRF extension to the Capacity Scheduler. https://issues.apache.org/jira/browse/YARN-2.

[19]

The Next Generation of Apache Hadoop MapReduce. http://developer.yahoo.com/blogs/hadoop/posts/2011/02/mapreduce-nextgen.

[20]

YARN DRF extension to the Fair Scheduler. https://issues.apache.org/jira/browse/YARN-326.

[21]

Hadoop Yarn 2.0.2--alpha. http://hadoop.apache.org/docs/current/.

[22]

Abhishek Chandra and Prashant Shenoy. Hierarchical scheduling for symmetric multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 19: 418--431, 2008.

Digital Library

[23]

Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andrew Konwinski, Scott Shenker, and Ion Stoica. Dominant resource fairness: Fair allocation of multiple resource types. Technical Report UCB/EECS-2011-18, EECS Department, University of California, Berkeley, Mar 2011.

[24]

Jon C. R. Bennett and Hui Zhang. Hierarchical packet fair queueing algorithms. In IEEE/ACM Transactions on Networking, pages 143--156, 1997.

Digital Library

[25]

Pawan Goyal, Xingang Guo, and Harrick M. Vin. A Hierarchical CPU Scheduler for Multimedia Operating Systems. In OSDI, pages 107--121, 1996.

Digital Library

[26]

C. A. Waldspurger. Lottery and Stride Scheduling: Flexible Proportional Share Resource Management. PhD thesis, MIT, Laboratory of Computer Science, September 1995. MIT/LCS/TR-667.

Digital Library

[27]

Ajay Gulati, Ganesha Shanmuganathan, Xuechen Zhang, and Peter Varman. Demand based hierarchical QoS using storage resource pools. In Proceedings of the Annual USENIX Technical Conference, 2012.

Digital Library

[28]

Volker Hamscher, Uwe Schwiegelshohn, Achim Streit, and Ramin Yahyapour. Evaluation of job-scheduling strategies for grid computing. In Grid ComputingGRID 2000, pages 191--202. Springer, 2000.

Digital Library

[29]

V. Subramani, R. Kettimuthu, S. Srinivasan, and S. Sadayappan. Distributed job scheduling on computational grids using multiple simultaneous requests. In High Performance Distributed Computing, 2002. HPDC-11 2002. Proceedings. 11th IEEE International Symposium on, pages 359--366, 2002.

Digital Library

Cited By

Chavan PChavan P(2024)Automation of AD-OHC Dashbord and Monitoring of Cloud Resources using Genrative AI to Reduce Costing and Enhance Performance2024 International Conference on Innovations and Challenges in Emerging Technologies (ICICET)10.1109/ICICET59348.2024.10616299(1-9)Online publication date: 7-Jun-2024
https://doi.org/10.1109/ICICET59348.2024.10616299
Tong YLiu JWang HHe MZhou KHe RZhang QWang C(2024)DAG-aware harmonizing job scheduling and data caching for disaggregated analytics frameworksFuture Generation Computer Systems10.1016/j.future.2024.03.005156(116-129)Online publication date: Jul-2024
https://doi.org/10.1016/j.future.2024.03.005
You CZhao YFeng GQuek TLi L(2023)Hierarchical Multiresource Fair Queueing for Packet ProcessingIEEE Transactions on Network and Service Management10.1109/TNSM.2022.319774720:1(726-740)Online publication date: Mar-2023
https://doi.org/10.1109/TNSM.2022.3197747
Show More Cited By

Index Terms

Hierarchical scheduling for diverse datacenter workloads
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Process management
        Scheduling

Recommendations

A Fair Scheduling Algorithm for Adaptive Heterogeneous Resources in Data Centers
Internetware '18: Proceedings of the 10th Asia-Pacific Symposium on Internetware

The resource scheduling problem of data center clusters has always been a hot topic in the field of cloud computing. Existing research efforts focus on fairness, resource utilization and energy efficiency, and lack of research on heterogeneous ...
Mixed Pfair/ERfair scheduling of asynchronous periodic tasks
Abstract
Pfair scheduling was proposed by Baruah, Cohen, Plaxton, and Varvel as a non-work-conserving way of optimally and efficiently scheduling periodic tasks on a multiprocessor. In this paper, we introduce a work-conserving variant of Pfair scheduling ...
Non-clairvoyant scheduling with precedence constraints
SODA '08: Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms

We consider Edmonds's model (1999) extended by precedence constraints. In our setting, a scheduler has to schedule non-clairvoyantly jobs consisting in DAGs of tasks arriving over time, each task going through phases of different degrees of parallelism, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOCC '13: Proceedings of the 4th annual Symposium on Cloud Computing

October 2013

427 pages

ISBN:9781450324281

DOI:10.1145/2523616

General Chair:
Guy Lohman

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

AmazonWeb Services
Cisco Systems
Samsung
Division of Computing and Communication Foundations
Defense Advanced Research Projects Agency
Intel Corporation
Yahoo!
National Science Foundation
Division of Computer and Network Systems
Google
FitWave
Hortonworks
Splunk
Clearstory Data
Ericsson
Facebook
VMware
SAP America
Cloudera
Oracle
NetApp
General Electric
Microsoft

Conference

SOCC '13

Sponsor:

SOCC '13: ACM Symposium on Cloud Computing

October 1 - 3, 2013

California, Santa Clara

Acceptance Rates

SOCC '13 Paper Acceptance Rate 23 of 114 submissions, 20%;

Overall Acceptance Rate 169 of 722 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

82
Total Citations
View Citations
947
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)5

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chavan PChavan P(2024)Automation of AD-OHC Dashbord and Monitoring of Cloud Resources using Genrative AI to Reduce Costing and Enhance Performance2024 International Conference on Innovations and Challenges in Emerging Technologies (ICICET)10.1109/ICICET59348.2024.10616299(1-9)Online publication date: 7-Jun-2024
https://doi.org/10.1109/ICICET59348.2024.10616299
Tong YLiu JWang HHe MZhou KHe RZhang QWang C(2024)DAG-aware harmonizing job scheduling and data caching for disaggregated analytics frameworksFuture Generation Computer Systems10.1016/j.future.2024.03.005156(116-129)Online publication date: Jul-2024
https://doi.org/10.1016/j.future.2024.03.005
You CZhao YFeng GQuek TLi L(2023)Hierarchical Multiresource Fair Queueing for Packet ProcessingIEEE Transactions on Network and Service Management10.1109/TNSM.2022.319774720:1(726-740)Online publication date: Mar-2023
https://doi.org/10.1109/TNSM.2022.3197747
Li HZhao HXu ZLi XXu H(2023)ExplSched: Maximizing Deep Learning Cluster Efficiency for Exploratory Jobs2023 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER52292.2023.00022(173-184)Online publication date: 31-Oct-2023
https://doi.org/10.1109/CLUSTER52292.2023.00022
Kim ELee KYoo C(2023)Network SLO-aware container scheduling in KubernetesThe Journal of Supercomputing10.1007/s11227-023-05122-579:10(11478-11494)Online publication date: 28-Feb-2023
https://doi.org/10.1007/s11227-023-05122-5
Tang SYu CLi Y(2022)Fairness-Efficiency Scheduling for Cloud Computing With Soft Fairness GuaranteesIEEE Transactions on Cloud Computing10.1109/TCC.2020.302108410:3(1806-1818)Online publication date: 1-Jul-2022
https://doi.org/10.1109/TCC.2020.3021084
Fakhartousi AMeacham SPhalp K(2022)Autonomic Dominant Resource Fairness (A-DRF) in Cloud Computing2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC54236.2022.00258(1626-1631)Online publication date: Jun-2022
https://doi.org/10.1109/COMPSAC54236.2022.00258
Lingam G(2022)Reinforcement learning based energy efficient resource allocation strategy of MapReduce jobs with deadline constraintCluster Computing10.1007/s10586-022-03761-626:5(2719-2735)Online publication date: 14-Oct-2022
https://doi.org/10.1007/s10586-022-03761-6
Kim ELee KYoo C(2022)DepCon: Achieving Network SLO for High Performance CloudsEuro-Par 2021: Parallel Processing Workshops10.1007/978-3-031-06156-1_27(339-351)Online publication date: 9-Jun-2022
https://doi.org/10.1007/978-3-031-06156-1_27
Sikdar SGuo XWang HXia LCao YDignum FLomuscio AEndriss UNowé A(2021)Sequential Mechanisms for Multi-type Resource AllocationProceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems10.5555/3463952.3464092(1209-1217)Online publication date: 3-May-2021
https://dl.acm.org/doi/10.5555/3463952.3464092
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents