research-article

Scarlett: coping with skewed content popularity in mapreduce clusters

Authors:

Ganesh Ananthanarayanan,

Sameer Agarwal,

Srikanth Kandula,

Albert Greenberg,

Ed HarrisAuthors Info & Claims

EuroSys '11: Proceedings of the sixth conference on Computer systems

Pages 287 - 300

https://doi.org/10.1145/1966445.1966472

Published: 10 April 2011 Publication History

Abstract

To improve data availability and resilience MapReduce frameworks use file systems that replicate data uniformly. However, analysis of job logs from a large production cluster shows wide disparity in data popularity. Machines and racks storing popular content become bottlenecks; thereby increasing the completion times of jobs accessing this data even when there are machines with spare cycles in the cluster. To address this problem, we present Scarlett, a system that replicates blocks based on their popularity. By accurately predicting file popularity and working within hard bounds on additional storage, Scarlett causes minimal interference to running jobs. Trace driven simulations and experiments in two popular MapReduce frameworks (Hadoop, Dryad) show that Scarlett effectively alleviates hotspots and can speed up jobs by 20.2%.

References

[1]

Akamai. Akamai content distribution network. http://www.akamai.com/.

[2]

M. Al-Fares, A. Loukissas, and A. Vahdat. A Scalable, Commodity Data Center Network Architecture. In SIGCOMM'08: Proceedings of the ACM SIGCOMM 2008 conference on Data communication, 2008.

Digital Library

[3]

G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, E. Harris, and B. Saha. Reining in the Outliers in Map-Reduce Clusters using Mantri. In OSDI'10: Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, 2010.

Digital Library

[4]

J. Appavoo, A. Waterland, D. Da Silva, V. Uhlig, B. Rosenburg, E. Van Hensbergen, J. Stoess, R. Wisniewski, and U. Steinberg. Providing a Cloud Network Infrastructure on a Supercomputer. In HPDC'10: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, 2010.

Digital Library

[5]

N. M. Belaramani, J. Zheng, A. Nayate, R. Soul&#233;, M. Dahlin, and R. Grimm. PADS: A Policy Architecture for Distributed Storage Systems. In NSDI'09: Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation, 2009.

Digital Library

[6]

T. Benzel, R. Braden, D. Kim, C. Neuman, A. Joseph, K. Sklower, R. Ostrenga, and S. Schwab. Experience with DETER: A Testbed for Security Research. In International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities, 2006.

[7]

Ronnie Chaiken, Bob Jenkins, Perke Larson, Bill Ramsey, Darren Shakib, Simon Weaver, and Jingren Zhou. SCOPE: Easy and Efficient Parallel Processing of Massive Datasets. In VLDB'08: Proceedings of the 34th Conference on Very Large Data Bases, 2008.

[8]

Y. Chen, A. Ganapathi, and R. Katz. To Compress or Not To Compress - Compute vs. IO tradeoffs for MapReduce Energy Efficiency. In Proceedings of the First ACM SIGCOMM Workshop on Green Networking, 2010.

Digital Library

[9]

Cloud Benefits. Cloud compute can save govt agencies 25--50% in costs, 2010. http://googlepublicpolicy.blogspot.com/2010/04/brookings-cloud-computing-can-save-govt.html.

[10]

Coral CDN. The coral content distribution network. http://www.coralcdn.org/.

[11]

K. Curewitz, P. Krishnan, and J. S. Vitter. Practical Prefetching via Data Compression. In SIGMOD'93: Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, 1993.

Digital Library

[12]

J. Dean. Designs, lessons and advice from building large distributed systems, 2009. http://www.cs.cornell.edu/projects/ladis2%09/talks/dean-keynote-ladis2009.pdf.

[13]

J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI'04: Proceedings of the 6th Symposium on Operating Systems Design and Implementation, 2004.

Digital Library

[14]

S. Ghemawat, H. Gobioff, and S. Leung. The Google File System. In SOSP'09: Proceedings of the 19th ACM Symposium on Operating Systems Principles, 2003.

Digital Library

[15]

A. Greenberg, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: A Scalable and Flexible Data Center Network. In SIGCOMM'09: Proceedings of the ACM SIGCOMM 2009 conference on Data communication, 2009.

Digital Library

[16]

P. K. Gunda, L. Ravindranath, C. A. Thekkath, Y. Yu, and L. Zhuang. Nectar: Automatic Management of Data and Computation in Data Centers. In OSDI'10: Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, 2010.

Digital Library

[17]

Hadoop. http://hadoop.apache.org.

[18]

Hadoop Apps. Applications and organizations using hadoop, 2010. http://wiki.apache.org/hadoop/PoweredBy.

[19]

HDFS. Hadoop distributed file system. http://hadoop.apache.org/hdfs.

[20]

M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed Data-parallel Programs from Sequential Building Blocks. In EuroSys'07: Proceedings of the European Conference on Computer Systems, 2007.

Digital Library

[21]

M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: Fair Scheduling for Distributed Computing Clusters. In SOSP'09: Proceedings of the 22nd ACM Symposium on Operating Systems Principles, 2009.

Digital Library

[22]

S. Kandula, S. Sengupta, A. Greenberg, P. Patel, and R. Chaiken. Nature of Datacenter Traffic: Measurements and Analysis. In IMC'09: Proceedings of the Ninth Internet Measurement Conference, 2009.

Digital Library

[23]

D. Narayanan, A. Donnelly, E. Thereska, S. Elnikety, and A. Rowstron. Everest: Scaling Down Peak Loads Through I/O Off-Loading. In OSDI'08: Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation, 2008.

Digital Library

[24]

V. Ramasubramanian and E. G. Sirer. Beehive: O(1) Lookup Performance for Power-law Query Distributions in peer-to-peer Overlays. In NSDI'04: Proceedings of the First Symposium on Networked Systems Design and Implementation, 2004.

Digital Library

[25]

M. Shreedhar and G. Varghese. Efficient Fair Queuing Using Deficit Round-Robin. In IEEE/ACM Transactions on Networking, 1996.

Digital Library

[26]

P. Skibinski and S. Grabowski. Variable-length contexts for PPM. In DCC'04: Proceedings of IEEE Data Compression Conference, 2004.

Digital Library

[27]

P. Skibinski and J. Swacha. Fast and Efficient Log File Compression. In CEUR ADBIS'07: Advances in Databases and Information Systems, 2007.

[28]

G. Soundararajan, C. Amza, and A. Goel. Database Replication Policies for Dynamic Content Applications. In EuroSys'06: Proceedings of the European Conference on Computer Systems, 2006.

Digital Library

[29]

J. Stribling, Y. Sovran, I. Zhang, X. Pretzer, J. Li, F. Kaashoek, and R. Morris. Flexible, Wide-Area Storage for Distributed Systems with WheelFS. In NSDI'09: 6th USENIX Symposium on Networked Systems Design and Implementation, 2009.

Digital Library

[30]

A. Verma, R. Koller, L. Useche, and R. Rangaswami. SRCMap: Energy Proportional Storage Using Dynamic Consolidation. In FAST'10: Proceedings of the 8th USENIX Conference on File and Storage Technologies, 2010.

Digital Library

[31]

T. Yeh, D. E. Long, and S. A. Brandt. Increasing Predictive Accuracy by Prefetching Multiple Program and User Specific Files. HPCS'02: Proceedings of the 16th Annual International Symposium on High Performance Computing Systems and Applications, 2002.

Digital Library

[32]

M. Zaharia, D. Borthakur, J. Sen Sharma, K. Elmeleegy, S. Shenker, and I. Stoica. Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling. In EuroSys'10: Proceedings of the European Conference on Computer Systems, 2010.

Digital Library

Cited By

Nan FWu RShen ZYang JCheng LChen ZZhang YShu J(2025)AC-Cache: A Memory-Efficient Caching System for Small Objects via Exploiting Access CorrelationsProceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming10.1145/3710848.3710856(142-155)Online publication date: 28-Feb-2025
https://dl.acm.org/doi/10.1145/3710848.3710856
Wu BZhu RZhang ZSun PLiu XJin XGavrilovska ATerry D(2024)dLoRAProceedings of the 18th USENIX Conference on Operating Systems Design and Implementation10.5555/3691938.3691987(911-927)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691938.3691987
Wang SLuo JLi Y(2024)An Access-Oriented Placement Strategy with Online Erasure Coding in Memory StoresProceedings of the 2024 9th International Conference on Intelligent Information Processing10.1145/3696952.3696974(154-161)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3696952.3696974
Show More Cited By

Index Terms

Scarlett: coping with skewed content popularity in mapreduce clusters
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Memory management
        Distributed memory
    2. Software system structures
      1. Distributed systems organizing principles

Recommendations

An efficient Hadoop data replication method design for heterogeneous clusters
SAC '16: Proceedings of the 31st Annual ACM Symposium on Applied Computing

The performance of each datanode in a heterogeneous Hadoop cluster differs, and the number of slots that can be numbered to simultaneously execute tasks differs. For this reason, Hadoop is susceptible to replica placement problems and data replication ...
PARS: A Page-Aware Replication System for Efficiently Storing Virtual Machine Snapshots
VEE '15

Virtual machine (VM) snapshot enhances the system availability by saving the running state into stable storage during failure-free execution and rolling back to the snapshot point upon failures. Unfortunately, the snapshot state may be lost due to disk ...
Consistent and automatic replica regeneration

Reducing management costs and improving the availability of large-scale distributed systems require automatic replica regeneration, that is, creating new replicas in response to replica failures. A major challenge to regeneration is maintaining ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EuroSys '11: Proceedings of the sixth conference on Computer systems

April 2011

370 pages

ISBN:9781450306348

DOI:10.1145/1966445

General Chair:
Christoph Kirsch
University of Salzburg, Austria
,
Program Chair:
Gernot Heiser
University of New South Wales, Australia

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 April 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

EuroSys '11

Sponsor:

SIGOPS

EuroSys '11: Sixth EuroSys Conference 2011

April 10 - 13, 2011

Salzburg, Austria

Acceptance Rates

EuroSys '11 Paper Acceptance Rate 24 of 161 submissions, 15%;

Overall Acceptance Rate 241 of 1,308 submissions, 18%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

214
Total Citations
View Citations
992
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nan FWu RShen ZYang JCheng LChen ZZhang YShu J(2025)AC-Cache: A Memory-Efficient Caching System for Small Objects via Exploiting Access CorrelationsProceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming10.1145/3710848.3710856(142-155)Online publication date: 28-Feb-2025
https://dl.acm.org/doi/10.1145/3710848.3710856
Wu BZhu RZhang ZSun PLiu XJin XGavrilovska ATerry D(2024)dLoRAProceedings of the 18th USENIX Conference on Operating Systems Design and Implementation10.5555/3691938.3691987(911-927)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.5555/3691938.3691987
Wang SLuo JLi Y(2024)An Access-Oriented Placement Strategy with Online Erasure Coding in Memory StoresProceedings of the 2024 9th International Conference on Intelligent Information Processing10.1145/3696952.3696974(154-161)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3696952.3696974
Wu SShen ZLee PBai ZXu Y(2024)Elastic Reed-Solomon Codes for Efficient Redundancy Transitioning in Distributed Key-Value StoresIEEE/ACM Transactions on Networking10.1109/TNET.2023.330386532:1(670-685)Online publication date: Feb-2024
https://doi.org/10.1109/TNET.2023.3303865
Demirbaga UAujla G(2024)RootPath: Root Cause and Critical Path Analysis to Ensure Sustainable and Resilient Consumer-Centric Big Data Processing Under Fault ScenariosIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332954570:1(1493-1500)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3329545
Chen JLi ZZhou RSu LWang N(2024)Advanced Elastic Reed–Solomon Codes for Erasure-Coded Key–Value StoresIEEE Internet of Things Journal10.1109/JIOT.2023.329957411:3(4747-4762)Online publication date: 1-Feb-2024
https://doi.org/10.1109/JIOT.2023.3299574
Li QLyu MXu LXu Y(2024)Fast recovery for large disk enclosures based on RAID2.0: Algorithms and evaluationJournal of Parallel and Distributed Computing10.1016/j.jpdc.2024.104854(104854)Online publication date: Feb-2024
https://doi.org/10.1016/j.jpdc.2024.104854
Herodotou HKakoulli E(2023)Cost-based Data Prefetching and Scheduling in Big Data Platforms over Tiered Storage SystemsACM Transactions on Database Systems10.1145/362538948:4(1-40)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3625389
Tang ZXiao ZYang LHe KLi K(2023)A Network Load Perception Based Task Scheduler for Parallel Distributed Data Processing SystemsIEEE Transactions on Cloud Computing10.1109/TCC.2021.313262711:2(1352-1364)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TCC.2021.3132627
Baganal-Krishna NMunstein DRizk A(2023)LETHE: Combined Time-to-Live Caching and Load Balancing on the Network Data Plane2023 IEEE 29th International Symposium on Local and Metropolitan Area Networks (LANMAN)10.1109/LANMAN58293.2023.10189809(1-6)Online publication date: 10-Jul-2023
https://doi.org/10.1109/LANMAN58293.2023.10189809
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten