research-article

Random Slicing: Efficient and Scalable Data Placement for Large-Scale Storage Systems

Authors:

Alberto Miranda,

Ethan L. Miller,

Andre Brinkmann,

Tom Friedetzky,

Toni CortesAuthors Info & Claims

ACM Transactions on Storage (TOS), Volume 10, Issue 3

Article No.: 9, Pages 1 - 35

https://doi.org/10.1145/2632230

Published: 07 August 2014 Publication History

Abstract

The ever-growing amount of data requires highly scalable storage solutions. The most flexible approach is to use storage pools that can be expanded and scaled down by adding or removing storage devices. To make this approach usable, it is necessary to provide a solution to locate data items in such a dynamic environment. This article presents and evaluates the Random Slicing strategy, which incorporates lessons learned from table-based, rule-based, and pseudo-randomized hashing strategies and is able to provide a simple and efficient strategy that scales up to handle exascale data. Random Slicing keeps a small table with information about previous storage system insert and remove operations, drastically reducing the required amount of randomness while delivering a perfect load distribution.

References

[1]

S. Amarasinghe, D. Campbell, W. Carlson, A. Chien, W. Dally, E. Elnohazy, M. Hall, et al. 2010. ExaScale software study: Software challenges in extreme scale systems. Tech. Rep., sponsored by DARPA IPTO in the context of the ExaScale Computing Study.

[2]

A. Azagury, V. Dreizin, M. Factor, E. Henis, D. Naor, Y. Rinetzky, O. Rodeh, J. Satran, A. Tavory, and L. Yerushalmi. 2003. Towards an object store. In Proceedings of the 20th IEEE Conference on Mass Storage Systems and Technologies (MSST). 165--176.

Digital Library

[3]

Y. Azar, A. Broder, A. Karlin, and E. Upfal. 1999. Balanced allocations. SIAM J. Comput. 29, 1, 180--200.

Digital Library

[4]

J. L. Bentley. 1977. Solutions to Klees rectangle problems. Tech. Rep. Carnegie-Mellon University, Pittsburgh, PA.

[5]

M. Blaum, J. Brady, J. Bruck, and J. Menon. 1994. EVENODD: An optimal scheme for tolerating double disk failures in RAID architectures. In Proceedings of the 21st International Symposium on Computer Architecture (ISCA). 245--254.

Digital Library

[6]

R. P. Brent. 1992. Uniform random number generators for supercomputers. In Proceedings of the 5th Australian Supercomputer Conference. 95--104.

[7]

A. Brinkmann and S. Effert. 2008. Redundant data placement strategies for cluster storage environments. In Proceedings of the 12th International Conference on Principles of DIstributed Systems (OPODIS).

Digital Library

[8]

A. Brinkmann, S. Effert, F. Meyer Auf Der Heide, and C. Scheideler. 2007. Dynamic and redundant data placement. In Proceedings of the 27th IEEE International Conference on Distributed Computing Systems (ICDCS).

Digital Library

[9]

A. Brinkmann, M. Heidebuer, F. Meyer Auf Der Heide, U. Rückert, K. Salzwedel, and M. Vodisek. 2004. V: Drive - Costs and Benefits of an Out-of-Band Storage Virtualization System. In Proceedings of the 21st IEEE Conference on Mass Storage Systems and Technologies (MSST). 153--157.

[10]

A. Brinkmann, K. Salzwedel, and C. Scheideler. 2000. Efficient, distributed data placement strategies for storage area networks. In Proceedings of the 12th ACM Symposium on Parallel Algorithms and Architectures (SPAA). 119--128.

Digital Library

[11]

A. Brinkmann, Kay Salzwedel, and C. Scheideler. 2002. Compact, adaptive placement schemes for non-uniform distribution requirements. In Proceedings of the 14th ACM Symposium on Parallel Algorithms and Architectures (SPAA). 53--62.

Digital Library

[12]

W. S. Chou. 1995. On inversive maximal period polynomials over finite fields. Appl. Algeb. Eng. Commun. Comput. 6, 4--5, 245--250.

[13]

P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong, and S. Sankar. 2004. Row-diagonal parity for double disk failure correction. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies (FAST). 1--14.

Digital Library

[14]

T. Cortes and J. Labarta. 2001. Extending heterogeneity to RAID level 5. In Proceedings of the USENIX Annual Technical Conference. 119--132.

Digital Library

[15]

M. De Berg, O. Cheong, and M. Van Kreveld. 2008. Computational Geometry: Algorithms and Applications. Springer.

Digital Library

[16]

G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. 2007. Dynamo: Amazon's highly available key-value store. ACM SIGOPS Oper. Syst. Rev. 41, 6, 205--220.

Digital Library

[17]

A. Devulapalli, D. Dalessandro, and P. Wyckoff. 2008. Data structure consistency using atomic operations in storage devices. In Proceedings of the 5th International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI). 65--73.

Digital Library

[18]

D. Eastlake and P. Jones. 2001. US secure hash algorithm 1 (SHA1).

[19]

J. Gonzalez and T. Cortes. 2008. Distributing orthogonal redundancy on adaptive disk arrays. In Proceedings of the International Conference on Grid Computing, High-Performance and Distributed Applications (GADA).

Digital Library

[20]

R. J. Honicky and E. L. Miller. 2003. A fast algorithm for online placement and reorganization of replicated data. In Proceedings of the 17th IEEE International Parallel and Distributed Processing Symposium (IPDPS).

Digital Library

[21]

R. J. Honicky and E. L. Miller. 2004. Replication under scalable hashing: A family of algorithms for scalable decentralized data distribution. In Proceedings of the 18th IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[22]

N. L. Johnson and S. Kotz. 1977. Urn Models and Their Applications. Wiley, New York.

[23]

D. Karger, E. Lehman, T. Leighton, M. Levine, D. Lewin, and R. Panigrahy. 1997. Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the world wide web. In Proceedings of the 29th ACM Symposium on Theory of Computing (STOC). 654--663.

Digital Library

[24]

D. E. Knuth. 1997. Volume 2: Seminumerical Algorithms. In the Art of Computer Programming, 192.

[25]

D. H. Lehmer. 1951. Mathematical methods in large-scale computing units. Ann. Comput. Lab. Harvard Univ. 26, 141--146.

[26]

M. Luby. 1996. Pseudorandomness and Cryptographic Applications. Princeton University Press.

Digital Library

[27]

G. Marsaglia and A. Zaman. 1991. A new class of random number generators. Ann. Appl. Probab. 462--480.

[28]

M. Mense and C. Scheideler. 2008. SPREAD: An adaptive scheme for redundant and fair storage in dynamic heterogeneous storage systems. In Proceedings of the 19th ACM-SIAM Symposium on Discrete Algorithms (SODA).

Digital Library

[29]

M. Mitzenmacher. 1996. The power of two choices in randomized load balancing. Ph.D. thesis. Computer Science Department, University of California, Berkeley.

Digital Library

[30]

D. A. Patterson, G. Gibson, and R. H. Katz. 1988. A case for redundant arrays of inexpensive disks (RAID). In Proceedings of the ACM Conference on Management of Data (SIGMOD). 109--116.

Digital Library

[31]

I. Popov, A. Brinkmann, and T. Friedetzky. 2012. On the influence of PRNGs on data distribution. In Proceedings of the 20th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP). IEEE, 536--543.

Digital Library

[32]

M. Raab and A. Steger. 1998. Balls into BINSA simple and tight analysis. Random. Approx. Tech. Comput. Sci., 159--170.

Digital Library

[33]

P. Sanders. 2001. Reconciling simplicity and realism in parallel disk models. In Proceedings of the 12th ACM-SIAM Symposium on Discrete Algorithms (SODA). SIAM, 67--76.

Digital Library

[34]

C. Schindelhauer and G. Schomaker. 2005. Weighted distributed hash tables. In Proceedings of the 17th ACM Symposium on Parallel Algorithms and Architectures (SPAA). 218--227.

Digital Library

[35]

F. L. Severence. 2009. System Modeling and Simulation: An Introduction. Wiley.

[36]

M. Stevens, A. Lenstra, and B. de Weger. 2007. Chosen-prefix collisions for md5 and colliding x.509 certificates for different identities. In Proceedings of the 26th Annual International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT). 1--22.

Digital Library

[37]

M. Stevens, A. Sotirov, J. Appelbaum, A. Lenstra, D. Molnar, D. Osvik, and B. De Weger. 2009. Short Chosen-Prefix Collisions for MD5 and the Creation of a rogue CA certificate. In Proceedings of the 29th Annual International Cryptology Conference (CRYPTO). 55--69.

Digital Library

[38]

I. Stoica, R. Morris, D. Liben-Nowell, D. Karger, M. Kaashoek, F. Dabek, and H. Balakrishnan. 2003. Chord: A scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Trans. Netw. 11, 1, 17--32.

Digital Library

[39]

J. Viega. 2003. Practical random number generation in software. In Proceedings of the 19th Annual Computer Security Applications Conference (ACSAC). 129--141.

Digital Library

[40]

J. Walker. 1998. ENT Test suite. http://www.fourmilab.ch/random.

[41]

T. Wang. 2007. Integer hash function. http://www.concentric.net/ttwang/tech/inthash.htm.

[42]

X. Wang, Y. Yin, and H. Yu. 2005. Finding Collisions in the Full SHA-1. In Proceedings of the 25th Annual International Cryptology Conference (CRYPTO). 17--36.

Digital Library

[43]

S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn. 2006a. Ceph: A scalable, high-performance distributed file system. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI). 307--320.

Digital Library

[44]

S. A. Weil, S. A. Brandt, E. L. Miller, and C. Maltzahn. 2006b. CRUSH: Controlled, scalable and decentralized placement of replicated data. In Proceedings of the ACM/IEEE Conference on Supercomputing.

Digital Library

[45]

W. Zheng and G. Zhang. 2011. FastScale: Accelerate RAID Scaling by minimizing data migration. In Proceedings of the 9th USENIX Conference on File and Storage Technologies (FAST).

Digital Library

Cited By

Bangash YIqbal WMussiraliyeva SRubab SRauf B(2024)Reliability through an optimal SDS controller’s placement in a SDDC and smart cityCluster Computing10.1007/s10586-024-04325-627:6(7219-7240)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10586-024-04325-6
Barron ASanchez-Gallegos DCarrizales-Espinoza DGonzalez-Compean JMorales-Sandoval M(2022)On the Efficient Delivery and Storage of IoT Data in Edge–Fog–Cloud EnvironmentsSensors10.3390/s2218701622:18(7016)Online publication date: 16-Sep-2022
https://doi.org/10.3390/s22187016
Lu KZhao NWan JFei CZhao WDeng T(2022)RLRP: High-Efficient Data Placement with Reinforcement Learning for Modern Distributed Storage Systems2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS53621.2022.00064(595-605)Online publication date: May-2022
https://doi.org/10.1109/IPDPS53621.2022.00064
Show More Cited By

Index Terms

Random Slicing: Efficient and Scalable Data Placement for Large-Scale Storage Systems
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Memory management
        Allocation / deallocation strategies

Recommendations

Opportunities and challenges of storage grid enabled by grid service

The explosive growth of data generated worldwide by information digitization has been identified as the key driver of storage system evolution. In this paper, we discuss the existing storage system architectures and pose the challenges they are facing. ...
The Tiger Shark file system
COMPCON '96: Proceedings of the 41st IEEE International Computer Conference

Tiger Shark is a parallel file system for IBM's AIX operating system. It is designed to support interactive multimedia, particularly large-scale systems such as interactive television (ITV). Tiger Shark scales across the entire RS/6000 product line, ...
Efficient detection of large-scale redundancy in enterprise file systems

In order to catch and reduce waste in the exponentially increasing demand for disk storage, we have developed very efficient technology to detect approximate duplication of large directory hierarchies. Such duplication can be caused, for example, by ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage

ACM Transactions on Storage Volume 10, Issue 3

July 2014

113 pages

ISSN:1553-3077

EISSN:1553-3093

DOI:10.1145/2661087

Editor:
Darrell Long
University of California Santa Cruz, USA

Issue’s Table of Contents

Copyright © 2014 ACM.

© 2014 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2014

Accepted: 01 September 2013

Revised: 01 June 2013

Received: 01 November 2012

Published in TOS Volume 10, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
564
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bangash YIqbal WMussiraliyeva SRubab SRauf B(2024)Reliability through an optimal SDS controller’s placement in a SDDC and smart cityCluster Computing10.1007/s10586-024-04325-627:6(7219-7240)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10586-024-04325-6
Barron ASanchez-Gallegos DCarrizales-Espinoza DGonzalez-Compean JMorales-Sandoval M(2022)On the Efficient Delivery and Storage of IoT Data in Edge–Fog–Cloud EnvironmentsSensors10.3390/s2218701622:18(7016)Online publication date: 16-Sep-2022
https://doi.org/10.3390/s22187016
Lu KZhao NWan JFei CZhao WDeng T(2022)RLRP: High-Efficient Data Placement with Reinforcement Learning for Modern Distributed Storage Systems2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS53621.2022.00064(595-605)Online publication date: May-2022
https://doi.org/10.1109/IPDPS53621.2022.00064
Kim J(2021)Erasure-Coding-Based Storage and Recovery for Distributed Exascale Storage SystemsApplied Sciences10.3390/app1108329811:8(3298)Online publication date: 7-Apr-2021
https://doi.org/10.3390/app11083298
Zhou JSu LWang WChen Y(2021)Hashing Based Data Distribution in Heterogeneous Storage2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00095(652-659)Online publication date: Sep-2021
https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00095
Morales-Ferreira PSantiago-Duran MGaytan-Diaz CGonzalez-Compean JSosa-Sosa VLopez-Arevalo I(2018)A Data Distribution Service for Cloud and Containerized Storage Based on Information Dispersal2018 IEEE Symposium on Service-Oriented System Engineering (SOSE)10.1109/SOSE.2018.00020(86-95)Online publication date: Mar-2018
https://doi.org/10.1109/SOSE.2018.00020
Xu QXi WYang KJin C(2018)An Efficient Segment Grouping Approach for Active Disk-Based Storage Systems2018 IEEE 24th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/PADSW.2018.8644992(178-185)Online publication date: Dec-2018
https://doi.org/10.1109/PADSW.2018.8644992
Liu CChu XLiu HLeung Y(2016)ESet: Placing Data Towards Efficient Recovery for Large-Scale Erasure-Coded Storage Systems2016 25th International Conference on Computer Communication and Networks (ICCCN)10.1109/ICCCN.2016.7568521(1-9)Online publication date: Aug-2016
https://doi.org/10.1109/ICCCN.2016.7568521
Liu JShen HZhang X(2016)A Survey of Mobile Crowdsensing Techniques: A Critical Component for the Internet of Things2016 25th International Conference on Computer Communication and Networks (ICCCN)10.1109/ICCCN.2016.7568484(1-6)Online publication date: Aug-2016
https://doi.org/10.1109/ICCCN.2016.7568484

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents