Article

Dynamic Metadata Management for Petabyte-Scale File Systems

Authors:

Kristal T. Pollack,

Scott A. Brandt,

Ethan L. MillerAuthors Info & Claims

SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing

Page 4

https://doi.org/10.1109/SC.2004.22

Published: 06 November 2004 Publication History

Abstract

In petabyte-scale distributed file systems that decouple read and write from metadata operations, behavior of the metadata server cluster will be critical to overall system performance and scalability. We present a dynamic subtree partitioning and adaptive metadata management system designed to efficiently manage hierarchical metadata workloads that evolve over time. We examine the relative merits of our approach in the context of traditional workload partitioning strategies, and demonstrate the performance, scalability and adaptability advantages in a simulation environment.

References

[1]

{1} P. Braam, M. Callahan, and P. Schwan. The intermezzo file system. In Proceedings of the 3rd of the Perl Conference, O'Reilly Open Source Convention, Monterey, CA, USA, Aug. 1999.

[2]

{2} P. J. Braam. The Lustre storage architecture, 2002.

[3]

{3} S. A. Brandt, L. Xue, E. L. Miller, and D. D. E. Long. Efficient metadata management in large distributed file systems. In Proceedings of the 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies , pages 290-298, Apr. 2003.

Digital Library

[4]

{4} L.-F. Cabrera and D. D. E. Long. Swift: Using distributed disk striping to provide high I/O data rates. Computing Systems, 4(4):405-436, 1991.

[5]

{5} P. F. Corbett and D. G. Feitelson. The Vesta parallel file system. ACM Transactions on Computer Systems, 14(3):225-264, 1996.

Digital Library

[6]

{6} R. A. Floyd and C. S. Ellis. Directory reference patterns in hierarchical file systems. IEEE Transactions on Knowledge and Data Engineering, 1(2):238-247, 1989.

Digital Library

[7]

{7} G. R. Ganger and M. F. Kaashoek. Embedded inodes and explicit groupings: Exploiting disk bandwidth for small files. In Proceedings of the 1997 USENIX Annual Technical Conference, pages 1-17. USENIX Association, Jan. 1997.

Digital Library

[8]

{8} S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google file system. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP'03), Bolton Landing, NY, Oct. 2003. ACM.

Digital Library

[9]

{9} G. A. Gibson, D. F. Nagle, K. Amiri, J. Butler, F. W. Chang, H. Gobioff, C. Hardin, E. Riedel, D. Rochberg, and J. Zelenka. A cost-effective, high-bandwidth storage architecture. In Proceedings of the 8th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 92-103, San Jose, CA, Oct. 1998.

Digital Library

[10]

{10} D. Hitz, J. Lau, and M. Malcom. File system design for an NFS file server appliance. In Proceedings of the Winter 1994 USENIX Technical Conference, pages 235-246, San Francisco, CA, Jan. 1994.

Digital Library

[11]

{11} R. J. Honicky and E. L. Miller. Replication under scalable hashing: A family of algorithms for scalable decentralized data distribution. In Proceedings of the 18th International Parallel & Distributed Processing Symposium (IPDPS 2004), Santa Fe, NM, Apr. 2004. IEEE.

[12]

{12} D. Long, S. Brandt, E. Miller, F. Wang, Y. Lin, L. Xue, and Q. Xin. Design and implementation of large scale object-based storage system. Technical Report ucsc-crl-02-35, University of California, Santa Cruz, Nov. 2002.

[13]

{13} M. K. McKusick, W. N. Joy, S. J. Leffler, and R. S. Fabry. A fast file system for UNIX. ACM Transactions on Computer Systems, 2(3):181-197, Aug. 1984.

Digital Library

[14]

{14} E. L. Miller and R. H. Katz. RAMA: An easy-to-use, high-performance parallel file system. Parallel Computing , 23(4):419-446, 1997.

Digital Library

[15]

{15} J. H. Morris, M. Satyanarayanan, M. H. Conner, J. H. Howard, D. S. H. Rosenthal, and F. D. Smith. Andrew: A distributed personal computing environment. Communications of the ACM, 29(3):184-201, Mar. 1986.

Digital Library

[16]

{16} J. K. Ousterhout, A. R. Cherenson, F. Douglis, M. N. Nelson, and B. B. Welch. The Sprite network operating system. IEEE Computer, 21(2):23-36, Feb. 1988.

Digital Library

[17]

{17} B. Pawlowski, C. Juszczak, P. Staubach, C. Smith, D. Lebel, and D. Hitz. NFS version 3: Design and implementation. In Proceedings of the Summer 1994 USENIX Technical Conference, pages 137-151, 1994.

[18]

{18} O. Rodeh and A. Teperman. zFS - a scalable distributed file system using object disks. In Proceedings of the 20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, pages 207-218, Apr. 2003.

Digital Library

[19]

{19} D. Roselli, J. Lorch, and T. Anderson. A comparison of file system workloads. In Proceedings of the 2000 USENIX Annual Technical Conference, pages 41-54, June 2000.

Digital Library

[20]

{20} M. Satyanarayanan, J. J. Kistler, P. Kumar, M. E. Okasaki, E. H. Siegel, and D. C. Steere. Coda: A highly available file system for a distributed workstation environment. IEEE Transactions on Computers, 39(4):447-459, 1990.

Digital Library

[21]

{21} F. Schmuck and R. Haskin. GPFS: A shared-disk file system for large computing clusters. In Proceedings of the 2002 Conference on File and Storage Technologies (FAST), pages 231-244. USENIX, Jan. 2002.

Digital Library

[22]

{22} P. Schwan. Lustre: Building a file system for 1000-node clusters. In Proceedings of the 2003 Linux Symposium, July 2003.

[23]

{23} A. Sweeney, D. Doucette, W. Hu, C. Anderson, M. Nishimoto, and G. Peck. Scalability in the XFS file system. In Proceedings of the 1996 USENIX Annual Technical Conference , pages 1-14, Jan. 1996.

Digital Library

[24]

{24} C. A. Thekkath, J. Wilkes, and E. D. Lazowska. Techniques for file system simulation. Software--Practice and Experience (SPE), 24(11):981-999, Nov. 1994.

Digital Library

[25]

{25} F. Wang, S. A. Brandt, E. L. Miller, and D. D. E. Long. OBFS: A file system for object-based storage devices. In Proceedings of the 21st IEEE/12th NASA Goddard Conference on Mass Storage Systems and Technologies, College Park, MD, Apr. 2004. IEEE.

[26]

{26} F. Wang, Q. Xin, B. Hong, S. A. Brandt, E. L. Miller, D. D. E. Long, and T. T. McLarty. File system workload analysis for large scale scientific computing applications. In Proceedings of the 21st IEEE/12th NASA Goddard Conference on Mass Storage Systems and Technologies, College Park, MD, Apr. 2004.

[27]

{27} J. Wilkes. The Pantheon storage-system simulator. Technical Report HPL-SSP-95-14, Storage Systems Program, Computer Systems Laboratory, Hewlett-Packard Laboratories, Palo Alto, CA, May 1996.

[28]

{28} Q. Xin, E. L. Miller, T. J. Schwarz, D. D. E. Long, S. A. Brandt, and W. Litwin. Reliability mechanisms for very large storage systems. In Proceedings of the 20th IEEE/ 11th NASA Goddard Conference on Mass Storage Systems and Technologies, pages 146-156, Apr. 2003.

Digital Library

Cited By

Carver BHan RZhang JZheng MCheng YAamodt TSwift MJerger N(2023)λFS: A Scalable and Elastic Distributed File System Metadata Service using Serverless FunctionsProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 410.1145/3623278.3624765(394-411)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3623278.3624765
Wang YWu YLi CZheng PCao BSun YZhou FXu YWang YXie GFedorova ANarayanan DDi Luna GQuerzoni L(2023)CFS: Scaling Metadata Service for Distributed File System via Pruned Scope of Critical SectionsProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3587443(331-346)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3552326.3587443
Wang YLi CShao XChen YYan FXu Yde Supinski BHall MGamblin T(2021)LunuleProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3458817.3476196(1-16)Online publication date: 14-Nov-2021
https://dl.acm.org/doi/10.1145/3458817.3476196
Show More Cited By

Recommendations

Optimizing of metadata management in large-scale file systems

As modern computer systems face the challenge of large data, filesystems have to deal with a large number of files. This leads to amplified concerns of metadata operations as well as data operations. Most filesystems manage metadata of files by ...
Metadata Management of the SANtopia File System
ICPADS '01: Proceedings of the Eighth International Conference on Parallel and Distributed Systems

Abstract: Recently, LINUX cluster file systems using the storage area network (SAN) have been developed. The Global File System (GFS) implemented at the University of Minnesota is such and example. These systems do not use a central file server, and ...
High Performance Metadata Management Engine for Large-Scale Distributed File Systems
FGCN '15: Proceedings of the 2015 9th International Conference on Future Generation Communication and Networking (FGCN)

This paper presents a metadata management engine with considerable performance and scalability that can be used in a nationwide file storage service. While a metadata server cluster is needed to support a huge number of metadata requests in a large-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing

November 2004

724 pages

ISBN:0769521533

General Chair:
Jeff Huskamp

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

IEEE Computer Society

United States

Publication History

Published: 06 November 2004

Check for updates

Qualifiers

Article

Conference

SC '04

Sponsor:

SIGARCH
IEEE-CS

SC '04: International Conference for High Performance Computing, Networking, Storage and Analysis

November 6 - 12, 2004

Acceptance Rates

SC '04 Paper Acceptance Rate 60 of 200 submissions, 30%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

51
Total Citations
View Citations
76
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Carver BHan RZhang JZheng MCheng YAamodt TSwift MJerger N(2023)λFS: A Scalable and Elastic Distributed File System Metadata Service using Serverless FunctionsProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 410.1145/3623278.3624765(394-411)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3623278.3624765
Wang YWu YLi CZheng PCao BSun YZhou FXu YWang YXie GFedorova ANarayanan DDi Luna GQuerzoni L(2023)CFS: Scaling Metadata Service for Distributed File System via Pruned Scope of Critical SectionsProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3587443(331-346)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3552326.3587443
Wang YLi CShao XChen YYan FXu Yde Supinski BHall MGamblin T(2021)LunuleProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3458817.3476196(1-16)Online publication date: 14-Nov-2021
https://dl.acm.org/doi/10.1145/3458817.3476196
Sun JHuang JSnir Mde Supinski BHall MGamblin T(2021)Pinpointing crash-consistency bugs in the HPC I/O stackProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3458817.3476144(1-13)Online publication date: 14-Nov-2021
https://dl.acm.org/doi/10.1145/3458817.3476144
Cai SWang DWang ZWang H(2020)CARD: A Congestion-Aware Request Dispatching Scheme for Replicated Metadata Server ClusterProceedings of the 49th International Conference on Parallel Processing10.1145/3404397.3404411(1-11)Online publication date: 17-Aug-2020
https://dl.acm.org/doi/10.1145/3404397.3404411
Mandal ASarkar A(2019)A Novel Meta-Information Management System for SaaSInternational Journal of Cloud Applications and Computing10.4018/IJCAC.20190701019:3(1-21)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.4018/IJCAC.2019070101
Gao YGao XChen G(2019)DeepHashProceedings of the 48th International Conference on Parallel Processing10.1145/3337821.3337924(1-10)Online publication date: 5-Aug-2019
https://dl.acm.org/doi/10.1145/3337821.3337924
Cao SGao YGao XChen G(2019)AdaMProceedings of the 48th International Conference on Parallel Processing10.1145/3337821.3337822(1-10)Online publication date: 5-Aug-2019
https://dl.acm.org/doi/10.1145/3337821.3337822
Liu HDing WChen YGuo WLiu SLi TZhang MZhao JZhu HZhu ZBoncz PManegold SAilamaki ADeshpande AKraska T(2019)CFSProceedings of the 2019 International Conference on Management of Data10.1145/3299869.3314046(1729-1742)Online publication date: 25-Jun-2019
https://dl.acm.org/doi/10.1145/3299869.3314046
Zheng QCranor CGuo DGanger GAmvrosiadis GGibson GSettlemyer BGrider GGuo F(2018)Scaling embedded in-situ indexing with deltaFSProceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis10.5555/3291656.3291660(1-15)Online publication date: 11-Nov-2018
https://dl.acm.org/doi/10.5555/3291656.3291660
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents