Article

Integrating collective I/O and cooperative caching into the "clusterfile" parallel file system

Authors:

Walter TichyAuthors Info & Claims

ICS '04: Proceedings of the 18th annual international conference on Supercomputing

Pages 58 - 67

https://doi.org/10.1145/1006209.1006219

Published: 26 June 2004 Publication History

Abstract

This paper presents the integration of two collective I/O techniques into the Clusterfile parallel file system: disk-directed I/O and two-phase I/O. We show that global cooperative cache management improves the collective I/O performance. The solution focuses on integrating disk parallelism with other types of parallelism: memory (by buffering and caching on several nodes), network (by parallel I/O scheduling strategies) and processors (by redistributing the I/O related computation over several nodes). The performance results show considerable throughput increases over ROMIO's extended two-phase I/O.

References

[1]

T. Anderson, M. Dahlin, J. M. Neefe, D. Patterson, D. Rosseli, and R. Y. Wang. Serverless Network File Systems. In The 15th Symposium on Operating System Principles, Dec. 1995.]]

Digital Library

[2]

R. Bagrodia, S. Docy, and A. Kahn. Parallel simulation of parallel file systems and i/o programs. In Proceedings of the 1997 ACM/IEEE conference on Supercomputing (CDROM), pages 1--17. ACM Press, 1997.]]

Digital Library

[3]

R. Bordawekar. Implementation of Collective I/O in the Intel Paragon Parallel File System: Initial Experiences. In Proc. 11th International Conference on Supercomputing, July 1997. To appear.]]

Digital Library

[4]

F. Chen and S. Majumdar. Performance of parallel I/O scheduling strategies on a network of workstations. In Proceedings of ICPADS 2001, pages 157--164, April 2001.]]

Digital Library

[5]

Y. Cho, M. Winslett, Y. Chen, and S. wen Kuo. Parallel I/O performance of fine grained data distributions. In Proceedings of the Seventh IEEE International Symposium on High Performance Distributed Computing. IEEE Computer Society Press, 1998.]]

Digital Library

[6]

P. Corbett and D. Feitelson. The Vesta Parallel File System. ACM Transactions on Computer Systems, 1996.]]

Digital Library

[7]

T. Cortes, S. Girona, and L. Labarta. PACA: A Distributed File System Cache for Parallel Machines. Performance under Unix-like workload. Technical Report UPC-DAC-RR-95/20 or UPC-CEPBA-RR-95/13, Departament d'Arquitectura de Computadors, Universitat Politecnica de Catalunya, 1995.]]

[8]

M. Dahlin, R. Wang, T. Anderson, and D. Patterson. Cooperative Caching: Using Remote Client Memory to Improve File System Performance. In The First Symp. on Operating Systems Design and Implementation, Nov. 1994.]]

Digital Library

[9]

E. DeBenedictis and J. D. Rosario. nCUBE Parallel I/O Software. In Proceedings of 11th International Phoenix Conference on Computers and Communication, 1992.]]

[10]

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. of IPPS Workshop on Input/Output in Parallel Computer Systems, 1993.]]

Digital Library

[11]

J. Huber, C. Elford, D. Reed, A. Chien, and D. Blumenthal. Ppfs: A high performance portable file system. In Proceedings of the 9th ACM International Conference on Supercomputing, 1995.]]

Digital Library

[12]

W. L. III and R. Ross. An Overview of the Parallel Virtual File System. In Proceedings of the Extreme Linux Workshop, June 1999.]]

[13]

Intel Corporation. Paragon System User's Guide, April 1996.]]

[14]

F. Isaila and W. Tichy. Clusterfile: A flexible physical layout parallel file system. In First IEEE International Conference on Cluster Computing, Oct. 2001.]]

Digital Library

[15]

F. Isaila and W. Tichy. View I/O: Improving the performance of non-contiguous I/O. In Third IEEE International Conference on Cluster Computing, Dec. 2003.]]

[16]

R. Jain, K. Somalwar, J. Werth, and J. C. Browne. Heuristics for scheduling I/O operations. IEEE Transactions on Parallel and Distributed Systems, 8(3):310--320, March 1997.]]

Digital Library

[17]

D. Kotz. Disk-directed I/O for MIMD Multiprocessors. In Proc. of the First USENIX Symp. on Operating Systems Design and Implementation, 1994.]]

Digital Library

[18]

D. B. Loveman. High Performance Fortran. IEEE Parallel and Distributed Technology, 1993.]]

Digital Library

[19]

Message Passing Interface Forum. MPI: A Message-Passing Interface Standard, 1995.]]

[20]

Message Passing Interface Forum. MPI2: Extensions to the Message Passing Interface, 1997.]]

[21]

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. Ellis, and M. Best. File Access Characteristics of Parallel Scientific Workloads. In IEEE Transactions on Parallel and Distributed Systems, 7(10), Oct. 1996.]]

Digital Library

[22]

F. Schmuck and R. Haskin. GPFS: A Shared-Disk File System for Large Computing Clusters. In Proceedings of FAST, 2002.]]

Digital Library

[23]

K. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett. Server-directed collective I/O in Panda. In Proceedings of Supercomputing '95.]]

Digital Library

[24]

H. Simitici and D. Reed. A Comparison of Logical and Physical Parallel I/O Patterns. In International Journal of High Performance Computing Applications, special issue (I/O in Parallel Applications), 12(3), 1998.]]

[25]

R. Thakur, W. Gropp, and E. Lusk. Data Sieving and Collective I/O in ROMIO. In Proc. of the 7th Symposium on the Frontiers of Massively Parallel Computation, pages 182--189, February 1999.]]

Digital Library

[26]

M. I. G. the low-level message-passing system for Myrinet networks. http://www.myri.com/scs/index.html.]]

[27]

G. M. Voelker, E. J. Anderson, T. Kimbrel, M. J. Feeley, J. S. Chase, A. R. Karlin, and H. M. Levy. Implementing cooperative prefetching and caching in a globally-managed memory system. In Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems, pages 33--43. ACM Press, 1998.]]

Digital Library

[28]

P. Wong and R. der Wijngaart. NAS Parallel Benchmarks I/O Version 2.4. Technical Report NAS-03-002, NASA Ames Research Center, Moffet Field, CA, Jan. 2003.]]

Cited By

Qin HZawad SZhou YYang LZhao DYan FTaufer MBalaji PPeña A(2019)Swift machine learning model serving schedulingProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3295500.3356164(1-23)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1145/3295500.3356164
Isaila FCarretero JRoss RVarela CCastro HBarrios C(2016)CLARISSEProceedings of the 16th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing10.1109/CCGrid.2016.24(346-355)Online publication date: 16-May-2016
https://dl.acm.org/doi/10.1109/CCGrid.2016.24
Dong WLiu GYu JHu WLiu X(2015)SFDCProceedings of the 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conf on Embedded Software and Systems10.1109/HPCC-CSS-ICESS.2015.185(342-350)Online publication date: 24-Aug-2015
https://dl.acm.org/doi/10.1109/HPCC-CSS-ICESS.2015.185
Show More Cited By

Index Terms

Integrating collective I/O and cooperative caching into the "clusterfile" parallel file system

Recommendations

Caching and writeback policies in parallel file systems
SPDP '91: Proceedings of the 1991 Third IEEE Symposium on Parallel and Distributed Processing

Improvements in the processing speed of multiprocessors are outpacing improvements in the speed of disk hardware. Parallel disk I/O subsystems have been proposed as one way to close the gap between processor and disk speeds. Such parallel disk systems ...
Cooperative Client-Side File Caching for MPI Applications

Client-side file caching is one of many I/O strategies adopted by today's parallel file systems that were initially designed for distributed systems. Most of these implementations treat each client independently because ...
Compiler-Directed Collective-I/O

Current approaches to parallel I/O demand extensive user effort to obtain acceptable performance. This is in part due to difficulties in understanding the characteristics of a wide variety of I/O devices and in part due to inherent complexity of I/O ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICS '04: Proceedings of the 18th annual international conference on Supercomputing

June 2004

360 pages

ISBN:1581138393

DOI:10.1145/1006209

General Chair:
Paul Feautrier
LIP, ENS Lyon
,
Program Chairs:
James Goodman
University of Auckland
,
André Seznec
IRISA, INRIA

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICS04

Sponsor:

ICS04: International Conference on Supercomputing 2004

June 26 - July 1, 2004

Malo, France

Acceptance Rates

Overall Acceptance Rate 629 of 2,180 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
677
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Qin HZawad SZhou YYang LZhao DYan FTaufer MBalaji PPeña A(2019)Swift machine learning model serving schedulingProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3295500.3356164(1-23)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1145/3295500.3356164
Isaila FCarretero JRoss RVarela CCastro HBarrios C(2016)CLARISSEProceedings of the 16th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing10.1109/CCGrid.2016.24(346-355)Online publication date: 16-May-2016
https://dl.acm.org/doi/10.1109/CCGrid.2016.24
Dong WLiu GYu JHu WLiu X(2015)SFDCProceedings of the 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conf on Embedded Software and Systems10.1109/HPCC-CSS-ICESS.2015.185(342-350)Online publication date: 24-Aug-2015
https://dl.acm.org/doi/10.1109/HPCC-CSS-ICESS.2015.185
Huang YJin HShi XWu SChen Y(2013)Cost-Aware Client-Side File Caching for Data-Intensive ApplicationsProceedings of the 2013 IEEE International Conference on Cloud Computing Technology and Science - Volume 0210.1109/CloudCom.2013.140(248-251)Online publication date: 2-Dec-2013
https://dl.acm.org/doi/10.1109/CloudCom.2013.140
Song HYin YChen YSun X(2013)Cost-intelligent application-specific data layout optimization for parallel file systemsCluster Computing10.1007/s10586-012-0200-416:2(285-298)Online publication date: 1-Jun-2013
https://dl.acm.org/doi/10.1007/s10586-012-0200-4
Fazenda ARodrigues ETomita SPanetta JMendes C(2012)Improving the Scalability of an Operational Scientific Application in a Large Multi-core ClusterProceedings of the 2012 13th Symposium on Computing Systems10.1109/WSCAD-SSC.2012.29(126-132)Online publication date: 17-Oct-2012
https://dl.acm.org/doi/10.1109/WSCAD-SSC.2012.29
Arai JIshikawa Y(2012)Scalable Cooperative Caching with RDMA-Based Directory Management for Large-Scale Data ProcessingProceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis10.1109/SC.Companion.2012.312(1538-1539)Online publication date: 10-Nov-2012
https://dl.acm.org/doi/10.1109/SC.Companion.2012.312
Fazenda APanetta JKatsurayama DRodrigues LMotta LNavaux P(2011)Challenges and solutions to improve the scalability of an operational regional meteorological forecasting modelInternational Journal of High Performance Systems Architecture10.1504/IJHPSA.2011.0404623:2/3(87-97)Online publication date: 1-May-2011
https://dl.acm.org/doi/10.1504/IJHPSA.2011.040462
Song HYin YChen YSun XMaccabe AThain D(2011)A cost-intelligent application-specific data layout scheme for parallel file systemsProceedings of the 20th international symposium on High performance distributed computing10.1145/1996130.1996138(37-48)Online publication date: 8-Jun-2011
https://dl.acm.org/doi/10.1145/1996130.1996138
Isaila FGarcia Blas JCarretero JLatham RRoss R(2011)Design and Evaluation of Multiple-Level Data Staging for Blue Gene SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2010.12722:6(946-959)Online publication date: 1-Jun-2011
https://dl.acm.org/doi/10.1109/TPDS.2010.127
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten