Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

An Implementation of MPI-IO on Expand: A Parallel File System Based on NFS Servers

  • Conference paper
  • First Online:
Recent Advances in Parallel Virtual Machine and Message Passing Interface (EuroPVM/MPI 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2474))

Abstract

This paper describes an implementation of MPI-IO using a new parallel file system, called Expand (Expandable Parallel File System)1, that is based on NFS servers. Expand combines multiple NFS servers to create a distributed partition where files are declustered. Expand requires no changes to the NFS server and uses RPC operations to provide parallel access to the same file. Expand is also independent of the clients, because all operations are implemented using RPC and NFS protocol. The paper describes the design, the implementation and the evaluation of Expand with MPI-IO. This evaluation has been made in Linux clusters and compares Expand and PVFS.

This work has been partially support by the Spanish Ministry of Science and Technology under the TIC2000-0469 and TIC2000-0471 contracts, and by the Community of Madrid under the 07T/0013/2001 contract.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. P.H. Carns, W.B. Ligon III, R.B. Ross, and R. Takhur. PVFS: A Parallel File System for Linux Clusters. Technical Report ANL/MCS-P804-0400, 2000.

    Google Scholar 

  2. J. Carretero, F. Perez, P. de Miguel, F. Garcia, and L. Alonso. Improving the Performance of Parallel File Systems. Parallel Computing: Special Issue on Parallel I/O Systems. Elsevier, (3):525–542, April 1997.

    Google Scholar 

  3. P. Corbett, S. Johnson, and D. Feitelson. Overview of the Vesta Parallel File System. ACM Computer Architecture News, 21(5):7–15, December 1993.

    Google Scholar 

  4. J. S. Chase D.C. Anderson and A.M. Vahdat. Interposed request routing for scalable network storage. In Fourth Symposium on Operating System Design and Implementation (OSDI2000), 2000.

    Google Scholar 

  5. G. Gibson. The Scotch Paralell Storage Systems. Technical Report CMU-CS-95-107, Scholl of Computer Science, Carnegie Mellon University, Pittsburbh, Pennsylvania, 1995.

    Google Scholar 

  6. J. Huber, C.L. Elford, and et al. PPFS: A High Performance Portable Parallel File System. In Proceedings of the 9th ACM International Conference on Supercomputing, pages 385–394. IEEE, July 1995.

    Google Scholar 

  7. G. H. Kim and R. G. Minninch. Bigfoot-NFS: A Parallel File-Striping NFS Server. Technical report, Sun Microsystems Computer Corp., 1994.

    Google Scholar 

  8. O. Krieger. HFS: A Flexible File System for Shared-Memory Multiprocessors. PhD thesis, Department of Electrical and Computer Engineering, University of Toronto, 1994.

    Google Scholar 

  9. T. Madhyastha. Automatic Classification of Input/Output Access Patterns. PhD thesis, niversidad de Illinois, Urbana-Champaign, 1997.

    Google Scholar 

  10. S.A. Moyer and V.S. Sunderam. PIOUS: A Scalable Parallel I/O System for Distributed Computing Environments. In Proceedings of the Scalable High-Performance Computing Conferece, pages 71–78, 1994.

    Google Scholar 

  11. N. Nieuwejaar and D. Kotz. The Galley Parallel File System. In Proceedings of the 10th ACM International Conference on Supercomputing, May 1996.

    Google Scholar 

  12. R. Olfield and D. Kotz. The armada parallel file system, 1998. http://www.cs.dartmouth.edu/~dfk/armada/design.html.

  13. P. Pierce. A Concurrent File System for a Highly Parallel Mass Storage Subsystem. In John L. Gustafson, editor, Proceedings of the Fourth Conference on Hy-percubes Concurrent Computers and Applications, pages 155–161. HCCA, March 1989.

    Google Scholar 

  14. W. Gropp R. Takhur and E. Lusk. On Implementing MPI-IO Portably and with High Performance. In of the Sixth Workshop on I/O in Parallel and Distributed Systems, pages 23–32, 1999.

    Google Scholar 

  15. R. Rabenseifner and A. E. Koniges. Effective Communication and File-I/O Bandwidth Benchmark. In J. Dongarra and Yiannis Cotronis (Eds.), Recent Advances in Parallel Virtual Machine and Message Passing Interface, proceedings of the 8th European PVM/MPI Users’ Group Meeting, EuroPVM/MPI2001, Santorini, Greece, Sep. 23–26, LNCS 2131, pp 24–35.

    Chapter  Google Scholar 

  16. R. Sandberg, D. Goldberg, S. Kleiman, D Walsh, and B. Lyon. Design and Implementation of the SUN Network Filesystem. In Proc. of the 1985 USENIX Conference. USENIX, 1985.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Architecture Group, Computer Science Department, Universidad Carlos III de Madrid, Leganés, Madrid, Spain

    Alejandro Calderón, Félix García, Jesús Carretero, Jose M. Pérez & Javier Fernández

Authors
  1. Alejandro Calderón
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Félix García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jesús Carretero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jose M. Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Javier Fernández
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Abteilung für Graphische und Parallele Datenverarbeitung (GUP) Institut für Technische Informatik und Telematik, Johannes Kepler Universität Linz, Altenbergstr. 69, 4040, Linz, Austria

    Dieter Kranzlmüller  & Jens Volkert  & 

  2. Computer and Automation Research Institute, MTA SZTAKI, Hungarian Academy of Sciences, Victor Hugo u. 18-22, 1132, Budapest, Hungary

    Peter Kacsuk

  3. Computer Science Department Innovative Computing Laboratory, University of Tennessee, 1122 Volunteer Blvd, Knoxville, TN, 37996-3450, USA

    Jack Dongarra

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Calderón, A., García, F., Carretero, J., Pérez, J.M., Fernández, J. (2002). An Implementation of MPI-IO on Expand: A Parallel File System Based on NFS Servers. In: Kranzlmüller, D., Volkert, J., Kacsuk, P., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2002. Lecture Notes in Computer Science, vol 2474. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45825-5_47

Download citation

  • DOI: https://doi.org/10.1007/3-540-45825-5_47

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-44296-7

  • Online ISBN: 978-3-540-45825-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation