Abstract
Parallel IO (PIO) support for larger scale computing is becoming more important as application developers better understand its importance in reducing overall execution time by avoiding IO overheads. This situation has been made more critical as processor speed and overall system size has increased at a far greater rate than sequential IO performance. Systems such as MPI_Connect and PACX-MPI allow multiple MPPs to be interconnected, complicating IO issues further. MPI_Connect implemented Parallel IO support for distributed applications in the MPI_Conn_IO package by transferring complete sections of files to remote machines, supporting the case that all the applications and the file storage were completely distributed. This system had a number of performance drawbacks compared to the more common usage of metacomputing where some files and applications have an affinity to a home site and thus less data transfer is required. Here we present the new PACX-MPI PIO system based initially on MPI_Connect IO, and attempt to demonstrate multiple methods of handling MPI PIO that cover a greater number of possible usage scenarios. Given are some preliminary performance results as well as a comparison to other PIO grid systems such as the Asian Pacific GridFarm, and Globus gridFTP, GASS and RIO.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
G. E. Fagg, K. S. London, and J. J. Dongarra, “MPI_Connect: managing heterogeneous MPI applications interoperation and process control”, in V. Alexandrov and J. Dongarra, editors, Recent advances in Parallel Virtual Machine and Message Passing Interface, volume 1497 of Lecture notes of Computer Science, pages 93–96. Springer, 1998. 5th European PVM/MPI User’s Group Meeting.
D. Cronk, G. Fagg, and S. Moore, “Parallel I/O for EQM Applications”, Department of Computer Science technical report, University of Tenneesee, Knoxville, July 2000.
E. Gabriel, M. Resch, T. Beisel and R. Keller“Distributed Computing in a heterogeneous computing environment”, in V. Alexandrov and J. Dongarra, editors, Recent advances in Parallel Virtual Machine and Message Passing Interface, volume 1497 of Lecture notes of Computer Science, pages 180–188. Springer, 1998. 5th European PVM/MPI User’s Group Meeting.
T. Imamura, Y. Tsujita, H. Koide, and H. Takemiya, “An architecture of Stampi: MPI library on a cluster of parallel computers”, in J. Dongarra, P. Kacsuk, and N. Podhorszki, editors, Recent advances in Parallel Virtual Machine and Message Passing Interface, volume 1908 of Lecture notes of Computer Science, pages 200–207. Springer, 2000. 7th European PVM/MPI User’s Group Meeting.
Rolf Rabenseifner and Alice E. Koniges,’ Effective File-I/O Bandwidth Benchmark’, in A. Bode, T. Ludwig, R. Wissmüller, editors, Proceedings of Euro-Par 200, pages 1273–1283, Springer 2000.
B. Allock et all, “Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing”, submitted to IEEE Mass Storage Conference, April 2001.
R. Thakur, W. Gropp, and E. Lusk, “On Implementing MPI-IO Portably and with High Performance”, in Proc. Of the Sixth Workshop on I/O in Parallel and Distributed Systems, May 1999, pages 23–32.
J. Bester, I. Foster, C. Kesselmann, J. Tedesco, S. Tuecke, “GASS: A Data Movement and Access Service for Wide Area Computing Systems”, in Sixth Workshop on I/O in Parallel and Distributed Systems, 1999.
I. Foster, D. Kohr, R. Krishnaiyer, J. Mogill, “Remote I/O: Fast Access to Distant Storage”, in Proc. Workshop on I/O in Parallel and Distributed Systems, (IOPADS), pages 14–25, 1997.
O. Tatebe et all., “Grid Data Fram for Petascale Data Instensive Computing”, Elechtro-technical Laboratories, Technical Report, TR-2001-4, http://datafarm.apgrid.org.
B. Tierny, W. Johnston, J. Lee, M. Thompson, “A Data Intensive Distributed Computing Architecture for Grid Applications”, Future Generation Computer Systems, volume 16 no 5, pages 473–481, Elsevier Science, 2000.
J. Morris, M. Satyanarayanan, M. Conner, J. Howard, D. Rosenthal and F. Smith. “Andrew: A distributed personal computing environment”, Communications of the ACM, 29(3):184–201, 1986.
William Gropp, et. Al., MPI-The Complete Reference, Volume 2, The MPI Extensions, The MIT Press, Cambridge, MA 1999
William Gropp, Ewing Lusk, and Rajeev Thakur, Using MPI-2, Advanced Features of the Message-Passing Interface, The MIT Press, Cambridge, MA 1999
James S. Plank, Micah Beck, Wael R. Elwasif, Terry Moore, Martin Swany, Rich Wolski “The Internet Backplane Protocol: Storage in the Network”, NetStore99: The Network Storage Symposium, (Seattle, WA, 1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fagg, G.E., Gabriel, E., Resch, M., Dongarra, J.J. (2001). Parallel IO Support for Meta-computing Applications: MPI_Connect IO Applied to PACX-MPI. In: Cotronis, Y., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2001. Lecture Notes in Computer Science, vol 2131. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45417-9_22
Download citation
DOI: https://doi.org/10.1007/3-540-45417-9_22
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42609-7
Online ISBN: 978-3-540-45417-5
eBook Packages: Springer Book Archive