This book offers a practical guide to the advanced features of the MPI (Message-Passing Interface) standard library for writing programs for parallel computers. It covers new features added in MPI-3, the latest version of the MPI standard, and updates from MPI-2. Like its companion volume, Using MPI, the book takes an informal, example-driven, tutorial approach. The material in each chapter is organized according to the complexity of the programs used as examples, starting with the simplest example and moving to more complex ones. Using Advanced MPI covers major changes in MPI-3, including changes to remote memory access and one-sided communication that simplify semantics and enable better performance on modern hardware; new features such as nonblocking and neighborhood collectives for greater scalability on large systems; and minor updates to parallel I/O and dynamic processes. It also covers support for hybrid shared-memory/message-passing programming; MPI_Message, which aids in certain types of multithreaded programming; features that handle very large data; an interface that allows the programmer and the developer to access performance data; and a new binding of MPI to Fortran.
Cited By
Jasny M, Thostrup L, Tamimi S, Koch A, István Z and Binnig C (2024). Zero-sided RDMA: Network-driven Data Shuffling for Disaggregated Heterogeneous Cloud DBMSs, Proceedings of the ACM on Management of Data, 2:1, (1-28), Online publication date: 12-Mar-2024.- Schneider N, Kadosh T, Hasabnis N, Mattson T, Pinter Y and Oren G MPI-RICAL: Data-Driven MPI Distributed Parallelism Assistance with Transformers Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, (2-10)
- Lei X, Gu T, Graillat S, Xu X and Meng J Comparison of Reproducible Parallel Preconditioned BiCGSTAB Algorithm Based on ExBLAS and ReproBLAS Proceedings of the International Conference on High Performance Computing in Asia-Pacific Region, (46-54)
- Yang J, Puri S and Zhou H Fine-grained dynamic load balancing in spatial join by work stealing on distributed memory Proceedings of the 30th International Conference on Advances in Geographic Information Systems, (1-12)
- Ben-Nun T and Hoefler T (2019). Demystifying Parallel and Distributed Deep Learning, ACM Computing Surveys, 52:4, (1-43), Online publication date: 31-Jul-2020.
- Rivas-Gomez S, Fanfarillo A, Narasimhamurthy S and Markidis S Persistent coarrays Proceedings of the 26th European MPI Users' Group Meeting, (1-8)
- Gerstenberger R, Besta M and Hoefler T (2018). Enabling highly scalable remote memory access programming with MPI-3 one sided, Communications of the ACM, 61:10, (106-113), Online publication date: 26-Sep-2018.
- Puri S, Paudel A and Prasad S MPI-Vector-IO Proceedings of the 47th International Conference on Parallel Processing, (1-11)
- Raffenetti K, Amer A, Oden L, Archer C, Bland W, Fujita H, Guo Y, Janjusic T, Durnov D, Blocksome M, Si M, Seo S, Langer A, Zheng G, Takagi M, Coffman P, Jose J, Sur S, Sannikov A, Oblomov S, Chuvelev M, Hatanaka M, Zhao X, Fischer P, Rathnayake T, Otten M, Min M and Balaji P Why is MPI so slow? Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, (1-12)
- Rivas-Gomez S, Gioiosa R, Peng I, Kestor G, Narasimhamurthy S, Laure E and Markidis S MPI windows on storage for HPC applications Proceedings of the 24th European MPI Users' Group Meeting, (1-11)
- Zerpa L The Message-Passing Interface and Parallel SAT-Solvers Proceedings of the International Conference on Future Networks and Distributed Systems, (1-7)
- Lazzaro A, VandeVondele J, Hutter J and Schütt O Increasing the Efficiency of Sparse Matrix-Matrix Multiplication with a 2.5D Algorithm and One-Sided MPI Proceedings of the Platform for Advanced Scientific Computing Conference, (1-9)
- Kaplan F, Tuncer O, Leung V, Hemmert S and Coskun A Unveiling the Interplay Between Global Link Arrangements and Network Management Algorithms on Dragonfly Networks Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, (325-334)
- Rivas-Gomez S, Markidis S, Peng I, Laure E, Kestor G and Gioiosa R Extending Message Passing Interface Windows to Storage Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, (727-730)
- Kouetcha D, Ramézani H and Cohaut N (2017). Ultrafast scalable parallel algorithm for the radial distribution function histogramming using MPI maps, The Journal of Supercomputing, 73:4, (1629-1653), Online publication date: 1-Apr-2017.
- Barthels C, Müller I, Schneider T, Alonso G and Hoefler T (2017). Distributed join algorithms on thousands of cores, Proceedings of the VLDB Endowment, 10:5, (517-528), Online publication date: 1-Jan-2017.
- Markidis S, Peng I, Iakymchuk R, Laure E, Kestor G and Gioiosa R (2016). A Performance Characterization of Streaming Computing on Supercomputers, Procedia Computer Science, 80:C, (98-107), Online publication date: 1-Jun-2016.
- Schmid P, Besta M and Hoefler T High-Performance Distributed RMA Locks Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing, (19-30)
Index Terms
- Using Advanced MPI: Modern Features of the Message-Passing Interface
Recommendations
MPI + MPI: a new hybrid approach to parallel programming with MPI plus shared memory
Hybrid parallel programming with the message passing interface (MPI) for internode communication in conjunction with a shared-memory programming model to manage intranode parallelism has become a dominant approach to scalable parallel programming. While ...