Article

Dynamic Malleability in Iterative MPI Applications

Authors:

Kaoutar El Maghraoui,

Travis J. Desell,

Boleslaw K. Szymanski,

Carlos A. VarelaAuthors Info & Claims

CCGRID '07: Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid

Pages 591 - 598

https://doi.org/10.1109/CCGRID.2007.45

Published: 14 May 2007 Publication History

Publisher Site

Abstract

Malleability enables a parallel application's execution system to split or merge processes modifying granularity. While process migration is widely used to adapt applications to dynamic execution environments, it is limited by the granularity of the application's processes. Malleability empowers process migration by allowing the application's processes to expand or shrink following the availability of resources. We have implemented malleability as an extension to the PCM (Process Checkpointing and Migration) library, a user-level library for iterative MPI applications. PCM is integrated with the Internet Operating System (IOS), a framework for middleware-driven dynamic application reconfiguration. Our approach requires minimal code modifications and enables transparent middlewaretriggered reconfiguration. Experimental results using a two-dimensional data parallel program that has a regular communication structure demonstrate the usefulness of malleability.

Cited By

View all

Finnerty PPosner JBürger JTakaoka LKanzaki T(2024)On the Performance of Malleable APGAS Programs and Batch Job SchedulersSN Computer Science10.1007/s42979-024-02641-75:4Online publication date: 27-Mar-2024
https://dl.acm.org/doi/10.1007/s42979-024-02641-7
Eberius DRahman MOzog D(2023)Evaluating the Potential of Elastic Jobs in HPC SystemsProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624199(1324-1333)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624199
Martín Álvarez IAliaga JCastillo MIserte S(2023)Efficient data redistribution for malleable applicationsProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624110(416-426)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624110
Show More Cited By

Index Terms

Dynamic Malleability in Iterative MPI Applications

Recommendations

Malleable iterative MPI applications
The Best of CCGrid'2007: A Snapshot of an ‘Adolescent’ Area

Malleability enables a parallel application's execution system to split or merge processes modifying granularity. While process migration is widely used to adapt applications to dynamic execution environments, it is limited by the granularity of the ...
Towards Achieving Transparent Malleability Thanks to MPI Process Virtualization
High Performance Computing
Abstract
The field of High-Performance Computing is rapidly evolving, driven by the race for computing power and the emergence of new architectures. Despite these changes, the process of launching programs has remained largely unchanged, even with the rise ...
Dynamic spawning of MPI processes applied to malleability

Malleability allows computing facilities to adapt their workloads through resource management systems to maximize the throughput of the facility and the efficiency of the executed jobs. This technique is based on reconfiguring a job to a different ...

Comments

Information & Contributors

Information

Published In

CCGRID '07: Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid

May 2007

900 pages

ISBN:0769528333

Publisher

IEEE Computer Society

United States

Publication History

Published: 14 May 2007

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Finnerty PPosner JBürger JTakaoka LKanzaki T(2024)On the Performance of Malleable APGAS Programs and Batch Job SchedulersSN Computer Science10.1007/s42979-024-02641-75:4Online publication date: 27-Mar-2024
https://dl.acm.org/doi/10.1007/s42979-024-02641-7
Eberius DRahman MOzog D(2023)Evaluating the Potential of Elastic Jobs in HPC SystemsProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624199(1324-1333)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624199
Martín Álvarez IAliaga JCastillo MIserte S(2023)Efficient data redistribution for malleable applicationsProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624110(416-426)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624110
Finnerty PTakaoka LKanzaki TPosner J(2023)Malleable APGAS Programs and Their Support in Batch Job SchedulersEuro-Par 2023: Parallel Processing Workshops10.1007/978-3-031-48803-0_8(89-101)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-48803-0_8
Agullo EAltenbernd MAnzt HBautista-Gomez LBenacchio TBonaventura LBungartz HChatterjee SCiorba FDeBardeleben NDrzisga DEibl SEngelmann CGansterer WGiraud LGöddeke DHeisig MJézéquel FKohl NLi XLion RMehl MMycek PObersteiner MQuintana-Ortí ERizzi FRüde USchulz MFung FSpeck RStals LTeranishi KThibault SThönnes DWagner AWohlmuth B(2022)Resiliency in numerical algorithm design for extreme scale simulationsInternational Journal of High Performance Computing Applications10.1177/1094342021105518836:2(251-285)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1177/10943420211055188
Huber DStreubel MComprés ISchulz MSchreiber MPritchard H(2022)Towards Dynamic Resource Management with MPI Sessions and PMIxProceedings of the 29th European MPI Users' Group Meeting10.1145/3555819.3555856(57-67)Online publication date: 14-Sep-2022
https://dl.acm.org/doi/10.1145/3555819.3555856
Posner JFohry C(2021)Transparent Resource Elasticity for Task-Based Cluster Environments with Work Stealing50th International Conference on Parallel Processing Workshop10.1145/3458744.3473361(1-10)Online publication date: 9-Aug-2021
https://dl.acm.org/doi/10.1145/3458744.3473361
D'Amico MGarcia-Gasulla MLópez VJokanovic ASirvent RCorbalan J(2018)DROMWorkshop Proceedings of the 47th International Conference on Parallel Processing10.1145/3229710.3229752(1-10)Online publication date: 13-Aug-2018
https://dl.acm.org/doi/10.1145/3229710.3229752
Rodrigo Álvarez GÖstberg PElmroth ERamakrishnan LDesprez FLebre AElmroth E(2015)A2L2Proceedings of the 8th International Workshop on Virtualization Technologies in Distributed Computing10.1145/2755979.2755983(11-19)Online publication date: 15-Jun-2015
https://dl.acm.org/doi/10.1145/2755979.2755983
Sarood OLanger AGupta AKale LDamkroger TDongarra J(2014)Maximizing throughput of overprovisioned HPC data centers under a strict power budgetProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC.2014.71(807-818)Online publication date: 16-Nov-2014
https://dl.acm.org/doi/10.1109/SC.2014.71
Show More Cited By

Abstract

Cited By

Index Terms

Recommendations

Malleable iterative MPI applications

Towards Achieving Transparent Malleability Thanks to MPI Process Virtualization

Dynamic spawning of MPI processes applied to malleability

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations