research-article

Extending the OpenSHMEM Memory Model to Support User-Defined Spaces

Authors:

Swaroop Pophale,

Oscar Hernandez,

Barbara ChapmanAuthors Info & Claims

PGAS '14: Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models

Article No.: 11, Pages 1 - 10

https://doi.org/10.1145/2676870.2676884

Published: 06 October 2014 Publication History

Abstract

OpenSHMEM is an open standard for SHMEM libraries. With the standardisation process complete, the community is looking towards extending the API for increasing programmer flexibility and extreme scalability. According to the current OpenSHMEM specification (revision 1.1), allocation of symmetric memory is collective across all PEs executing the application. For better work sharing and memory utilisation, we are proposing the concepts of teams and spaces for OpenSHMEM that together allow allocation of memory only across user-specified teams. Through our implementation we show that by using teams we can confine memory allocation and usage to only the PEs that actually communicate via symmetric memory. We provide our preliminary results that demonstrate creating spaces for teams allows for less consumption of memory resources than the current alternative. We also examine the impact of our extensions on Scalable Synthetic Compact Applications #3 (SSCA3), which is a sensor processing and knowledge formation kernel involving file I/O, and show that up to 30% of symmetric memory allocation can be eliminated without affecting the correctness of the benchmark.

References

[1]

MPI: A message-passing interface standard version 2.2.

[2]

OpenSHMEM specification.

[3]

W. W. Carlson, J. M. Draper, D. E. Culler, K. Yelick, E. Brooks, and K. Warren. Introduction to UPC and language specification. Center for Computing Sciences, Institute for Defense Analyses, 1999.

[4]

B. Chamberlain, D. Callahan, and H. Zima. Parallel programmability and the chapel language. Int. J. High Perform. Comput. Appl., 21(3):291--312, Aug. 2007.

Digital Library

[5]

B. L. Chamberlain, C. Inc, B. L. Chamberlain, and C. Inc. Chapel, 2013.

[6]

P. Charles, C. Grothoff, V. Saraswat, C. Donawa, A. Kielstra, K. Ebcioglu, C. von Praun, and V. Sarkar. X10: An object-oriented approach to non-uniform cluster computing. In Proceedings of the 20th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA '05, pages 519--538, New York, NY, USA, 2005. ACM.

Digital Library

[7]

T. El-Ghazawi, W. Carlson, T. Sterling, and K. Yelick. UPC: distributed shared memory programming, volume 40. John Wiley & Sons, 2005.

Digital Library

[8]

S. Fraunhofer and F. Jülich. GASPI--a partitioned global address space programming interface. Facing the Multicore-Challenge III, page 135, 2013.

[9]

M. Garland, M. Kudlur, and Y. Zheng. Designing a unified programming model for heterogeneous machines. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12, pages 67:1--67:11, Los Alamitos, CA, USA, 2012. IEEE Computer Society Press.

Digital Library

[10]

D. Gay and A. Aiken. Memory management with explicit regions. In Proceedings of the ACM SIGPLAN 1998 Conference on Programming Language Design and Implementation, PLDI '98, pages 313--323, New York, NY, USA, 1998. ACM.

Digital Library

[11]

D. Grünewald and C. Simmendinger. The GASPI API specification and its implementation gpi 2.0. In 7th International Conference on PGAS Programming Models, volume 243, 2013.

[12]

J. R. Hammond, S. Ghosh, and B. M. Chapman. Implementing openshmem using MPI-3 one-sided communication. In OpenSHMEM and Related Technologies. Experiences, Implementations, and Tools - First Workshop, OpenSHMEM 2014, Annapolis, MD, USA, March 4-6, 2014. Proceedings, pages 44--58, 2014.

Digital Library

[13]

S. Lyberis, P. Pratikakis, D. S. Nikolopoulos, M. Schulz, T. Gamblin, and B. R. de Supinski. The myrmics memory allocator: Hierarchical, message-passing allocation for global address spaces. In Proceedings of the 2012 International Symposium on Memory Management, ISMM '12, pages 15--24, New York, NY, USA, 2012. ACM.

Digital Library

[14]

J. Mellor-Crummey, L. Adhianto, W. N. Scherer, III, and G. Jin. A new vision for coarray fortran. In Proceedings of the Third Conference on Partitioned Global Address Space Programing Models, PGAS '09, pages 5:1--5:9, New York, NY, USA, 2009. ACM.

Digital Library

[15]

S. W. Poole, O. Hernandez, and P. Shamis, editors. OpenSHMEM and Related Technologies. Experiences, Implementations, and Tools - First Workshop, OpenSHMEM 2014, Annapolis, MD, USA, March 4-6, 2014. Proceedings, volume 8356 of Lecture Notes in Computer Science. Springer, 2014.

Digital Library

[16]

S. W. Poole, P. Shamis, A. Welch, S. Pophale, M. G. Venkata, O. Hernandez, G. A. Koenig, T. Curtis, and C.-H. Hsu. Openshmem extensions and a vision for its future direction. In OpenSHMEM'14, pages 149--162, 2014.

Digital Library

[17]

S. S. Pophale. Src: Openshmem library development. In Proceedings of the International Conference on Supercomputing, ICS '11, pages 374--374, New York, NY, USA, 2011. ACM.

Digital Library

[18]

V. Saraswat, G. Almasi, G. Bikshandi, C. Cascaval, D. Cunningham, D. Grove, S. Kodali, I. Peshansky, and O. Tardieu. The asynchronous partitioned global address space model. Technical report, Toronto, Canada, June 2010.

[19]

V. A. Saraswat, V. Sarkar, and C. von Praun. X10: concurrent programming for modern architectures. In Proceedings of the 12th ACM SIGPLAN symposium on Principles and practice of parallel programming, pages 271--271. ACM, 2007.

Digital Library

[20]

P. Shamis, M. Venkata, S. Poole, A. Welch, and T. Curtis. Designing a high performance openshmem implementation using universal common communication substrate as a communication middleware. In S. Poole, O. Hernandez, and P. Shamis, editors, OpenSHMEM and Related Technologies. Experiences, Implementations, and Tools, volume 8356 of Lecture Notes in Computer Science, pages 1--13. Springer International Publishing, 2014.

Digital Library

[21]

C. Symeonidou, P. Pratikakis, A. Bilas, and D. S. Nikolopoulos. Drasync: Distributed region-based memory allocation and synchronization. In Proceedings of the 20th European MPI Users' Group Meeting, EuroMPI '13, pages 49--54, New York, NY, USA, 2013. ACM.

Digital Library

[22]

M. ten Bruggencate, D. Roweth, and S. Oyanagi. Thread-safe SHMEM extensions. In OpenSHMEM and Related Technologies. Experiences, Implementations, and Tools - First Workshop, OpenSHMEM 2014, Annapolis, MD, USA, March 4-6, 2014. Proceedings, pages 178--185, 2014.

Digital Library

[23]

M. Tofte, L. Birkedal, M. Elsman, and N. Hallenberg. A retrospective on region-based memory management. Higher Order Symbol. Comput., 17(3):245--265, Sept. 2004.

Digital Library

[24]

M. Tofte and J.-P. Talpin. Region-based memory management. Inf. Comput., 132(2):109--176, Feb. 1997.

Digital Library

[25]

UPC Consortium. UPC language specifications, v1.2. Tech Report LBNL-59208, Lawrence Berkeley National Lab, 2005.

[26]

C. Yang, K. Murthy, and J. Mellor-Crummey. Managing asynchronous operations in coarray fortran 2.0. In Parallel & Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on, pages 1321--1332. IEEE, 2013.

Digital Library

Cited By

Langer AHowell SPotluri SDinan JKraus J(2022)Dynamic Symmetric Heap Allocation in NVSHMEMOpenSHMEM and Related Technologies. OpenSHMEM in the Era of Exascale and Smart Networks10.1007/978-3-031-04888-3_12(187-198)Online publication date: 20-May-2022
https://doi.org/10.1007/978-3-031-04888-3_12
Namashivayam NLong BEachempati DCernohous BPagel M(2020)A Modern Fortran Interface in OpenSHMEM Need for Interoperability with Parallel Fortran Using CoarraysACM Transactions on Parallel Computing10.1145/34180847:4(1-25)Online publication date: 27-Sep-2020
https://dl.acm.org/doi/10.1145/3418084
Ozog DRahman MTaylor GDinan J(2019)Designing, Implementing, and Evaluating the Upcoming OpenSHMEM Teams API2019 IEEE/ACM Parallel Applications Workshop, Alternatives To MPI (PAW-ATM)10.1109/PAW-ATM49560.2019.00009(37-46)Online publication date: Nov-2019
https://doi.org/10.1109/PAW-ATM49560.2019.00009
Show More Cited By

Recommendations

Introducing OpenSHMEM: SHMEM for the PGAS community
PGAS '10: Proceedings of the Fourth Conference on Partitioned Global Address Space Programming Model

The OpenSHMEM community would like to announce a new effort to standardize SHMEM, a communications library that uses one-sided communication and utilizes a partitioned global address space.

OpenSHMEM is an effort to bring together a variety of SHMEM and ...
Asymmetric Memory Extension for Openshmem
PGAS '14: Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models

Memory allocation in Openshmem is a global operation that requires in-step calls from all Processing Elements (PEs). Although this approach works with applications that split the work evenly, it prevents using Openshmem in cases where the workload and ...
A bounded memory allocator for software-defined global address spaces
ISMM '16

This paper presents a memory allocator targeting manycore architec- tures with distributed memory. Among the family of Multi Processor System on Chip (MPSoC), these devices are composed of multiple nodes linked by an on-chip network; most nodes have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PGAS '14: Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models

October 2014

199 pages

ISBN:9781450332477

DOI:10.1145/2676870

Conference Chair:
Allen D. Malony,
Program Chair:
Jeff Hammond

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing
University of Oregon: University of Oregon

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 October 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

PGAS '14

PGAS '14: 8th International Conference on Partitioned Global Address Space Programming Models

October 6 - 10, 2014

OR, Eugene, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
102
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Langer AHowell SPotluri SDinan JKraus J(2022)Dynamic Symmetric Heap Allocation in NVSHMEMOpenSHMEM and Related Technologies. OpenSHMEM in the Era of Exascale and Smart Networks10.1007/978-3-031-04888-3_12(187-198)Online publication date: 20-May-2022
https://doi.org/10.1007/978-3-031-04888-3_12
Namashivayam NLong BEachempati DCernohous BPagel M(2020)A Modern Fortran Interface in OpenSHMEM Need for Interoperability with Parallel Fortran Using CoarraysACM Transactions on Parallel Computing10.1145/34180847:4(1-25)Online publication date: 27-Sep-2020
https://dl.acm.org/doi/10.1145/3418084
Ozog DRahman MTaylor GDinan J(2019)Designing, Implementing, and Evaluating the Upcoming OpenSHMEM Teams API2019 IEEE/ACM Parallel Applications Workshop, Alternatives To MPI (PAW-ATM)10.1109/PAW-ATM49560.2019.00009(37-46)Online publication date: Nov-2019
https://doi.org/10.1109/PAW-ATM49560.2019.00009
Aderholdt FPophale SGorentla Venkata MImam NKindler HWeber FKühne OHalder G(2019)OpenSHMEM Sets and Groups: An Approach to Worksharing and Memory ManagementWissenschaftlich Arbeiten in Geographie und Raumwissenschaften10.1007/978-3-030-04918-8_1(3-21)Online publication date: 19-Mar-2019
https://doi.org/10.1007/978-3-030-04918-8_1
Namashivayam NCernohous BKandalla KPou DRobichaux JDinan JPagel M(2018)Symmetric Memory Partitions in OpenSHMEM: A Case Study with Intel KNLOpenSHMEM and Related Technologies. Big Compute and Big Data Convergence10.1007/978-3-319-73814-7_1(3-18)Online publication date: 10-Jan-2018
https://doi.org/10.1007/978-3-319-73814-7_1
Qian XSen KHargrove PIancu C(2016)SReplayProceedings of the 2016 International Conference on Supercomputing10.1145/2925426.2926264(1-13)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1145/2925426.2926264
Welch AVenkata M(2016)On Synchronisation and Memory Reuse in OpenSHMEMOpenSHMEM and Related Technologies. Enhancing OpenSHMEM for Hybrid Environments10.1007/978-3-319-50995-2_6(82-94)Online publication date: 15-Dec-2016
https://doi.org/10.1007/978-3-319-50995-2_6
Li MHamidouche KLu XZhang JLin JPanda D(2015)High Performance OpenSHMEM Strided Communication Support with InfiniBand UMRProceedings of the 2015 IEEE 22nd International Conference on High Performance Computing (HiPC)10.1109/HiPC.2015.48(244-253)Online publication date: 16-Dec-2015
https://dl.acm.org/doi/10.1109/HiPC.2015.48
Khaldi DEachempati DGe SJouvelot PChapman B(2015)A Team-Based Methodology of Memory Hierarchy-Aware Runtime Support in Coarray FortranProceedings of the 2015 IEEE International Conference on Cluster Computing10.1109/CLUSTER.2015.67(448-451)Online publication date: 8-Sep-2015
https://dl.acm.org/doi/10.1109/CLUSTER.2015.67

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents