research-article

Mixed-size concurrency: ARM, POWER, C/C++11, and SC

Authors:

Christopher Pulte,

Kyndylan Nienhuis,

Kathryn E. Gray,

Peter SewellAuthors Info & Claims

POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages

Pages 429 - 442

https://doi.org/10.1145/3009837.3009839

Published: 01 January 2017 Publication History

Abstract

Previous work on the semantics of relaxed shared-memory concurrency has only considered the case in which each load reads the data of exactly one store. In practice, however, multiprocessors support mixed-size accesses, and these are used by systems software and (to some degree) exposed at the C/C++ language level. A semantic foundation for software, therefore, has to address them.

We investigate the mixed-size behaviour of ARMv8 and IBM POWER architectures and implementations: by experiment, by developing semantic models, by testing the correspondence between these, and by discussion with ARM and IBM staff. This turns out to be surprisingly subtle, and on the way we have to revisit the fundamental concepts of coherence and sequential consistency, which change in this setting. In particular, we show that adding a memory barrier between each instruction does not restore sequential consistency. We go on to extend the C/C++11 model to support non-atomic mixed-size memory accesses.

This is a necessary step towards semantics for real-world shared-memory concurrent code, beyond litmus tests.

References

[1]

L. Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Trans. Comput., C-28(9):690– 691, 1979.

Digital Library

[2]

L. M. Censier and P. Feautrier. A new solution to coherence problems in multicache systems. IEEE Trans. Comput., 27(12):1112–1118, December 1978.

Digital Library

[3]

William W. Collier. Principles of architecture for systems of parallel processes. Technical Report TR 00.3100, IBM Poughkeepsie, 1981.

[4]

Michel Dubois, Christoph Scheurich, and Faye A. Briggs. Memory access buffering in multiprocessors. In Proc. ISCA ’86, pages 434– 442, 1986.

Digital Library

[5]

J. Misra. Axioms for memory access in asynchronous hardware systems. ACM Trans. Program. Lang. Syst., 8(1):142–153, 1986.

Digital Library

[6]

Dennis Shasha and Marc Snir. Efficient and correct execution of parallel programs that share memory. ACM Trans. Program. Lang. Syst., 10(2):282–312, April 1988.

Digital Library

[7]

James R. Goodman. Cache consistency and sequential consistency. Technical Report Technical Report 61, IEEE Scalable Coherent Interface (SCI) Working Group, March 1989.

[8]

Sarita V. Adve and Mark D. Hill. Weak ordering — a new definition. In Proc. ISCA ’90, pages 2–14. ACM, 1990.

Digital Library

[9]

Kourosh Gharachorloo, Daniel Lenoski, James Laudon, Phillip Gibbons, Anoop Gupta, and John Hennessy. Memory consistency and event ordering in scalable shared-memory multiprocessors. In Proc. ISCA ’90, pages 15–26. ACM, 1990.

Digital Library

[10]

William W. Collier. Reasoning About Parallel Architectures. Prentice-Hall, Inc., 1992.

Digital Library

[11]

Pradeep S. Sindhu, Jean-Marc Frailong, and Michel Cekleov. Formal Specification of Memory Models, pages 25–41. Springer US, 1992.

[12]

Prince Kohli, Gil Neiger, and Mustaque Ahamad. A characterization of scalable shared memories. In ICPP: International Conference on Parallel Processing, pages 332–335, 1993.

Digital Library

[13]

F. Corella, J. M. Stone, and C. M. Barton. A formal specification of the PowerPC shared memory architecture. Technical Report RC18638, IBM, 1993.

[14]

David L Dill, Seungjoon Park, and Andreas G. Nowatzyk. Formal specification of abstract memory models. In Proceedings of the 1993 Symposium on Research on Integrated Systems, pages 38–52. MIT Press, 1993.

Digital Library

[15]

The SPARC Architecture Manual, Version 9. SPARC Int., Inc., 1994.

Digital Library

[16]

Hagit Attiya and Roy Friedman. Programming DEC-Alpha based multiprocessors the easy way (extended abstract). In Proc. SPAA, pages 157–166, New York, NY, USA, 1994. ACM.

Digital Library

[17]

José M. Bernabéu-Aubán and Vicente Cholvi-juan. Formalizing memory coherency models. Journal of Computing and Information, 1:653–672, 1994.

[18]

K. Gharachorloo. Memory consistency models for shared-memory multiprocessors. WRL Research Report, 95(9), 1995.

Digital Library

[19]

Mustaque Ahamad, Gil Neiger, James E. Burns, Prince Kohli, and Phillip W. Hutto. Causal memory: definitions, implementation, and programming. Distributed Computing, 9(1):37–49, 1995.

Digital Library

[20]

Lisa Higham, Jalal Kawash, and Nathaly Verwaal. Weak memory consistency models. Part I: Definitions and comparisons. Technical report, Department of Computer Science, University of Calgary, 1998.

[21]

Prosenjit Chatterjee and Ganesh Gopalakrishnan. Towards a formal model of shared memory consistency for Intel Itaniumtm. In 19th International Conference on Computer Design (ICCD 2001), September 2001, Austin, TX, USA, pages 515–518, 2001.

Digital Library

[22]

Intel. A formal specification of Intel Itanium processor family memory ordering, 2002. http://download.intel.com/design/ Itanium/Downloads/25142901.pdf.

[23]

A. Adir, H. Attiya, and G. Shurek. Information-flow models for shared memory with an application to the PowerPC architecture. IEEE Trans. Parallel Distrib. Syst., 14(5):502–515, 2003.

Digital Library

[24]

Yue Yang, Ganesh Gopalakrishnan, Gary Lindstrom, and Konrad Slind. Nemos: A framework for axiomatic and executable specifications of memory consistency models. In 18th International Parallel and Distributed Processing Symposium (IPDPS), Santa Fe, New Mexico, USA, 2004.

[25]

Lisa Higham, LillAnne Jackson, and Jalal Kawash. Programmercentric conditions for Itanium memory consistency. In Proceedings of the 8th International Conference on Distributed Computing and Networking, ICDCN’06, pages 58–69. Springer-Verlag, 2006.

Digital Library

[26]

Arvind Arvind and Jan-Willem Maessen. Memory model = instruction reordering + store atomicity. In Proc. ISCA ’06, pages 29–40. IEEE Computer Society, 2006.

Digital Library

[27]

N. Chong and S. Ishtiaq. Reasoning about the ARM weakly consistent memory model. In MSPC, 2008.

Digital Library

[28]

Susmit Sarkar, Peter Sewell, Francesco Zappa Nardelli, Scott Owens, Tom Ridge, Thomas Braibant, Magnus Myreen, and Jade Alglave. The semantics of x86-CC multiprocessor machine code. In Proc. POPL 2009, pages 379–391, January 2009.

Digital Library

[29]

J. Alglave, A. Fox, S. Ishtiaq, M. O. Myreen, S. Sarkar, P. Sewell, and F. Zappa Nardelli. The semantics of Power and ARM multiprocessor machine code. In Proc. DAMP 2009, January 2009.

Digital Library

[30]

J. Alglave, L. Maranget, S. Sarkar, and P. Sewell. Fences in weak memory models. In Proc. CAV, 2010.

Digital Library

[31]

Scott Owens, Susmit Sarkar, and Peter Sewell. A better x86 memory model: x86-TSO. In Proceedings of TPHOLs 2009: Theorem Proving in Higher Order Logics, LNCS 5674, pages 391–407, 2009.

Digital Library

[32]

Peter Sewell, Susmit Sarkar, Scott Owens, Francesco Zappa Nardelli, and Magnus O. Myreen. x86-TSO: A rigorous and usable programmer’s model for x86 multiprocessors. Communications of the ACM, 53(7):89–97, July 2010. (Research Highlights).

Digital Library

[33]

Susmit Sarkar, Peter Sewell, Jade Alglave, Luc Maranget, and Derek Williams. Understanding POWER multiprocessors. In Proc. PLDI ’11, pages 175–186, 2011.

Digital Library

[34]

Mark Batty, Kayvan Memarian, Scott Owens, Susmit Sarkar, and Peter Sewell. Clarifying and Compiling C/C++ Concurrency: from C++11 to POWER. In Proc. POPL 2012, pages 509–520, 2012.

Digital Library

[35]

Susmit Sarkar, Kayvan Memarian, Scott Owens, Mark Batty, Peter Sewell, Luc Maranget, Jade Alglave, and Derek Williams. Synchronising C/C++ and POWER. In Proceedings of PLDI 2012, the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation (Beijing), pages 311–322, 2012.

Digital Library

[36]

Luc Maranget, Susmit Sarkar, and Peter Sewell. A tutorial introduction to the ARM and POWER relaxed memory models. Draft available from http://www.cl.cam.ac.uk/~pes20/ ppc-supplemental/test7.pdf, 2012.

[37]

Jade Alglave, Luc Maranget, and Michael Tautschnig. Herding Cats: Modelling, Simulation, Testing, and Data Mining for Weak Memory. ACM TOPLAS, 36(2):7:1–7:74, July 2014.

Digital Library

[38]

Kathryn E. Gray, Gabriel Kerneis, Dominic Mulligan, Christopher Pulte, Susmit Sarkar, and Peter Sewell. An integrated concurrency and core-ISA architectural envelope definition, and test oracle, for IBM POWER multiprocessors. In Proc. MICRO-48, the 48th Annual IEEE/ACM International Symposium on Microarchitecture, December 2015.

Digital Library

[39]

Shaked Flur, Kathryn E. Gray, Christopher Pulte, Susmit Sarkar, Ali Sezgin, Luc Maranget, Will Deacon, and Peter Sewell. Modelling the ARMv8 architecture, operationally: Concurrency and ISA. In Proceedings of POPL: the 43rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, 2016.

Digital Library

[40]

Sizhuo Zhang, Arvind, and Muralidaran Vijayaraghavan. Taming weak memory models. CoRR, abs/1606.05416, 2016.

[41]

Linux kernel lockrefs. https://lwn.net/Articles/565734/, http://git.kernel.org/cgit/linux/kernel/git/torvalds/ linux.git/tree/lib/lockref.c, http://git.kernel.org/cgit/ linux/kernel/git/torvalds/linux.git/tree/include/linux/ lockref.h.

[42]

ARM Ltd. ARM Architecture Reference Manual (ARMv8, for ARMv8-A architecture profile), 2015. ARM DDI 0487A.h (ID092915).

[43]

Power ISATM Version 2.07. IBM, 2013.

[44]

Jade Alglave, Luc Maranget, Susmit Sarkar, and Peter Sewell. Litmus: running tests against hardware. In Proceedings of TACAS 2011, pages 41–44. Springer-Verlag, 2011.

Digital Library

[45]

H.-J. Boehm and S. Adve. Foundations of the C++ concurrency memory model. In Proc. PLDI, 2008.

Digital Library

[46]

M. Batty, S. Owens, S. Sarkar, P. Sewell, and T. Weber. Mathematizing C++ concurrency. In Proc. POPL, 2011.

Digital Library

[47]

Yatin A. Manerkar, Caroline Trippel, Daniel Lustig, Michael Pellauer, and Margaret Martonosi. Counterexamples and proof loophole for the C/C++ to POWER and ARMv7 trailing-sync compiler mappings. CoRR, abs/1611.01507, 2016.

[48]

Ori Lahav, Viktor Vafeiadis, Jeehoon Kang, Chung-Kil Hur, and Derek Dreyer. Repairing sequential consistency in C/C++11. Note, available at http://plv.mpi-sws.org/scfix/, 2016.

[49]

Susmit Sarkar and Peter Sewell. Corrigendum: C/C++11 to POWER concurrency compilation scheme correctness proof. Note, available at http://www.cl.cam.ac.uk/users/pes20/cppppc/corrigendum. html, December 2016.

[50]

P. Cenciarelli, A. Knapp, and E. Sibilio. The Java memory model: Operationally, denotationally, axiomatically. In ESOP, 2007.

Digital Library

[51]

J. Ševˇcík and D. Aspinall. On validity of program transformations in the Java memory model. In ECOOP, 2008.

Digital Library

[52]

Mark Batty, Kayvan Memarian, Kyndylan Nienhuis, Jean Pichon-Pharabod, and Peter Sewell. The problem of programming language concurrency semantics. In Proceedings of ESOP 2015, 2015.

[53]

Jean Pichon-Pharabod and Peter Sewell. A concurrency semantics for relaxed atomics that permits optimisation and avoids thin-air executions. In Proceedings of POPL, 2016.

Digital Library

[54]

Shaked Flur, Susmit Sarkar, Christopher Pulte, Kyndylan Nienhuis, Luc Maranget, Kathryn E. Gray, Ali Sezgin, Mark Batty, and Peter Sewell. Supplementary material. http://www.cl.cam.ac.uk/ ~pes20/popl17/,

[55]

Dominic P. Mulligan, Scott Owens, Kathryn E. Gray, Tom Ridge, and Peter Sewell. Lem: reusable engineering of real-world semantics. In Proceedings of ICFP 2014: the 19th ACM SIGPLAN International Conference on Functional Programming, pages 175–188, 2014.

Digital Library

[56]

P. Becker, editor. Programming Languages — C++. 2011. ISO/IEC 14882:2011. http://www.open-std.org/jtc1/sc22/wg21/docs/ papers/2011/n3242.pdf.

[57]

Mark John Batty. The C11 and C++11 Concurrency Model. PhD thesis, University of Cambridge Computer Laboratory, 2014.

[58]

P. E. McKenney and R. Silvera. Example POWER implementation for C/C++ memory model. http://www.rdrop.com/users/paulmck/ scalability/paper/N2745r.2011.03.04a.html, 2011.

[59]

Jade Alglave and Luc Maranget. The diy tool. http://diy.inria. fr/.

[60]

Mark Batty, Mike Dodds, and Alexey Gotsman. Library abstraction for C/C++ concurrency. In Proc. POPL ’13, pages 235–248. ACM, 2013.

Digital Library

[61]

Aaron Turon, Viktor Vafeiadis, and Derek Dreyer. GPS: Navigating weak memory with ghosts, protocols, and separation. In Proc. OOPSLA ’14, 2014.

Digital Library

[62]

Richard Bornat, Jade Alglave, and Matthew J. Parkinson. New lace and arsenic: adventures in weak memory with a program logic. CoRR, abs/1512.01416, 2015.

Cited By

Marmanis IKokologiannakis MVafeiadis V(2025)Model Checking C/C++ with Mixed-Size AccessesProceedings of the ACM on Programming Languages10.1145/37049119:POPL(2232-2252)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3704911
Hammond ALiu ZPérami TSewell PBirkedal LPichon-Pharabod J(2024)An Axiomatic Basis for Computer Programming on the Relaxed Arm-A Architecture: The AxSL LogicProceedings of the ACM on Programming Languages10.1145/36328638:POPL(604-637)Online publication date: 5-Jan-2024
https://dl.acm.org/doi/10.1145/3632863
Wrenger LTöllner DLohmann D(2024)Analyzing the memory ordering models of the Apple M1Journal of Systems Architecture: the EUROMICRO Journal10.1016/j.sysarc.2024.103102149:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.sysarc.2024.103102
Show More Cited By

Index Terms

Mixed-size concurrency: ARM, POWER, C/C++11, and SC

Recommendations

Modelling the ARMv8 architecture, operationally: concurrency and ISA
POPL '16: Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages

In this paper we develop semantics for key aspects of the ARMv8 multiprocessor architecture: the concurrency model and much of the 64-bit application-level instruction set (ISA). Our goal is to clarify what the range of architecturally allowable ...
Mixed-size concurrency: ARM, POWER, C/C++11, and SC
POPL '17

Previous work on the semantics of relaxed shared-memory concurrency has only considered the case in which each load reads the data of exactly one store. In practice, however, multiprocessors support mixed-size accesses, and these are used by systems ...
Synchronising C/C++ and POWER
PLDI '12

Shared memory concurrency relies on synchronisation primitives: compare-and-swap, load-reserve/store-conditional (aka LL/SC), language-level mutexes, and so on. In a sequentially consistent setting, or even in the TSO setting of x86 and Sparc, these ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages

January 2017

901 pages

ISBN:9781450346603

DOI:10.1145/3009837

General Chair:
Giuseppe Castagna
Paris Diderot University, France / CNRS, France
,
Program Chair:
Andrew D. Gordon
Microsoft Research, UK / University of Edinburgh, UK

ACM SIGPLAN Notices Volume 52, Issue 1
POPL '17
January 2017
901 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3093333
Editor:
Matthew Fluet
Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 January 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

POPL '17

Sponsor:

SIGPLAN

POPL '17: The 44th Annual ACM SIGPLAN Symposium on Principles of Programming Languages

January 15 - 21, 2017

Paris, France

Acceptance Rates

Overall Acceptance Rate 824 of 4,130 submissions, 20%

Upcoming Conference

POPL '25

Sponsor:
sigplan

The 52nd Annual ACM SIGPLAN Symposium on Principles of Programming Languages

January 19 - 25, 2025

Denver , CO , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
357
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)8

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Marmanis IKokologiannakis MVafeiadis V(2025)Model Checking C/C++ with Mixed-Size AccessesProceedings of the ACM on Programming Languages10.1145/37049119:POPL(2232-2252)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3704911
Hammond ALiu ZPérami TSewell PBirkedal LPichon-Pharabod J(2024)An Axiomatic Basis for Computer Programming on the Relaxed Arm-A Architecture: The AxSL LogicProceedings of the ACM on Programming Languages10.1145/36328638:POPL(604-637)Online publication date: 5-Jan-2024
https://dl.acm.org/doi/10.1145/3632863
Wrenger LTöllner DLohmann D(2024)Analyzing the memory ordering models of the Apple M1Journal of Systems Architecture: the EUROMICRO Journal10.1016/j.sysarc.2024.103102149:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.sysarc.2024.103102
Sato SMizuhashi TKimura GTaura K(2024)Efficiently Adapting Stateless Model Checking for C11/C++11 to Mixed-Size AccessesProgramming Languages and Systems10.1007/978-981-97-8943-6_17(346-364)Online publication date: 23-Oct-2024
https://dl.acm.org/doi/10.1007/978-981-97-8943-6_17
Goens AChakraborty SSarkar SAgarwal SOswald NNagarajan V(2023)Compound Memory ModelsProceedings of the ACM on Programming Languages10.1145/35912677:PLDI(1145-1168)Online publication date: 6-Jun-2023
https://doi.org/10.1145/3591267
Norman CGodbole AManerkar YAamodt TJerger NSwift M(2023)PipeSynth: Automated Synthesis of Microarchitectural Axioms for Memory ConsistencyProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582056(513-527)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582056
Kokologiannakis MLahav OVafeiadis V(2023)Kater: Automating Weak Memory Model Metatheory and Consistency CheckingProceedings of the ACM on Programming Languages10.1145/35712127:POPLOnline publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3571212
Wrenger LTöllner DLohmann D(2023)TOSTING: Investigating Total Store Ordering on ARMArchitecture of Computing Systems10.1007/978-3-031-42785-5_10(139-152)Online publication date: 26-Aug-2023
https://doi.org/10.1007/978-3-031-42785-5_10
Lustig DCooksey SGiroux OSalapura VZahran MChong FTang L(2022)Mixed-proxy extensions for the NVIDIA PTX memory consistency modelProceedings of the 49th Annual International Symposium on Computer Architecture10.1145/3470496.3533045(1058-1070)Online publication date: 18-Jun-2022
https://dl.acm.org/doi/10.1145/3470496.3533045
Nagarajan VSorin DHill MWood DNagarajan VSorin DHill MWood D(2022)Relaxed Memory ConsistencyA Primer on Memory Consistency and Cache Coherence10.1007/978-3-031-01764-3_5(55-89)Online publication date: 28-Mar-2022
https://doi.org/10.1007/978-3-031-01764-3_5
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents