A Family of Fast and Memory Efficient Lock- and Wait-Free Reclamation
Article No.: 235, Pages 2174 - 2198
Abstract
Historically, memory management based on lock-free reference counting was very inefficient, especially for read-dominated workloads. Thus, approaches such as epoch-based reclamation (EBR), hazard pointers (HP), or a combination thereof have received significant attention. EBR exhibits excellent performance but is blocking due to potentially unbounded memory usage. In contrast, HP are non-blocking and achieve good memory efficiency but are much slower. Moreover, HP are only lock-free in the general case. Recently, several new memory reclamation approaches such as WFE and Hyaline have been proposed. WFE achieves wait-freedom, but is less memory efficient and performs suboptimally in oversubscribed scenarios; Hyaline achieves higher performance and memory efficiency, but lacks wait-freedom.
We present a family of non-blocking memory reclamation schemes, called Crystalline, that simultaneously addresses the challenges of high performance, high memory efficiency, and wait-freedom. Crystalline can guarantee complete wait-freedom even when threads are dynamically recycled, asynchronously reclaims memory in the sense that any thread can reclaim memory retired by any other thread, and ensures (an almost) balanced reclamation workload across all threads. The latter two properties result in Crystalline's high performance and memory efficiency. Simultaneously ensuring all three properties requires overcoming unique challenges. Crystalline supports ubiquitous x86-64 and ARM64 architectures, while achieving superior throughput than prior fast schemes such as EBR as the number of threads grows.
We also accentuate that many recent approaches, unlike HP, lack strict non-blocking guarantees when used with multiple data structures. By providing full wait-freedom, Crystalline addresses this problem as well.
References
[1]
2020. Private communication with PEBR’s authors.
[2]
2024. Crossbeam: epoch-based memory reclamation. https://docs.rs/crossbeam/latest/crossbeam/epoch/index.html
[3]
Dan Alistarh, William Leiserson, Alexander Matveev, and Nir Shavit. 2017. Forkscan: Conservative Memory Reclamation for Modern Operating Systems. In Proceedings of the 12th European Conference on Computer Systems (EuroSys ’17). ACM, New York, NY, USA. 483–498. isbn:978-1-4503-4938-3 https://doi.org/10.1145/3064176.3064214
[4]
Dan Alistarh, William M. Leiserson, Alexander Matveev, and Nir Shavit. 2015. ThreadScan: Automatic and Scalable Memory Reclamation. In Proceedings of the 27th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’15). ACM, New York, NY, USA. 123–132. isbn:978-1-4503-3588-1 https://doi.org/10.1145/2755573.2755600
[5]
Daniel Anderson, Guy E. Blelloch, and Yuanhao Wei. 2021. Concurrent Deferred Reference Counting with Constant-Time Overhead. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI ’21). ACM, New York, NY, USA. 526–541. isbn:9781450383912 https://doi.org/10.1145/3453483.3454060
[6]
Daniel Anderson, Guy E. Blelloch, and Yuanhao Wei. 2022. Turning Manual Concurrent Memory Reclamation into Automatic Reference Counting. In Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI ’22). 61–75. isbn:9781450392655 https://doi.org/10.1145/3519939.3523730
[7]
Oana Balmau, Rachid Guerraoui, Maurice Herlihy, and Igor Zablotchi. 2016. Fast and Robust Memory Reclamation for Concurrent Data Structures. In Proceedings of the 28th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’16). ACM, 349–359. isbn:978-1-4503-4210-0 https://doi.org/10.1145/2935764.2935790
[8]
Anastasia Braginsky, Alex Kogan, and Erez Petrank. 2013. Drop the Anchor: Lightweight Memory Management for Non-blocking Data Structures. In Proceedings of the 25th Annual ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’13). ACM, 33–42. isbn:978-1-4503-1572-2 https://doi.org/10.1145/2486159.2486184
[9]
Trevor Alexander Brown. 2015. Reclaiming Memory for Lock-Free Data Structures: There Has to Be a Better Way. In Proceedings of the 2015 ACM Symposium on Principles of Distributed Computing (PODC ’15). ACM, New York, NY, USA. 261–270. isbn:978-1-4503-3617-8 https://doi.org/10.1145/2767386.2767436
[10]
Nachshon Cohen. 2018. Every Data Structure Deserves Lock-free Memory Reclamation. Proc. ACM Program. Lang., 2, OOPSLA (2018), Article 143, Oct., 24 pages. issn:2475-1421 https://doi.org/10.1145/3276513
[11]
Nachshon Cohen and Erez Petrank. 2015. Automatic Memory Reclamation for Lock-free Data Structures. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2015). ACM, 260–279. isbn:978-1-4503-3689-5 https://doi.org/10.1145/2814270.2814298
[12]
Nachshon Cohen and Erez Petrank. 2015. Efficient Memory Management for Lock-Free Data Structures with Optimistic Access. In Proceedings of the 27th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’15). ACM, New York, NY, USA. 254–263. isbn:978-1-4503-3588-1 https://doi.org/10.1145/2755573.2755579
[13]
Andreia Correia, Pedro Ramalhete, and Pascal Felber. 2020. A Wait-Free Universal Construction for Large Objects. In Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’20). ACM, New York, NY, USA. 102–116. isbn:9781450368186 https://doi.org/10.1145/3332466.3374523
[14]
Andreia Correia, Pedro Ramalhete, and Pascal Felber. 2021. OrcGC: Automatic Lock-Free Memory Reclamation. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’21). ACM, 205–218. isbn:9781450382946 https://doi.org/10.1145/3437801.3441596
[15]
Christina Delimitrou and Christos Kozyrakis. 2018. Amdahl’s Law for Tail Latency. Commun. ACM, 61, 8 (2018), jul, 65–72. issn:0001-0782 https://doi.org/10.1145/3232559
[16]
David L. Detlefs, Paul A. Martin, Mark Moir, and Guy L. Steele Jr. 2002. Lock-free reference counting. Distributed Computing, 15, 4 (2002), 01 Dec, 255–271. issn:1432-0452 https://doi.org/10.1007/s00446-002-0079-z
[17]
Jason Evans. 2006. A scalable concurrent malloc (3) implementation for FreeBSD. In Proceedings of the BSDCan Conference, Ottawa, Canada. https://www.bsdcan.org/2006/papers/jemalloc.pdf
[18]
Panagiota Fatourou and Nikolaos D. Kallimanis. 2011. A Highly-Efficient Wait-Free Universal Construction. In Proceedings of the Twenty-Third Annual ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’11). ACM, New York, NY, USA. 325–334. isbn:9781450307437 https://doi.org/10.1145/1989493.1989549
[19]
Keir Fraser. 2004. Practical lock-freedom. Univ. of Cambridge, Computer Laboratory. https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-579.pdf
[20]
A. Gidenstam, M. Papatriantafilou, H. Sundell, and P. Tsigas. 2009. Efficient and Reliable Lock-Free Memory Reclamation Based on Reference Counting. IEEE Transactions on Parallel and Distributed Systems, 20, 8 (2009), Aug, 1173–1187. issn:1045-9219 https://doi.org/10.1109/TPDS.2008.167
[21]
Timothy L. Harris. 2001. A Pragmatic Implementation of Non-blocking Linked-lists. In Proceedings of the 15th International Conference on Distributed Computing (DISC ’01). Springer Berlin Heidelberg, Berlin, Heidelberg. 300–314. isbn:978-3-540-45414-4 https://doi.org/10.1007/3-540-45414-4_21
[22]
Thomas E. Hart, Paul E. McKenney, Angela Demke Brown, and Jonathan Walpole. 2007. Performance of memory reclamation for lockless synchronization. J. Parallel and Distrib. Comput., 67, 12 (2007), 1270 – 1285. issn:0743-7315 https://doi.org/10.1016/j.jpdc.2007.04.010
[23]
Maurice Herlihy, Victor Luchangco, Paul Martin, and Mark Moir. 2005. Nonblocking Memory Management Support for Dynamic-sized Data Structures. ACM Trans. Comput. Syst., 23, 2 (2005), May, 146–196. issn:0734-2071 https://doi.org/10.1145/1062247.1062249
[24]
Maurice Herlihy, Victor Luchangco, and Mark Moir. 2002. The Repeat Offender Problem: A Mechanism for Supporting Dynamic-Sized, Lock-Free Data Structures. In Proceedings of the 16th International Conference on Distributed Computing (DISC ’02). Springer-Verlag, Berlin, Heidelberg. 339–353. isbn:3-540-00073-9 https://doi.org/10.1007/3-540-36108-1_23
[25]
Maurice Herlihy and Nir Shavit. 2008. The Art of Multiprocessor Programming. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. isbn:0123705916, 9780123705914
[26]
Jeehoon Kang and Jaehwang Jung. 2020. A Marriage of Pointer- and Epoch-Based Reclamation. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’20). ACM, New York, NY, USA. 314–328. isbn:9781450376136 https://doi.org/10.1145/3385412.3385978
[27]
Alex Kogan and Erez Petrank. 2011. Wait-free Queues with Multiple Enqueuers and Dequeuers. In Proceedings of the 16th ACM Symposium on Principles and Practice of Parallel Programming (PPoPP ’11). ACM, New York, NY, USA. 223–234. isbn:978-1-4503-0119-0 https://doi.org/10.1145/1941553.1941585
[28]
Daan Leijen, Benjamin G. Zorn, and Leonardo Mendonça de Moura. 2019. Mimalloc: Free List Sharding in Action. In Asian Symposium on Programming Languages and Systems (APLAS ’19). https://doi.org/10.1007/978-3-030-34175-6_13
[29]
Maged M. Michael. 2004. Hazard pointers: safe memory reclamation for lock-free objects. IEEE Transactions on Parallel and Distributed Systems, 15, 6 (2004), June, 491–504. issn:1045-9219 https://doi.org/10.1109/TPDS.2004.8
[30]
Maged M. Michael and Michael L. Scott. 1995. Correction of a Memory Management Method for Lock-Free Data Structures. University of Rochester, CS. https://www.cs.rochester.edu/u/scott/papers/1995_TR599.pdf
[31]
Pedro Moreno and Ricardo Rocha. 2023. Releasing Memory with Optimistic Access: A Hybrid Approach to Memory Reclamation and Allocation in Lock-Free Programs. In Proceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’23). ACM, 177–186. isbn:9781450395458 https://doi.org/10.1145/3558481.3591089
[32]
Adam Morrison and Yehuda Afek. 2013. Fast Concurrent Queues for x86 Processors. In Proceedings of the 18th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’13). ACM, New York, NY, USA. 103–112. isbn:978-1-4503-1922-5 https://doi.org/10.1145/2442516.2442527
[33]
Ruslan Nikolaev. 2019. A Scalable, Portable, and Memory-Efficient Lock-Free FIFO Queue. In Proceedings of the 33rd International Symposium on Distributed Computing (DISC 2019) (Leibniz International Proceedings in Informatics (LIPIcs), Vol. 146). Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 28:1–28:16. isbn:978-3-95977-126-9 issn:1868-8969 https://doi.org/10.4230/LIPIcs.DISC.2019.28
[34]
Ruslan Nikolaev and Binoy Ravindran. 2019. Brief Announcement: Hyaline: Fast and Transparent Lock-Free Memory Reclamation. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing (PODC ’19). ACM, New York, NY, USA. 419–421. isbn:9781450362177 https://doi.org/10.1145/3293611.3331575
[35]
Ruslan Nikolaev and Binoy Ravindran. 2020. Universal Wait-Free Memory Reclamation. In Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’20). ACM, New York, NY, USA. 130–143. isbn:9781450368186 https://doi.org/10.1145/3332466.3374540
[36]
Ruslan Nikolaev and Binoy Ravindran. 2021. Brief Announcement: Crystalline: Fast and Memory Efficient Wait-Free Reclamation. In Proceedings of the 35th International Symposium on Distributed Computing (DISC 2021) (Leibniz International Proceedings in Informatics (LIPIcs), Vol. 209). Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl, Germany. 60:1–60:4. isbn:978-3-95977-210-5 issn:1868-8969 https://doi.org/10.4230/LIPIcs.DISC.2021.60
[37]
Ruslan Nikolaev and Binoy Ravindran. 2021. Snapshot-Free, Transparent, and Robust Memory Reclamation for Lock-Free Data Structures. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI ’21). ACM, New York, NY, USA. 987–1002. isbn:9781450383912 https://doi.org/10.1145/3453483.3454090
[38]
Ruslan Nikolaev and Binoy Ravindran. 2022. wCQ: A Fast Wait-Free Queue with Bounded Memory Usage. In Proceedings of the 34th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’22). ACM, New York, NY, USA. 307–319. isbn:9781450391467 https://doi.org/10.1145/3490148.3538572
[39]
Ruslan Nikolaev and Binoy Ravindran. 2024. A Family of Fast and Memory Efficient Lock- and Wait-Free Reclamation (an extended arXiv version of this paper). arxiv:2108.02763
[40]
Ruslan Nikolaev and Binoy Ravindran. 2024. Artifact for PLDI’24. https://doi.org/10.5281/zenodo.10775789
[41]
Manuel Pöter and Jesper Larsson Träff. 2018. Brief Announcement: Stamp-it, a More Thread-efficient, Concurrent Memory Reclamation Scheme in the C++ Memory Model. In Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures (SPAA ’18). ACM, 355–358. isbn:978-1-4503-5799-9 https://doi.org/10.1145/3210377.3210661
[42]
Pedro Ramalhete and Andreia Correia. 2016. A Wait-Free Queue with Wait-Free Memory Reclamation (Full Version). https://github.com/pramalhe/ConcurrencyFreaks/raw/master/papers/crturnqueue-2016.pdf
[43]
Pedro Ramalhete and Andreia Correia. 2017. Brief Announcement: Hazard Eras - Non-Blocking Memory Reclamation. In Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’17). ACM, New York, NY, USA. 367–369. isbn:978-1-4503-4593-4 https://doi.org/10.1145/3087556.3087588
[44]
Pedro Ramalhete and Andreia Correia. 2017. Hazard Eras - Non-Blocking Memory Reclamation (Full Version). https://github.com/pramalhe/ConcurrencyFreaks/raw/master/papers/hazarderas-2017.pdf
[45]
Pedro Ramalhete and Andreia Correia. 2017. POSTER: A Wait-Free Queue with Wait-Free Memory Reclamation. In Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’17). ACM, New York, NY, USA. 453–454. isbn:978-1-4503-4493-7 https://doi.org/10.1145/3018743.3019022
[46]
Pedro Ramalhete, Andreia Correia, Pascal Felber, and Nachshon Cohen. 2019. OneFile: A Wait-Free Persistent Transactional Memory. In 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 151–163. issn:1530-0889 https://doi.org/10.1109/DSN.2019.00028
[47]
Gali Sheffi, Maurice Herlihy, and Erez Petrank. 2021. VBR: Version Based Reclamation. In 35th International Symposium on Distributed Computing, DISC 2021, October 4-8, 2021, Freiburg, Germany (Virtual Conference) (LIPIcs, Vol. 209). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 35:1–35:18. https://doi.org/10.4230/LIPIcs.DISC.2021.35
[48]
Gali Sheffi and Erez Petrank. 2023. The ERA Theorem for Safe Memory Reclamation. In Proceedings of the 2023 ACM Symposium on Principles of Distributed Computing (PODC ’23). ACM, New York, NY, USA. 102–112. isbn:9798400701214 https://doi.org/10.1145/3583668.3594564
[49]
Ajay Singh, Trevor Brown, and Ali Mashtizadeh. 2021. NBR: Neutralization Based Reclamation. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’21). ACM, 175–190. isbn:9781450382946 https://doi.org/10.1145/3437801.3441625
[50]
Shahar Timnat, Anastasia Braginsky, Alex Kogan, and Erez Petrank. 2012. Wait-Free Linked-Lists. In Proceedings of the International Conference on Principles of Distributed Systems (OPODIS 2012). Springer Berlin Heidelberg, Berlin, Heidelberg. 330–344. isbn:978-3-642-35476-2 https://doi.org/10.1007/978-3-642-35476-2_23
[51]
Shahar Timnat and Erez Petrank. 2014. A Practical Wait-Free Simulation for Lock-Free Data Structures. In Proceedings of the 19th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’14). ACM, New York, NY, USA. 357–368. isbn:9781450326568 https://doi.org/10.1145/2555243.2555261
[52]
R. K. Treiber. 1986. Systems Programming: Coping with Parallelism. IBM Almaden R. Center.
[53]
John D. Valois. 1995. Lock-free Linked Lists Using Compare-and-swap. In Proceedings of the 14th Annual ACM Symposium on Principles of Distributed Computing (PODC ’95). ACM, New York, NY, USA. 214–222. isbn:0-89791-710-3 https://doi.org/10.1145/224964.224988
[54]
Haosen Wen, Joseph Izraelevitz, Wentao Cai, H. Alan Beadle, and Michael L. Scott. 2018. Interval-based Memory Reclamation. In Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’18). ACM, New York, NY, USA. 1–13. isbn:978-1-4503-4982-6 https://doi.org/10.1145/3178487.3178488
[55]
Chaoran Yang and John Mellor-Crummey. 2016. A Wait-free Queue As Fast As Fetch-and-add. In Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’16). ACM, New York, NY, USA. Article 16, 13 pages. isbn:978-1-4503-4092-2 https://doi.org/10.1145/2851141.2851168
Index Terms
- A Family of Fast and Memory Efficient Lock- and Wait-Free Reclamation
Recommendations
Universal wait-free memory reclamation
PPoPP '20: Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel ProgrammingIn this paper, we present a universal memory reclamation scheme, Wait-Free Eras (WFE), for deleted memory blocks in wait-free concurrent data structures. WFE's key innovation is that it is completely wait-free. Although some prior techniques provide ...
Releasing Memory with Optimistic Access: A Hybrid Approach to Memory Reclamation and Allocation in Lock-Free Programs
SPAA '23: Proceedings of the 35th ACM Symposium on Parallelism in Algorithms and ArchitecturesLock-free data structures are an important tool for the development of concurrent programs as they provide scalability, low latency and avoid deadlocks, livelocks and priority inversion. However, they require some sort of additional support to guarantee ...
Performance of memory reclamation for lockless synchronization
Achieving high performance for concurrent applications on modern multiprocessors remains challenging. Many programmers avoid locking to improve performance, while others replace locks with non-blocking synchronization to protect against deadlock, ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 20 June 2024
Published in PACMPL Volume 8, Issue PLDI
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Funding Sources
- Office of Naval Research
- Air Force Office of Scientific Research
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 323Total Downloads
- Downloads (Last 12 months)323
- Downloads (Last 6 weeks)142
Reflects downloads up to 15 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in