research-article

Efficient deterministic multithreading through schedule relaxation

Authors:

John Gallagher,

Junfeng YangAuthors Info & Claims

SOSP '11: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles

Pages 337 - 351

https://doi.org/10.1145/2043556.2043588

Published: 23 October 2011 Publication History

Abstract

Deterministic multithreading (DMT) eliminates many pernicious software problems caused by nondeterminism. It works by constraining a program to repeat the same thread interleavings, or schedules, when given same input. Despite much recent research, it remains an open challenge to build both deterministic and efficient DMT systems for general programs on commodity hardware. To deterministically resolve a data race, a DMT system must enforce a deterministic schedule of shared memory accesses, or mem-schedule, which can incur prohibitive overhead. By using schedules consisting only of synchronization operations, or sync-schedule, this overhead can be avoided. However, a sync-schedule is deterministic only for race-free programs, but most programs have races.

Our key insight is that races tend to occur only within minor portions of an execution, and a dominant majority of the execution is still race-free. Thus, we can resort to a mem-schedule only for the "racy" portions and enforce a sync-schedule otherwise, combining the efficiency of sync-schedules and the determinism of mem-schedules. We call these combined schedules hybrid schedules.

Based on this insight, we have built Peregrine, an efficient deterministic multithreading system. When a program first runs on an input, Peregrine records an execution trace. It then relaxes this trace into a hybrid schedule and reuses the schedule on future compatible inputs efficiently and deterministically. Peregrine further improves efficiency with two new techniques: determinism-preserving slicing to generalize a schedule to more inputs while preserving determinism, and schedule-guided simplification to precisely analyze a program according to a specific schedule. Our evaluation on a diverse set of programs shows that Peregrine is deterministic and efficient, and can frequently reuse schedules for half of the evaluated programs.

References

[1]

ab -- Apache HTTP server benchmarking tool. http://httpd. apache.org/docs/2.2/programs/ab.html.

[2]

The LLVM Compiler Framework. http://llvm.org.

[3]

Parallel BZIP2 (PBZIP2). http://compression.ca/pbzip2/.

[4]

A. V. Aho, M. S. Lam, R. Sethi, and J. D. Ullman. Compilers: Principles, Techniques, and Tools (2nd Edition). Addison-Wesley. 2006.

Digital Library

[5]

G. Altekar and I. Stoica. ODR: output-deterministic replay for multicore debugging. In Proceedings of the 22nd ACM Symposium on Operating Systems Principles (SOSP '09), pages 193--206, Oct. 2009.

Digital Library

[6]

Apache Web Server. http://www.apache.org.

[7]

A. Aviram, S.-C. Weng, S. Hu, and B. Ford. Efficient system-enforced deterministic parallelism. In Proceedings of the Ninth Symposium on Operating Systems Design and Implementation (OSDI '10), Oct. 2010.

Digital Library

[8]

D. Avots, M. Dalton, V. B. Livshits, and M. S. Lam. Improving software security with a C pointer analysis. In Proceedings of the 27th International Conference on Software Engineering (ICSE '05), pages 332--341, May 2005.

Digital Library

[9]

T. Bergan, O. Anderson, J. Devietti, L. Ceze, and D. Grossman. Core-Det: a compiler and runtime system for deterministic multithreaded execution. In Fifteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '10), pages 53--64, Mar. 2010.

Digital Library

[10]

T. Bergan. N. Hunt, L. Ceze, and S. D. Gribble. Deterministic process groups in dOS. In Proceedings of the Ninth Symposium on Operating Systems Design and Implementation (OSDI '10), pages 1--16, Oct. 2010.

Digital Library

[11]

T. Bergan, J. Devietti, N. Hunt, and L. Ceze. The deterministic execution hammer: how well does it actually pound nails? In The 2nd Workshop on Determinism and Correctness in Parallel Programming (WODET '11), Mar. 2011.

[12]

E. Berger, T. Yang, T. Liu, D. Krishnan, and A. Novark. Grace: safe and efficient concurrent programming. In Conference on Object-Oriented Programming Systems, Languages, and Applications (OOPSLA '09), pages 81--96, Oct. 2009.

Digital Library

[13]

S. Bhansali, W.-K. Chen, S. de Jong, A. Edwards, R. Murray, M. Drinić, D. Mihocka, and J. Chau. Framework for instruction-level tracing and analysis of program executions. In Proceedings of the 2nd International Conference on Virtual Execution Environments (VEE '06), pages 154--163, June 2006.

Digital Library

[14]

P. Boonstoppel, C. Cadar, and D. Engler. RWset: attacking path explosion in constraint-based test generation. In Proceedings of the Theory and practice of software, 14th international conference on Tools and algorithms for the construction and analysis of systems, pages 351--366, Mar. 2008.

Digital Library

[15]

C. Cadar, D. Dunbar, and D. Engler. KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the Eighth Symposium on Operating Systems Design and Implementation (OSDI '08), pages 209--224, Dec. 2008.

Digital Library

[16]

M. Castro, M. Costa, and J.-P. Martin. Better bug reporting with better privacy. In Thirteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '08), pages 319--328, Mar. 2008.

Digital Library

[17]

P. Collingbourne, C. Cadar, and P. H. Kelly. Symbolic crosschecking of floating-point and SIMD code. In Proceedings of the 6th ACM European Conference on Computer Systems (EUROSYS '11), pages 315--328, Apr. 2011.

Digital Library

[18]

M. Costa, M. Castro, L. Zhou, L. Zhang, and M. Peinado. Bouncer: securing software by blocking bad input. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP '07), pages 117--130, Oct. 2007.

Digital Library

[19]

H. Cui, J. Wu, C.-C. Tsai, and J. Yang. Stable deterministic multithreading through schedule memoization. In Proceedings of the Ninth Symposium on Operating Systems Design and Implementation (OSDI '10), Oct. 2010.

Digital Library

[20]

J. Devietti, B. Lucia, L. Ceze, and M. Oskin. DMP: deterministic shared memory multiprocessing. In Fourteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '09), pages 85--96, Mar. 2009.

Digital Library

[21]

G. Dunlap, S. T. King, S. Cinar, M. Basrat, and P. Chen. ReVirt: enabling intrusion analysis through virtual-machine logging and replay. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation (OSDI '02), pages 211--224, Dec. 2002.

Digital Library

[22]

G. W. Dunlap, D. G. Lucchetti, M. A. Fetterman, and P. M. Chen. Execution replay of multiprocessor virtual machines. In Proceedings of the 4th International Conference on Virtual Execution Environments (VEE '08), pages 121--130, Mar. 2008.

Digital Library

[23]

D. Engler and K. Ashcraft. RacerX: effective, static detection of race conditions and deadlocks. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP '03), pages 237--252, Oct. 2003.

Digital Library

[24]

P. Fonseca, C. Li, and R. Rodrigues. Finding complex concurrency bugs in large multi-threaded applications. In Proceedings of the 6th ACM European Conference on Computer Systems (EUROSYS '11), pages 215--228, Apr. 2011.

Digital Library

[25]

Q. Gao, W. Zhang, Z. Chen, M. Zheng, and F. Qin. 2ndStrike: towards manifesting hidden concurrency typestate bugs. In Sixteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '11), pages 239--250, Mar. 2011.

Digital Library

[26]

D. Geels, G. Altekar, P. Maniatis, T. Roscoe, and I. Stoica. Friday: global comprehension for distributed replay. In Proceedings of the Fourth Symposium on Networked Systems Design and Implementation (NSDI '07), Apr. 2007.

Digital Library

[27]

Z. Guo, X. Wang, J. Tang, X. Liu, Z. Xu, M. Wu, M. F. Kaashoek, and Z. Zhang. R2: An application-level kernel for record and replay. In Proceedings of the Eighth Symposium on Operating Systems Design and Implementation (OSDI '08), pages 193--208, Dec. 2008.

Digital Library

[28]

B. Hackett and A. Aiken. How is aliasing used in systems software? In Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT '06/FSE-14), pages 69--80, Nov. 2006.

Digital Library

[29]

M. D. Hill and M. Xu. Racey: A stress test for deterministic execution. http://www.cs.wisc.edu/~markhill/racey.html.

[30]

T. Kim, X. Wang, N. Zeldovich, and M. F. Kaashoek. Intrusion recovery using selective re-execution. In Proceedings of the Ninth Symposium on Operating Systems Design and Implementation (OSDI '10), pages 1--9, Oct. 2010.

Digital Library

[31]

J. C. King. A new approach to program testing. In Proceedings of the international conference on Reliable software, pages 228--233, 1975.

Digital Library

[32]

R. Konuru, H. Srinivasan, and J.-D. Choi. Deterministic replay of distributed Java applications. In Proceedings of the 14th International Symposium on Parallel and Distributed Processing (IPDPS '00), pages 219--228, May 2000.

Digital Library

[33]

O. Laadan, N. Viennot, and J. Nieh. Transparent, lightweight application execution replay on commodity multiprocessor operating systems. In Proceedings of the ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '10), pages 155--166, June 2010.

Digital Library

[34]

L. Lamport. Time, clocks, and the ordering of events in a distributed system. Comm. ACM, 21(7):558--565, 1978.

Digital Library

[35]

D. Lee, B. Wester, K. Veeraraghavan, S. Narayanasamy, P. M. Chen, and J. Flinn. Respec: efficient online multiprocessor replayvia speculation and external determinism. In Fifteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '10), pages 77--90, Mar. 2010.

Digital Library

[36]

T. Liu, C. Curtsinger, and E. D. Berger. DTHREADS: efficient deterministic multithreading. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP '11), Oct. 2011.

Digital Library

[37]

S. Lu, J. Tucek, F. Qin, and Y. Zhou. AVIO: detecting atomicity violations via access interleaving invariants. In Twelfth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '06), pages 37--48, Oct. 2006.

Digital Library

[38]

S. Lu, S. Park, C. Hu, X. Ma, W. Jiang, Z. Li, R. A. Popa, and Y. Zhou. Muvi: automatically inferring multi-variable access correlations and detecting related semantic and concurrency bugs. SIGOPS Oper. Syst. Rev., 41(6):103--116, 2007.

Digital Library

[39]

S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In Thirteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '08), pages 329--339, Mar. 2008.

Digital Library

[40]

F. Mattern. Virtual time and global states of distributed systems. In Proceedings of the International Workshop on Parallel and Distributed Algorithms, pages 215--226. 1988.

[41]

P. Montesinos, M. Hicks, S. T. King, and J. Torrellas. Capo: a software-hardware interface for practical deterministic multiprocessor replay. In Fourteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '09), pages 73--84, Mar. 2009.

Digital Library

[42]

M. Olszewski, J. Ansel, and S. Amarasinghe. Kendo: efficient deterministic multithreading in software. In Fourteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '09), pages 97--108, Mar. 2009.

Digital Library

[43]

S. Park, S. Lu, and Y. Zhou. CTrigger: exposing atomicity violation bugs from their hiding places. In Fourteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '09), pages 25--36, Mar. 2009.

Digital Library

[44]

S. Park, Y. Zhou, W. Xiong, Z. Yin, R. Kaushik, K. H. Lee, and S. Lu. PRES: probabilistic replay with execution sketching on multiprocessors. In Proceedings of the 22nd ACM Symposium on Operating Systems Principles (SOSP '09), pages 177--192, Oct. 2009.

Digital Library

[45]

M. Ronsse and K. De Bosschere. Recplay: a fully integrated practical record/replay system. ACM Trans. Comput. Syst., 17(2):133--152, 1999.

Digital Library

[46]

R. Rugina and M. Rinard. Symbolic bounds analysis of pointers, array indices, and accessed memory regions. In Proceedings of the ACM SIGPLAN 2000 Conference on Programming Language Design and Implementation (PLDI '00), pages 182--195, June 2000.

Digital Library

[47]

S. Savage, M. Burrows, G. Nelson, P. Sobalvarro, and T. E. Anderson. Eraser: A dynamic data race detector for multithreaded programming. ACM Trans. Comput. Syst., pages 391--411, Nov. 1997.

Digital Library

[48]

S. M. Srinivasan, S. Kandula, C. R. Andrews, and Y. Zhou. Flashback: A lightweight extension for rollback and deterministic replay for software debugging. In Proceedings of the USENIX Annual Technical Conference (USENIX '04), pages 29--44, June 2004.

Digital Library

[49]

F. Tip. A survey of program slicing techniques. Journal of Programming Languages 3(3), pages 121--189, 1995.

[50]

VMWare Virtual Lab Automation. http://www.vmware.com/solutions/vla/.

[51]

J. Whaley. bddbddb Project. http://bddbddb.sourceforge.net. URL http://bddbddb.sourceforge.net.

[52]

J. Whaley and M. S. Lam. Cloning-based context-sensitive pointer alias analysis using binary decision diagrams. In Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation (PLDI '04), pages 131--144, June 2004.

Digital Library

[53]

J. Wu, H. Cui, and J. Yang. Bypassing races in live applications with execution filters. In Proceedings of the Ninth Symposium on Operating Systems Design and Implementation (OSDI '10), Oct. 2010.

Digital Library

[54]

W. Xiong, S. Park, J. Zhang, Y. Zhou, and Z. Ma. Ad hoc synchronization considered harmful. In Proceedings of the Ninth Symposium on Operating Systems Design and Implementation (OSDI '10), Oct. 2010.

Digital Library

[55]

Y. Yu, T. Rodeheffer, and W. Chen. RaceTrack: efficient detection of data race conditions via adaptive tracking. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP '05), pages 221--234, Oct. 2005.

Digital Library

[56]

D. Yuan, H. Mai, W. Xiong, L. Tan, Y. Zhou, and S. Pasupathy. Sher-Log: error diagnosis by connecting clues from run-time logs. In Fifteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '10), pages 143--154, Mar. 2010.

Digital Library

[57]

C. Zamfir and G. Candea. Execution synthesis: a technique for automated software debugging. In Proceedings of the 5th ACM European Conference on Computer Systems (EUROSYS '10), pages 321--334, Apr. 2010.

Digital Library

[58]

W. Zhang, C. Sun, and S. Lu. ConMem: detecting severe concurrency bugs through an effect-oriented approach. In Fifteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '10), pages 179--192, Mar. 2010.

Digital Library

[59]

W. Zhang, J. Lim, R. Olichandran, J. Scherpelz, G. Jin, S. Lu, and T. Reps. ConSeq: detecting concurrency bugs through sequential errors. In Sixteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '11), pages 251--264, Mar. 2011.

Digital Library

Cited By

Zhao QQiu ZShao SHui XKhan HJin GKloeckner AMoreira J(2022)Understanding and Reaching the Performance Limit of Schedule Tuning on Stable Synchronization DeterminismProceedings of the International Conference on Parallel Architectures and Compilation Techniques10.1145/3559009.3569669(223-238)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3559009.3569669
Chen ZCao Y(2020)JSKernel: Fortifying JavaScript against Web Concurrency Attacks via a Kernel-Like Structure2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN48063.2020.00026(64-75)Online publication date: Jun-2020
https://doi.org/10.1109/DSN48063.2020.00026
Metzler PSuri NWeissenbacher G(2020)Extracting safe thread schedules from incomplete model checking resultsInternational Journal on Software Tools for Technology Transfer10.1007/s10009-020-00575-yOnline publication date: 26-Jun-2020
https://doi.org/10.1007/s10009-020-00575-y
Show More Cited By

Index Terms

Efficient deterministic multithreading through schedule relaxation

Recommendations

Determinism at Standard-Library Level in TM-Based Applications

Deterministic execution of a multi-threaded application guarantees that threads access shared memory in the same order and the application gives the same output whenever it runs with the same input parameters. Determinism provides repeatability, which ...
Accelerated Deterministic Multithreading for Multichannel Video Decoder
COMPSAC '12: Proceedings of the 2012 IEEE 36th Annual Computer Software and Applications Conference

One of the major challenges in developing applications for multicore systems is dealing with non-deterministic behavior. One solution to making threads deterministic is to control their relative execution order based on their progress. That is, we ...
Intention Preservation in Deterministic Multithreading
FGCN '14: Proceedings of the 2014 8th International Conference on Future Generation Communication and Networking

In this paper, we present an analogy between the race condition resolution in a multithreaded application, and conflict resolution for concurrent updates in collaborative editing applications. We introduce the intention preservation problem in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSP '11: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles

October 2011

417 pages

ISBN:9781450309776

DOI:10.1145/2043556

General Chair:
Ted Wobber
MSR Silicon Valley
,
Program Chair:
Peter Druschel
MPI-SWS

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

INESC: Systems and Computer Engineering Institute
SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

SOSP '11

Sponsor:

INESC
SIGOPS

SOSP '11: ACM SIGOPS 23nd Symposium on Operating Systems Principles

October 23 - 26, 2011

Cascais, Portugal

Acceptance Rates

Overall Acceptance Rate 131 of 716 submissions, 18%

Upcoming Conference

SOSP '24

Sponsor:
sigops

ACM SIGOPS 30th Symposium on Operating Systems Principles

November 5 - 8, 2024

Austin , TX , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

63
Total Citations
View Citations
655
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)1

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao QQiu ZShao SHui XKhan HJin GKloeckner AMoreira J(2022)Understanding and Reaching the Performance Limit of Schedule Tuning on Stable Synchronization DeterminismProceedings of the International Conference on Parallel Architectures and Compilation Techniques10.1145/3559009.3569669(223-238)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3559009.3569669
Chen ZCao Y(2020)JSKernel: Fortifying JavaScript against Web Concurrency Attacks via a Kernel-Like Structure2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN48063.2020.00026(64-75)Online publication date: Jun-2020
https://doi.org/10.1109/DSN48063.2020.00026
Metzler PSuri NWeissenbacher G(2020)Extracting safe thread schedules from incomplete model checking resultsInternational Journal on Software Tools for Technology Transfer10.1007/s10009-020-00575-yOnline publication date: 26-Jun-2020
https://doi.org/10.1007/s10009-020-00575-y
Singh SKhurshid S(2020)Parallel Chopped Symbolic ExecutionFormal Methods and Software Engineering10.1007/978-3-030-63406-3_7(107-125)Online publication date: 19-Dec-2020
https://doi.org/10.1007/978-3-030-63406-3_7
Dou XChen PFlinn JBrecht TWilliamson C(2019)ShortCutProceedings of the 27th ACM Symposium on Operating Systems Principles10.1145/3341301.3359659(570-585)Online publication date: 27-Oct-2019
https://dl.acm.org/doi/10.1145/3341301.3359659
Lidbury CDonaldson AMcKinley KFisher K(2019)Sparse record and replay with controlled schedulingProceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3314221.3314635(576-593)Online publication date: 8-Jun-2019
https://dl.acm.org/doi/10.1145/3314221.3314635
Merrifield TRoghanchi SDevietti JEriksson JBahar IHerlihy MWitchel ELebeck A(2019)Lazy Determinism for Faster Deterministic MultithreadingProceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3297858.3304047(879-891)Online publication date: 4-Apr-2019
https://dl.acm.org/doi/10.1145/3297858.3304047
Zhao QQiu ZJin GHollingsworth JKeidar I(2019)Semantics-aware scheduling policies for synchronization determinismProceedings of the 24th Symposium on Principles and Practice of Parallel Programming10.1145/3293883.3295731(242-256)Online publication date: 16-Feb-2019
https://dl.acm.org/doi/10.1145/3293883.3295731
Metzler PSuri NWeissenbacher G(2019)Extracting Safe Thread Schedules from Incomplete Model Checking ResultsModel Checking Software10.1007/978-3-030-30923-7_9(153-171)Online publication date: 2-Oct-2019
https://doi.org/10.1007/978-3-030-30923-7_9
Liu HSilvestro SWang WTian CLiu T(2018)iReplayer: in-situ and identical record-and-replay for multithreaded applicationsACM SIGPLAN Notices10.1145/3296979.319238053:4(344-358)Online publication date: 11-Jun-2018
https://dl.acm.org/doi/10.1145/3296979.3192380
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents