research-article

Automated data race bugs addition

Authors:

Hongliang Liang,

Jianli WangAuthors Info & Claims

EuroSec '20: Proceedings of the 13th European workshop on Systems Security

Pages 37 - 42

https://doi.org/10.1145/3380786.3391401

Published: 27 April 2020 Publication History

Abstract

A challenge faced by concurrency bug detection techniques is the lack of ground-truth corpora, i.e., a lot of true concurrency bugs, making it difficult to evaluate and verify these technologies and tools, e.g., to precisely measure their false negative and false positive rates. In this paper, we present DRInject, a novel dynamic debugging based technique for producing ground-truth corpora by automatically and quickly injecting lots of realistic data race bugs into program source code. Each data race bug is assured by injecting modifying code to a global variable in two concurrency threads. These bugs are realistic in that they are embedded deep with programs and are triggered by real inputs. We have injected over 600 data race bugs into 10 benchmark or real-world programs, including water-nsquared, X264 and libvips. Moreover, we evaluated four data race detectors using the produced buggy programs and found there are much improvement space for these tools. Preliminary experiments show that DRInject can inject data race bugs in large scale programs and evaluate detect tools with fundamental quantities like false negative and false positive rate, which forms the basis to generate large bug corpora for the future research in concurrency software.

References

[1]

F. A. Bianchi, A. Margara, and M. Pezzè. 2018. A Survey of Recent Trends in Testing Concurrent Software Systems. <i>IEEE Transactions on Software Engineering</i> 44, 8 (Aug. 2018), 747–783. 0098-5589 5.

[2]

Christian Bienia, Sanjeev Kumar, Jaswinder Pal Singh, and Kai Li. 2008. The PARSEC Benchmark Suite: Characterization and Architectural Implications. In <i>Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques</i> <i>(PACT '08)</i>. ACM, New York, NY, USA, 72–81.

[3]

Anh-Tu Do-Mai, Thanh-Dang Diep, and Nam Thoai. 2016. Race Condition and Deadlock Detection for Large-Scale Applications. In <i>15th International Symposium on Parallel and Distributed Computing, ISPDC 2016, Fuzhou, China, July 8-10, 2016</i>, Riqing Chen, Chunming Rong, and Dan Grigoras (Eds.). IEEE Computer Society, 319–326.

[4]

B. Dolan-Gavitt, P. Hulin, E. Kirda, T. Leek, A. Mambretti, W. Robertson, F. Ulrich, and R. Whelan. 2016. LAVA: Large-Scale Automated Vulnerability Addition. In <i>2016 IEEE Symposium on Security and Privacy (SP)</i>. 110–121.

[5]

E. Farchi, Y. Nir, and S. Ur. 2003. Concurrent bug patterns and how to test them. In <i>Proceedings International Parallel and Distributed Processing Symposium</i>. 7 pp.–.

[6]

Haojie Fu, Zan Wang, Xiang Chen, and Xiangyu Fan. 2017. A systematic survey on automated concurrency bug detection, exposing, avoidance, and fixing techniques. <i>Software Quality Journal</i> 26 (2017), 855–889. 5.

[7]

Jian Gao, Xin Yang, Yu Jiang, Han Liu, Weiliang Ying, and Xian Zhang. 2018. Jbench: A Dataset of Data Races for Concurrency Testing. In <i>Proceedings of the 15th International Conference on Mining Software Repositories</i> <i>(MSR ’18)</i>. Association for Computing Machinery, New York, NY, USA, 6–9.

Digital Library

[8]

M. Kusano and Chao Wang. 2013. CCmutator: A mutation generator for concurrency constructs in multithreaded C/C++ applications. In <i>2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)</i>. 722–725. 5.

[9]

Chunhua Liao, Pei-Hung Lin, Joshua Asplund, Markus Schordan, and Ian Karlin. 2017. DataRaceBench: A Benchmark Suite for Systematic Evaluation of Data Race Detection Tools. In <i>Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis</i> <i>(SC '17)</i>. ACM, New York, NY, USA, 11:1–11:14.

[10]

Ziyi Lin, Darko Marinov, Hao Zhong, Yuting Chen, and Jianjun Zhao. 2015. JaConTeBe: A Benchmark Suite of Real-World Java Concurrency Bugs (T). In <i>30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9-13, 2015</i>, Myra B. Cohen, Lars Grunske, and Michael Whalen (Eds.). IEEE Computer Society, 178–189.

[11]

Bozhen Liu and Jeff Huang. 2018. D4: Fast Concurrency Debugging with Parallel Differential Analysis. In <i>Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation</i> <i>(PLDI 2018)</i>. ACM, New York, NY, USA, 359–373.

[12]

Radha Nakade, Eric Mercer, Peter Aldous, Kyle Storey, Benjamin Ogles, Joshua Hooker, Sheridan Jacob Powell, and Jay McCarthy. 2019. Model-checking task-parallel programs for data-race. <i>ISSE</i> 15, 3-4 (2019), 289–306.

[13]

Gary Nilson, Kent Wills, Jeffrey Stuckman, and James Purtilo. 2013. BugBox: A Vulnerability Corpus for PHP Web Applications. In <i>6th Workshop on Cyber Security Experimentation and Test (CSET 13)</i>. USENIX Association, Washington, D.C.

[14]

NIST-project. 2017. <i>Software Assurance Reference Dataset Project</i>.

[15]

Oracle. 2014. <i>Doucumentation of thread analyzer</i>.

[16]

Jannik Pewny and Thorsten Holz. 2016. EvilCoder: Automated Bug Insertion. In <i>Proceedings of the 32Nd Annual Conference on Computer Security Applications</i> <i>(ACSAC '16)</i>. ACM, New York, NY, USA, 214–225.

[17]

Valgrind. 2019. <i>Helgrind: a data-race detector</i>.

[18]

Jan Wen Voung, Ranjit Jhala, and Sorin Lerner. 2007. RELAY: Static Race Detection on Millions of Lines of Code. In <i>Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering</i> <i>(ESEC-FSE '07)</i>. ACM, New York, NY, USA, 205–214.

[19]

Steven Cameron Woo, Moriyoshi Ohara, Evan Torrie, Jaswinder Pal Singh, and Anoop Gupta. 1995. The SPLASH-2 Programs: Characterization and Methodological Considerations. In <i>Proceedings of the 22Nd Annual International Symposium on Computer Architecture</i> <i>(ISCA '95)</i>. ACM, New York, NY, USA, 24–36.

[20]

J. Yang, B. Jiang, and W. K. Chan. 2018. HistLock+: Precise Memory Access Maintenance Without Lockset Comparison for Complete Hybrid Data Race Detection. <i>IEEE Transactions on Reliability</i> 67, 3 (Sept. 2018), 786–801. 0018-9529

[21]

Z. Yang, Z. Yu, X. Su, and P. Ma. 2016. RaceTracker: Effective and efficient detection of data races. In <i>2016 17th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD)</i>. 293–300.

[22]

Jie Yu, Satish Narayanasamy, Cristiano Pereira, and Gilles Pokam. 2012. Maple: A Coverage-driven Testing Tool for Multithreaded Programs. In <i>Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications</i> <i>(OOPSLA '12)</i>. ACM, New York, NY, USA, 485–502.

[23]

Misun Yu and Doo-Hwan Bae. 2016. SimpleLock+: Fast and Accurate Hybrid Data Race Detection. <i>Comput. J.</i> 59, 6 (June 2016), 793–809. 0010-4620

[24]

Shixiong Zhao, Rui Gu, Haoran Qiu, Tsz On Li, Yuexuan Wang, Heming Cui, and Junfeng Yang. 2018. OWL: Understanding and Detecting Concurrency Attacks. In <i>48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2018, Luxembourg City, Luxembourg, June 25-28, 2018</i>. IEEE Computer Society, 219–230.

Cited By

Liang JYuan MDing ZMa SHan XZhang C(2023)RaceBench: A Triggerable and Observable Concurrency Bug BenchmarkProceedings of the 2023 ACM Asia Conference on Computer and Communications Security10.1145/3579856.3595787(415-428)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3579856.3595787
Zheng TTong ZYi PWu Y(2022)Automated Generation of Bug Samples Based on Source Code Analysis2022 29th Asia-Pacific Software Engineering Conference (APSEC)10.1109/APSEC57359.2022.00017(51-60)Online publication date: Dec-2022
https://doi.org/10.1109/APSEC57359.2022.00017
Giebas DWojszczyk R(2021)Detection of Concurrency Errors in Multithreaded Applications Based on Static Source Code AnalysisIEEE Access10.1109/ACCESS.2021.30738599(61298-61323)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3073859

Index Terms

Automated data race bugs addition

Recommendations

RAProducer: efficiently diagnose and reproduce data race bugs for binaries via trace analysis
ISSTA 2021: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

A growing number of bugs have been reported by vulnerability discovery solutions. Among them, some bugs are hard to diagnose or reproduce, including data race bugs caused by thread interleavings. Few solutions are able to well address this issue, due to ...
How Good is Static Analysis at Finding Concurrency Bugs?
SCAM '10: Proceedings of the 2010 10th IEEE Working Conference on Source Code Analysis and Manipulation

Detecting bugs in concurrent software is challenging due to the many different thread interleavings. Dynamic analysis and testing solutions to bug detection are often costly as they need to provide coverage of the interleaving space in addition to ...
Data race avoidance and replay scheme for developing and debugging parallel programs on distributed shared memory systems

Distributed shared memory (DSM) allows parallel programs to run on distributed computers by simulating a global virtual shared memory, but data racing bugs may easily occur when the threads of a multi-threaded process concurrently access the physically ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EuroSec '20: Proceedings of the 13th European workshop on Systems Security

April 2020

49 pages

ISBN:9781450375238

DOI:10.1145/3380786

Program Chairs:
Lorenzo Cavallaro
King's College London
,
Andrea Lanzi
University of Milan

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

EuroSys '20

Sponsor:

SIGOPS

EuroSys '20: Fifteenth EuroSys Conference 2020

April 27, 2020

Heraklion, Greece

Acceptance Rates

EuroSec '20 Paper Acceptance Rate 7 of 15 submissions, 47%;

Overall Acceptance Rate 47 of 113 submissions, 42%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
265
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liang JYuan MDing ZMa SHan XZhang C(2023)RaceBench: A Triggerable and Observable Concurrency Bug BenchmarkProceedings of the 2023 ACM Asia Conference on Computer and Communications Security10.1145/3579856.3595787(415-428)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3579856.3595787
Zheng TTong ZYi PWu Y(2022)Automated Generation of Bug Samples Based on Source Code Analysis2022 29th Asia-Pacific Software Engineering Conference (APSEC)10.1109/APSEC57359.2022.00017(51-60)Online publication date: Dec-2022
https://doi.org/10.1109/APSEC57359.2022.00017
Giebas DWojszczyk R(2021)Detection of Concurrency Errors in Multithreaded Applications Based on Static Source Code AnalysisIEEE Access10.1109/ACCESS.2021.30738599(61298-61323)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3073859

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten