Software Mining -- Investigating Correlation between Source Code Features and Michrobenchmark's Steady State
Pages 107 - 111
Abstract
Microbenchmarking is a widely used method for evaluating the performance of a piece of code. However, the results of microbenchmarks for applications that utilize the Java Virtual Machine (JVM) are often unstable during the initial phase of execution, known as the warmup phase. This is due to the JVM's use of just-in-time compiler optimization, which is to identify and compile a "hot set" of important code regions. In this study we examine the static features of 586 microbenchmarks from 30 Java applications. To do so, we first extract static source code features of the benchmarks and then employ manual and descriptive data mining methods to identify meaningful correlations between these static features and the benchmarks' ability to reach a steady state. Our findings indicate that the number of function calls and lines of code have a considerable influence on whether or not the microbenchmarks reach a steady state.
References
[1]
Barrett, E., Bolz-Tereick, C. F., Killick, R., Mount, S., and Tratt, L. Virtual machine warmup blows hot and cold. Proceedings of the ACM on Programming Languages 1, OOPSLA (2017), 1--27
[2]
Costa, D., Bezemer, C.-P., Leitner, P., and Andrzejak, A. What's wrong with my benchmark results? studying bad practices in jmh benchmarks. IEEE Transactions on Software Engineering 47, 7 (2019), 1452--1467.
[3]
Dongre, J., Prajapati, G. L., and Tokekar, S. The role of apriori algorithm for finding the association rules in data mining. In 2014 International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT) (2014), IEEE, pp. 657--660.
[4]
Gu, D., and Verbrugge, C. Phase-based adaptive recompilation in a jvm. In Pro- ceedings of the 6th annual IEEE/ACM international symposium on Code generation and optimization (2008), pp. 24--34.
[5]
Laaber, C., Basmaci, M., and Salza, P. Predicting unstable software benchmarks using static source code features. Empirical Software Engineering 26, 6 (2021), 1--53.
[6]
Laaber, C., Würsten, S., Gall, H. C., and Leitner, P. Dynamically reconfiguring software microbenchmarks: Reducing execution time without sacrificing result quality. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (2020), pp. 989--1001.
[7]
Leitner, P., and Bezemer, C.-P. An exploratory study of the state of practice of performance testing in java-based open source projects. In Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering (2017), pp. 373--384.
[8]
Liu, Y. Study on application of apriori algorithm in data mining. In 2010 Second international conference on computer modeling and simulation (2010), vol. 3, IEEE, pp. 111--114.
[9]
Maletic, J. I., and Collard, M. L. Exploration, analysis, and manipulation of source code using srcml. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering (2015), vol. 2, IEEE, pp. 951--952.
[10]
Ranjan, R., and Sharma, A. Evaluation of frequent itemset mining platforms using apriori and fp-growth algorithm. arXiv preprint arXiv:1902.10999 (2019).
[11]
Traini, L., Cortellessa, V., Di Pompeo, D., and Tucci, M. Towards effective assessment of steady state performance in java software: are we there yet? Empirical Software Engineering 28, 1 (2023), 1--57.
Index Terms
- Software Mining -- Investigating Correlation between Source Code Features and Michrobenchmark's Steady State
Recommendations
Analysing Static Source Code Features to Determine a Correlation to Steady State Performance in Java Microbenchmarks
ICPE '23 Companion: Companion of the 2023 ACM/SPEC International Conference on Performance EngineeringSource code analysis is an important aspect of software development that provides insight into a program's quality, security and performance. There are few methods for consistently predicting or determining when a written piece of code will end its warm-...
Towards effective assessment of steady state performance in Java software: are we there yet?
AbstractMicrobenchmarking is a widely used form of performance testing in Java software. A microbenchmark repeatedly executes a small chunk of code while collecting measurements related to its performance. Due to Java Virtual Machine optimizations, ...
Predicting unstable software benchmarks using static source code features
AbstractSoftware benchmarks are only as good as the performance measurements they yield. Unstable benchmarks show high variability among repeated measurements, which causes uncertainty about the actual performance and complicates reliable change ...
Comments
Information & Contributors
Information
Published In
April 2023
421 pages
ISBN:9798400700729
DOI:10.1145/3578245
- General Chairs:
- Marco Vieira,
- Valeria Cardellini,
- Program Chairs:
- Antinisca Di Marco,
- Petr Tuma
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 April 2023
Check for updates
Author Tags
Qualifiers
- Short-paper
Conference
ICPE '23
Sponsor:
ICPE '23: ACM/SPEC International Conference on Performance Engineering
April 15 - 19, 2023
Coimbra, Portugal
Acceptance Rates
Overall Acceptance Rate 252 of 851 submissions, 30%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 66Total Downloads
- Downloads (Last 12 months)14
- Downloads (Last 6 weeks)2
Reflects downloads up to 02 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in