research-article

Open access

Examining and Reducing the Influence of Sampling Errors on Feedback-Driven Optimizations

Authors:

Graham YiuAuthors Info & Claims

ACM Transactions on Architecture and Code Optimization (TACO), Volume 13, Issue 1

Article No.: 6, Pages 1 - 24

https://doi.org/10.1145/2851502

Published: 05 April 2016 Publication History

Abstract

Feedback-driven optimization (FDO) is an important component in mainstream compilers. By allowing the compiler to reoptimize the program based on some profiles of the program's dynamic behaviors, it often enhances the quality of the generated code substantially. A barrier for using FDO is that it often requires many training runs to collect enough profiles to amortize the sensitivity of program optimizations to program input changes. Various sampling techniques have been explored to alleviate this time-consuming process. However, the lowered profile accuracy caused by sampling often hurts the benefits of FDO.

This article gives the first systematic study in how sampling rates affect the accuracy of collected profiles and how the accuracy correlates with the usefulness of the profile for modern FDO. Studying basic block and edge profiles for FDO in two mature compilers reveals several counterintuitive observations, one of which is that profiling accuracy does not strongly correlate with the benefits of the FDO. A detailed analysis identifies three types of sampling-caused errors that critically impair the quality of the profiles for FDO. It then introduces a simple way to rectify profiles based on the findings. Experiments demonstrate that the simple rectification fixes most of those critical errors in sampled profiles and significantly enhances the effectiveness of FDO.

References

[1]

M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. 2000. Adaptive optimization in the JalapeNO JVM. In Proceedings of the 15th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications.

[2]

M. Arnold and D. Grove. 2005. Collecting and exploiting high-accuracy call graph profiles in virtual machines. In Proceedings of the International Symposium on Code Generation and Optimization.

[3]

M. Arnold and B. G. Ryder. 2001. A framework for reducing the cost of instrumented code. In Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation. Snowbird, Utah.

[4]

T. Ball and J. R. Larus. 1993. Branch prediction for free. In Proceedings of the ACM SIGPLAN 1993 Conference on Programming Language Design and Implementation (PLDI'93).

[5]

T. Ball and J. R. Larus. 1996. Efficient path profiling. In Proceedings of the 29th Annual ACM/IEEE International Symposium on Microarchitecture.

[6]

P. Berube, A. Preuss, and J. N. Amaral. 2011. Combined profiling: Practical collection of feedback information for code optimization. In Proceedings of the 2nd ACM/SPEC International Conference on Performance Engineering.

[7]

M. D. Bond and K. S. McKinley. 2005. Continuous path and edge profiling. In Proceedings of the 29th Annual ACM/IEEE International Symposium on Microarchitecture.

[8]

D. Buytaert, A. Georges, M. Hind, M. Arnold, L. Eeckhout, and K. De Bosschere. 2007. Using HPM-sampling to drive dynamic compilation. In Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA'07).

[9]

P. P. Chang, S. A. Mahlke, W. Y. Chen, and W. Hwu. 1992. Profile-guided automatic inline expansion for C programs. Software Practice & Experience 22, 5 (1992), 349--369.

Digital Library

[10]

D. Chen, N. Vachharajani, R. Hundt, S. Liao, V. Ramasamy, P. Yuan, W. Chen, and W. Zheng. 2010. Taming hardware event samples for FDO compilation. In Proceedings of the 8th Annual IEEE/ACM International Symposium on Code Generation and Optimization. New York, NY.

[11]

L. Eeckhout. 2010. Computer architecture performance evaluation methods. Synthesis Lectures on Computer Architecture (2010).

[12]

L. Eeckhout, H. Vandierendonck, and K. De Bosschere. 2003. Quantifying the impact of input data sets on program behavior and its applications. Journal of Instruction-Level Parallelism 5 (2003), 1--33.

[13]

J. A. Fisher and S. M. Freudenberger. 1992. Predicting conditional branch directions from previous runs of a program. In Proceedings of the 5th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'92).

[14]

A. H. Hashemi, D. R. Kaeli, and B. Calder. 1997. Procedure mapping using static call graph estimation. In Workshop on Interaction between Compiler and Computer Architecture.

[15]

M. Hirzel and T. M. Chilimbi. 2001. Bursty tracing: A framework for low-overhead temporal profiling. In Proceedings of ACM Workshop on Feedback-Directed and Dynamic Optimization.

[16]

Y. Jiang, E. Z. Zhang, K. Tian, F. Mao, M. Gethers, and X. Shen. 2010. Exploiting statistical correlations for proactive prediction of program behaviors. In International Symposium on Code Generation and Optimization (CGO'10).

[17]

H. Kim, M. Aater Suleman, O. Mutlu, and Y. N. Patt. 2006. 2D-profiling: Detecting input-dependent branches with a single input data set. In Proceedings of the International Symposium on Code Generation and Optimization.

[18]

D. Knuth and F. Stevenson. 1973. BIT Numerical Mathematics 13, 3 (1973), 313--322.

[19]

G. Langdale and T. Gross. 2003. Evaluating the relationship between the usefulness and accuracy of profiles. In Proceedings of the Workshop on Duplicating, Deconstructing, and Debunking.

[20]

B. Lee, K. Resnick, M. D. Bond, and K. S. Mckinley. 2007. Correcting the dynamic call graph using control flow constraints. In International Conference on Compiler Construction.

[21]

H. Mousa and C. Krintz. 2005. HPS: Hybrid profiling support. In Proceedings of the 14th International Conference on Parallel Architectures and Compilation Techniques.

[22]

T. Mytkowicz, A. Diwan, M. Hauswirth, and P. F. Sweeney. 2010. Evaluating the accuracy of Java profilers. In Proceedings of the 2010 ACM SIGPLAN Conference on Programming Language Design and Implementation. New York, NY.

[23]

M. Paleczny, C. Vick, and C. Click. 2001. The Java hotspotTM server compiler. In Proceedings of the 2001 Symposium on JavaTM Virtual Machine Research and Technology Symposium - Volume 1.

[24]

K. Pettis and R. C. Hansen. 1990. Profile guided code positioning. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation.

[25]

X. Shen, J. Shaw, B. Meeker, and C. Ding. 2007. Locality approximation using time. In Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages.

[26]

K. Tian, Y. Jiang, E. Zhang, and X. Shen. 2010. An input-centric paradigm for program dynamic optimizations. In the Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA'10).

[27]

K. Tian, E. Zhang, and X. Shen. 2011. A step towards transparent integration of input-consciousness into dynamic program optimizations. In the Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA).

[28]

K. Vaswani, A. V. Nori, and T. M. Chilimbi. 2007. Preferential path profiling: Compactly numbering interesting paths. In Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages.

[29]

B. Wu, M. Zhou, X. Shen, Y. Gao, R. Silvera, and G. Yiu. 2013. Simple profile rectifications go a long way. In Proceedings of the European Conference on Object-Oriented Programming. 654--678.

[30]

Y. Wu. 2002. Efficient discovery of regular stride patterns in irregular programs. In Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation.

[31]

Y. Wu and J. R. Larus. 1994. Static branch frequency and program profile analysis. In Proceedings of the 27th Annual International Symposium on Microarchitecture (MICRO'94).

Cited By

Ayupov APanchenko MPupyrev SRodríguez GSadayappan PSukumaran-Rajam A(2024)Stale Profile MatchingProceedings of the 33rd ACM SIGPLAN International Conference on Compiler Construction10.1145/3640537.3641573(162-173)Online publication date: 17-Feb-2024
https://dl.acm.org/doi/10.1145/3640537.3641573
He WMestre JPupyrev SWang LYu H(2022)Profile inference revisitedProceedings of the ACM on Programming Languages10.1145/34987146:POPL(1-24)Online publication date: 12-Jan-2022
https://dl.acm.org/doi/10.1145/3498714
Ray P(2021)Optimal Charge Scheduling of EVs Using Aggregator Based Charge Controller with Active Power Support to the Grid2021 IEEE 18th India Council International Conference (INDICON)10.1109/INDICON52576.2021.9691560(1-6)Online publication date: 19-Dec-2021
https://doi.org/10.1109/INDICON52576.2021.9691560
Show More Cited By

Index Terms

Examining and Reducing the Influence of Sampling Errors on Feedback-Driven Optimizations
1. Software and its engineering
  1. Software notations and tools
    1. Compilers

Recommendations

AutoFDO: automatic feedback-directed optimization for warehouse-scale applications
CGO '16: Proceedings of the 2016 International Symposium on Code Generation and Optimization

AutoFDO is a system to simplify real-world deployment of feedback-directed optimization (FDO). The system works by sampling hardware performance monitors on production machines and using those profiles in to guide optimization. Profile data is stale by ...
Taming Hardware Event Samples for Precise and Versatile Feedback Directed Optimizations

Feedback-directed optimization (FDO) is effective in improving application runtime performance, but has not been widely adopted due to the tedious dual-compilation model, the difficulties in generating representative training data sets, and the high ...
Influence of personality traits on users’ viewing behaviour

Different views on the role of personal factors in moderating individual viewing behaviour exist. This study examined the impact of personality traits on individual viewing behaviour of facial stimulus. A total of 96 students (46 males and 50 females, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Architecture and Code Optimization

ACM Transactions on Architecture and Code Optimization Volume 13, Issue 1

April 2016

347 pages

ISSN:1544-3566

EISSN:1544-3973

DOI:10.1145/2899032

Editor:
Koen De Bosschere
Ghent University

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 April 2016

Accepted: 01 November 2015

Revised: 01 November 2015

Received: 01 March 2015

Published in TACO Volume 13, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

DOE Early Career Award
Google Faculty Award
Career Award, IBM CAS Fellowship
National Science Foundation (NSF)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
400
Total Downloads

Downloads (Last 12 months)59
Downloads (Last 6 weeks)6

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ayupov APanchenko MPupyrev SRodríguez GSadayappan PSukumaran-Rajam A(2024)Stale Profile MatchingProceedings of the 33rd ACM SIGPLAN International Conference on Compiler Construction10.1145/3640537.3641573(162-173)Online publication date: 17-Feb-2024
He WMestre JPupyrev SWang LYu H(2022)Profile inference revisitedProceedings of the ACM on Programming Languages10.1145/34987146:POPL(1-24)Online publication date: 12-Jan-2022
Ray P(2021)Optimal Charge Scheduling of EVs Using Aggregator Based Charge Controller with Active Power Support to the Grid2021 IEEE 18th India Council International Conference (INDICON)10.1109/INDICON52576.2021.9691560(1-6)Online publication date: 19-Dec-2021
de Fine Licht JKwasniewski GHoefler TNeuendorffer SShannon L(2020)Flexible Communication Avoiding Matrix Multiplication on FPGA with High-Level SynthesisProceedings of the 2020 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays10.1145/3373087.3375296(244-254)Online publication date: 23-Feb-2020
Su XLiao XXue JReddi VSmith ATang L(2017)Automatic generation of fast BLAS3-GEMM: a portable compiler approachProceedings of the 2017 International Symposium on Code Generation and Optimization10.5555/3049832.3049846(122-133)Online publication date: 4-Feb-2017
Majo ZHartmann TMohler MGross T(2017)Integrating Profile Caching into the HotSpot Multi-Tier Compilation SystemProceedings of the 14th International Conference on Managed Languages and Runtimes10.1145/3132190.3132210(105-118)Online publication date: 27-Sep-2017
Huh JTuck JHunter HMoreno JEmer JSanchez D(2017)Improving the effectiveness of searching for isomorphic chains in superword level parallelismProceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3123939.3124554(718-729)Online publication date: 14-Oct-2017
Sui YFan XZhou HXue J(2016)Loop-oriented array- and field-sensitive pointer analysis for automatic SIMD vectorizationACM SIGPLAN Notices10.1145/2980930.290795751:5(41-51)Online publication date: 13-Jun-2016
Sui YFan XZhou HXue JKuo TWhalley D(2016)Loop-oriented array- and field-sensitive pointer analysis for automatic SIMD vectorizationProceedings of the 17th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, Tools, and Theory for Embedded Systems10.1145/2907950.2907957(41-51)Online publication date: 13-Jun-2016
Kuo TWhalley D(2016)Proceedings of the 17th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, Tools, and Theory for Embedded SystemsundefinedOnline publication date: 13-Jun-2016

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents