Abstract
Benchmarks are important tools for studying increasingly complex hardware architectures and software systems.
Two seemingly common assumptions are that the execution time of floating point operations do not change much with different input values, and that the execution time of a benchmark does not vary much if the input and computed values do not influence any branches. These assumption do not always hold.
There is significant overhead in handling denormalized floating point values (a representation automatically used by the CPU to represent values close to zero) on-chip on modern Intel hardware, even if the program can continue uninterrupted. We have observed that even a small fraction of denormal numbers in a textbook benchmark significantly increases the execution time of the benchmark, leading to the wrong conclusions about the relative efficiency of different hardware architectures and about scalability problems of a cluster benchmark.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
IEEE 754: Standard for Binary Floating-Point Arithmetic. http://grouper.ieee.org/groups/754/
Severance, C.: IEEE 754: An Interview with William Kahan. IEEE Computer, pp. 114–115 (March 1998), A longer version is available as An Interview with the Old Man of Floating Point http://www.cs.berkeley.edu/~wkahan/ieee754status/754story.html
Goldberg, D.: What every computer scientist should know about floating-point arithmetic. ACM Comput. Surv. 23(1), 5–48 (1991)
Intel: Intel Architecture Software Developer’s Manual – vol. 1: Basic Architecture. Intel, Order Number 243190 (1999)
Intel: Intel C++ Compiler and Intel Fortran Compiler 8.X Performance Guide, version 1.0.3. Intel Corporation (2004)
LAM-MPI homepage, http://www.lam-mpi.org/
Andrews, G.R.: Foundations of multithreaded, parallel, and distributed programming. Addison-Wesley, Reading (2000)
Whaley, R.C., Petitet, A., Dongarra, J.J.: Automated empirical optimization of software and the ATLAS project (Also available as University of Tennessee LAPACK Working Note #147, UT-CS-00-448, 2000). Parallel Computing 27(1-2), 3–35 (2001), www.netlib.org/lapack/lawns/lawn147.ps
The LINPACK Benchmark, http://www.netlib.org/linpack/
Colwell, B.: What’s the Worst That Can Happen? IEEE Computer, 12–15 (May 2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bjørndalen, J.M., Anshus, O.J. (2007). Trusting Floating Point Benchmarks – Are Your Benchmarks Really Data Independent?. In: Kågström, B., Elmroth, E., Dongarra, J., Waśniewski, J. (eds) Applied Parallel Computing. State of the Art in Scientific Computing. PARA 2006. Lecture Notes in Computer Science, vol 4699. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75755-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-540-75755-9_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75754-2
Online ISBN: 978-3-540-75755-9
eBook Packages: Computer ScienceComputer Science (R0)