Abstract
This chapter starts by defining the basic concepts: metric, measure, and measurement. It then introduces the different scales of measurement, allowing one to classify the types of values assigned by measures. Next, definitions of the most common performance metrics are presented. The chapter continues with a detailed discussion of the quality attributes of good metrics. Finally, the different types of averages are introduced while showing how they can be used to define composite metrics and aggregate results from multiple benchmarks.
“Measurements are not to provide numbers, but insight.”
—Ingrid Bucher
“It is much easier to make measurements than to know exactly what you are measuring.”
—J. W. N. Sullivan (1886–1937), English journalist and science writer
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
John, L. K. (2004). More on finding a single number to indicate overall performance of a benchmark suite. ACM SIGARCH Computer Architecture News, 32(1), 3–8.
John, L. K. (2006). Aggregating performance metrics over a benchmark suite. In L. K. John & L. Eeckhout (Eds.), Performance evaluation and benchmarking (pp. 47–58). CRC Press, Taylor and Francis Group.
Lilja, D. J. (2000). Measuring computer performance: a practitioner’s guide. Cambridge: Cambridge University Press.
Mashey, J. R. (2004). War of the benchmark means: time for a truce. ACM SIGARCH Computer Architecture News, 32(4), 1–14.
Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103(2684), 677–680.
Stevens, S. S. (1951). Mathematics, measurement, and psychophysics. In S. S. Stevens (Ed.), Handbook of experimental psychology (pp. 1–49). Hoboken, NJ: Wiley.
Stevens, S. S. (1959). Measurement, psychophysics, and utility. In C. Churchman & P. Ratoosh (Eds.), Measurement: definitions and theories (pp. 18–63) Hoboken, NJ: Wiley.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Kounev, S., Lange, KD., Kistowski, J.v. (2020). Metrics. In: Systems Benchmarking. Springer, Cham. https://doi.org/10.1007/978-3-030-41705-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-41705-5_3
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41704-8
Online ISBN: 978-3-030-41705-5
eBook Packages: Computer ScienceComputer Science (R0)