article

The harmonic or geometric mean: does it really matter?

Authors:

Daniel Citron,

Adham Hurani,

Alaa GnadreyAuthors Info & Claims

ACM SIGARCH Computer Architecture News, Volume 34, Issue 4

Pages 18 - 25

https://doi.org/10.1145/1186736.1186738

Published: 01 September 2006 Publication History

Get Access

Abstract

For several decades, computer scientists have been arguing which mean is more appropriate for summarizing computer performance: the harmonic or the geometric. We show that many test cases used in the past to discredit one mean or the other are either artificial or incidental. Changing only one of the benchmarks may result in totally different conclusions.In addition, we conclude that for the SPEC CPU2000 benchmark suite, the choice of averaging has very little influence on the relative standing of different machines. Therefore, the decision to purchase one system rather then another should not be influenced by the type of averaging used.

References

[1]

P. J. Fleming and J. J. Wallace. How not to lie with statistics: the correct way to summarize benchmark results. Commun. ACM, 29(3):218--221, 1986.

Digital Library

Google Scholar

[2]

R. Grappel and J. Hemenway. A tale of four uPs: Benchmarks quantify performance. Electronic Design News (EDN), 26(7):179--265, April 1981.

Google Scholar

[3]

J. L. Heath. Re-evaluation of the RISC I. SIGARCH Comput. Archit. News, 12(1):3--10, 1984.

Digital Library

Google Scholar

[4]

J. L. Hennessy and D. A. Patterson. Computer Architecture; A Quantitative Approach. Morgan Kaufmann Publishers Inc., 1990.

Digital Library

Google Scholar

[5]

R. Jain. The Art of Computer Systems Performance Analysis. John Wiley & Sons, 1991.

Google Scholar

[6]

L. K. John. More on finding a Single Number to indicate Overall Performance of a Benchmark Suite. SIGARCH Computer Architecture News, 32(1):3--8, 2004.

Digital Library

Google Scholar

[7]

D. J. Lilja. Measuring computer performance: a practitioner's guide. Cambridge University Press, 2000.

Digital Library

Google Scholar

[8]

J. R. Mashey. War of the Benchmark Means: Time for a Truce. SIGARCH Comput. Archit. News, 32(4):1--14, 2004.

Digital Library

Google Scholar

[9]

J. E. Smith. Characterizing Computer Performance with a Single Number. Commun. ACM, 31(10):1202--1206, 1988.

Digital Library

Google Scholar

[10]

All SPEC CPU2000 Results Published by SPEC. http://www.spec.org/cpu2000/results/cpu2000.html.

Google Scholar

[11]

SPEC CPU2000. http://www.spec.org/.

Google Scholar

Cited By

View all

Eeckhout L(2024)R.I.P. Geomean Speedup Use Equal-Work (Or Equal-Time) Harmonic Mean Speedup InsteadIEEE Computer Architecture Letters10.1109/LCA.2024.336192523:1(78-82)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1109/LCA.2024.3361925
Li YLi NZhang YGuo JHuang BXing MHuang W(2023)Hmem: A Holistic Memory Performance Metric for Cloud ComputingBenchmarking, Measuring, and Optimizing10.1007/978-981-97-0316-6_11(171-187)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1007/978-981-97-0316-6_11
Zhang QXu LXu B(2023)Python meets JIT compilers: A simple implementation and a comparative evaluationSoftware: Practice and Experience10.1002/spe.326754:2(225-256)Online publication date: 5-Sep-2023
https://doi.org/10.1002/spe.3267
Show More Cited By

Index Terms

The harmonic or geometric mean: does it really matter?

Recommendations

Geometric Bonferroni Mean Operators

The Bonferroni mean has been extensively applied in multicriteria decision-making and support system and developed intuitionistic fuzzy set theory. Based on the second interpretation of the Bonferroni mean, in this paper, we introduce the geometric ...
Harmonic Mean Operators for Aggregating Linguistic Information
ICNC '08: Proceedings of the 2008 Fourth International Conference on Natural Computation - Volume 03

Harmonic mean is widely used to aggregate central tendency data, which is usually expressed in exact numerical values. In this paper, we investigate the situations where the input data are given in the form of linguistic labels, and develop some ...
Fuzzy harmonic mean operators

Harmonic mean is a conservative average, which is widely used to aggregate central tendency data. In the existing literature, the harmonic mean is generally considered as a fusion technique of numerical data information. In this paper, we investigate ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGARCH Computer Architecture News

ACM SIGARCH Computer Architecture News Volume 34, Issue 4

September 2006

37 pages

ISSN:0163-5964

DOI:10.1145/1186736

Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2006

Published in SIGARCH Volume 34, Issue 4

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
1,200
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)2

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Eeckhout L(2024)R.I.P. Geomean Speedup Use Equal-Work (Or Equal-Time) Harmonic Mean Speedup InsteadIEEE Computer Architecture Letters10.1109/LCA.2024.336192523:1(78-82)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1109/LCA.2024.3361925
Li YLi NZhang YGuo JHuang BXing MHuang W(2023)Hmem: A Holistic Memory Performance Metric for Cloud ComputingBenchmarking, Measuring, and Optimizing10.1007/978-981-97-0316-6_11(171-187)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1007/978-981-97-0316-6_11
Zhang QXu LXu B(2023)Python meets JIT compilers: A simple implementation and a comparative evaluationSoftware: Practice and Experience10.1002/spe.326754:2(225-256)Online publication date: 5-Sep-2023
https://doi.org/10.1002/spe.3267
Zhang QXu LXu B(2022)RegCPython: A Register-based Python Interpreter for Better PerformanceACM Transactions on Architecture and Code Optimization10.1145/356897320:1(1-25)Online publication date: 16-Dec-2022
https://dl.acm.org/doi/10.1145/3568973
Elford CAggarwal DShekhar S(2021)Revisiting Issues in Benchmark Metric SelectionPerformance Evaluation and Benchmarking10.1007/978-3-030-84924-5_3(35-47)Online publication date: 4-Aug-2021
https://doi.org/10.1007/978-3-030-84924-5_3
Irving SLi BChen SPeng LZhang WDuan L(2020)Computer comparisons in the presence of performance variationFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-018-7319-214:1(21-41)Online publication date: 1-Feb-2020
https://dl.acm.org/doi/10.1007/s11704-018-7319-2
Tang XWang HMa XEl-Sayed NZhai JChen WAboulnaga ATaufer MBalaji PPeña A(2019)Spread-n-shareProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3295500.3356152(1-15)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1145/3295500.3356152
Costa LEspírito Santo IOliveira P(2016)Benchmarking deterministic optimization algorithms using an outranking approachOptimization Methods & Software10.1080/10556788.2016.119148431:6(1149-1168)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1080/10556788.2016.1191484
Jianming Yu Wei Zhou Yueming Yang Xiaodong Zhang Zhiyi Yu (2015)Many-Core Processors Granularity Evaluation by Considering Performance, Yield, and Lifetime ReliabilityIEEE Transactions on Very Large Scale Integration (VLSI) Systems10.1109/TVLSI.2014.235907623:10(2043-2053)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1109/TVLSI.2014.2359076
Tianshi Chen Qi Guo Temam OYue Wu Yungang Bao Zhiwei Xu Yunji Chen (2015)Statistical Performance Comparisons of ComputersIEEE Transactions on Computers10.1109/TC.2014.231561464:5(1442-1455)Online publication date: 1-May-2015
https://dl.acm.org/doi/10.1109/TC.2014.2315614
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Geometric Bonferroni Mean Operators

Harmonic Mean Operators for Aggregating Linguistic Information

Fuzzy harmonic mean operators

Comments

Published In

Publisher

Publication History

Check for updates

Qualifiers

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Geometric Bonferroni Mean Operators

Harmonic Mean Operators for Aggregating Linguistic Information

Fuzzy harmonic mean operators

Comments

Information

Published In

Publisher

Publication History

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations