Cited By
View all- Gregorek DGarcia-Ortiz A(2018)The Agamid design-space exploration frameworkDesign Automation for Embedded Systems10.1007/s10617-018-9214-322:4(293-314)Online publication date: 1-Dec-2018
Multicore OS benchmarks: we can do better
Abstract
Current multicore OS benchmarks do not provide workloads that sufficiently reflect real-world use: they typically run a single application, whereas real workloads consist of multiple concurrent programs. In this paper we show that this lack of mixed workloads leads to benchmarks that do not fully exercise the OS and are therefore inadequate at predicting real-world behavior. This implies that effective multicore OS benchmarks must includemixed workloads, but the main design challenge is choosing an appropriate mix. We present a principled approachwhich treats benchmark design as an optimization problem. Our solution leads to a workload mix that uses as much of a system's resources as possible, while also selecting applications whose performance is most sensitive to the availability of those resources.
References
[1]
ASANOVIC, K., BODIK, R., CATANZARO, B. C., GEBIS, J. J., Hus-BANDS, P., KEUTZER, K., PATTERSON, D. A., PLISHKER, W. L., SHALF, J., WILLIAMS, S. W., AND YELICK, K. A. The landscape of parallel computing research: A view from Berkeley. Tech. Rep. UCB/EECS-2006-183, EECS Department, University of California, Berkeley, Dec 2006.
[2]
BAUMANN, A., BARHAM, P., DAGAND, P., AND T. The Multikernel: A new OS architecture for scalable multicore systems. In SOSP'09.
[3]
BLACKBURN, S. M., GARNER, R., HOFFMAN, C., KHAN, A. M., McKINLEY, K. S., BENTZUR, R., DIWAN, A., FEINBERG, D., ET AL. The DaCapo benchmarks: Java benchmarking development and analysis. In OOPSLA '06.
[4]
BOYD-WICKIZER, S., CHEN, H., CHEN, R., MAO, Y., KAASHOEK, M. F., MORRIS, R., PESTEREV, A., STEIN, L., WU, M., HUA DAI, Y., ZHANG, Y., AND ZHANG, Z. Corey: An operating system for many cores. In OSDI'08.
[5]
BOYD-WICKIZER, S., CLEMENTS, A. T., MAO, Y., PESTEREV, A., KAASHOEK, M. F., MORRIS, R., AND ZELDOVICH, N. An analysis of Linux scalability to many cores. In OSDI'10.
[6]
CHEN, P. M., AND PATTERSON, D. A. A new approach to I/O performance evaluation-self-scaling I/O benchmarks, predicted I/O performance. In SIGMETRICS'93.
[7]
FRACHTENBERG, E., AND ETSION, Y. Hardware parallelism: Are operating systems ready? (case studies in mis-scheduling). In Workshop on the Interaction between Operating System and Computer Architecture (June 2006).
[8]
GAMSA, B., KRIEGER, O., APPAVOO, J., AND STUMM, M. Tornado: Maximizing locality and concurrency in a shared memory multiprocessor operating system. In OSDI'99.
[9]
MOGUL, J. Brittle metrics in operating systems research. In HotOS-VII (Mar. 1999).
[10]
NIGHTINGALE, E. B., HODSON, O., McILROY, R., HAWBLITZEL, C., AND HUNT, G. Helios: Heterogeneous multiprocessing with satellite kernels. In SOSP '09.
[11]
SELTZER, M., KRINSKY, D., AND SMITH, K. The case for application-specific benchmarking. In HotOS-VII (Mar. 1999).
[12]
WENTZLAFF, D., AND AGARWAL, A. Factored operating systems (fos): The case for a scalable operating system for multicores. ACM SIGOPS Operating Systems Review 43, 2 (Apr. 2009).
[13]
WENTZLAFF, D., III, C. G., AND BECKMANN, N. An operating system for multicore and clouds: Mechanisms and implementation. In ACM Symposium on Cloud Computing (June 2010).
- Multicore OS benchmarks: we can do better
Recommendations
Fast System Level Benchmarks for Multicore Architectures
DSD '14: Proceedings of the 2014 17th Euromicro Conference on Digital System DesignWe present a framework that automatically generates system level synthetic benchmarks from traditional benchmarks. Synthetic benchmarks have similar performance behavior as the original benchmarks that they are generated from and they can run faster. ...
The DaCapo benchmarks: java benchmarking development and analysis
OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applicationsSince benchmarks drive computer science research and industry product development, which ones we use and how we evaluate them are key questions for the community. Despite complex runtime tradeoffs due to dynamic compilation and garbage collection ...
Large System Performance of SPEC OMP2001 Benchmarks
ISHPC '02: Proceedings of the 4th International Symposium on High Performance ComputingPerformance characteristics of application programs on large-scale systems are often significantly different from those on smaller systems. SPEC OMP2001 is a benchmark suite intended for measuring performance of modern shared memory parallel systems. ...
Comments
Information & Contributors
Information
Published In
Sponsors
- VMware
- Google Inc.
- Intel: Intel
- Microsoft Research: Microsoft Research
Publisher
USENIX Association
United States
Publication History
Published: 09 May 2011
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Nov 2024
Other Metrics
Citations
Cited By
View all- Gregorek DGarcia-Ortiz A(2018)The Agamid design-space exploration frameworkDesign Automation for Embedded Systems10.1007/s10617-018-9214-322:4(293-314)Online publication date: 1-Dec-2018