Article

Experimental design for simulation

Authors:

W. David Kelton,

Russell R. BartonAuthors Info & Claims

WSC '03: Proceedings of the 35th conference on Winter simulation: driving innovation

Pages 59 - 65

Published: 07 December 2003 Publication History

Abstract

This tutorial introduces some of the ideas, issues, challenges, solutions, and opportunities in deciding how to experiment with simulation models to learn about their behavior. Careful planning, or designing, of simulation experiments is generally a great help, saving time and effort by providing efficient ways to estimate the effects of changes in the model's inputs on its outputs. Traditional experimental-design methods are discussed in the context of simulation experiments, as are the broader questions pertaining to planning computer-simulation experiments.

References

[1]

Andradóttir, S. 1998. A review of simulation optimization techniques. In Proceedings of the 1998 Winter Simulation Conference, ed. D. J. Medeiros, E. F. Watson, J. S. Carson, and M. S. Manivannan, 151--158. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

[2]

Banks, J., J. S. Carson, B. L. Nelson, and D. M. Nicol. 2001. Discrete-event system simulation. 3rd ed. Upper Saddle River, N.J.: Prentice-Hall.]]

[3]

Barton, R. R. 1998. Simulation metamodels. In Proceedings of the 1998 Winter Simulation Conference, ed. D. J. Medeiros, E. F. Watson, J. S. Carson, and M. S. Manivannan, 167--174. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

[4]

Barton, R. R. 1999. Graphical methods for the design of experiments. New York: Springer-Verlag.]]

[5]

Barton, R. R. 2002. Designing simulation experiments. In Proceedings of the 2002 Winter Simulation Conference, ed. E. Yücesan, C. -H. Chen, J. L. Snowdon, and J. M. Charnes, 45--51. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

[6]

Box, G.E.P. and N. R. Draper. 1987. Empirical model-building and response surfaces. New York: John Wiley.]]

Digital Library

[7]

Box, G.E.P., W. G. Hunter, and J. S. Hunter. 1978. Statistics for experimenters: an introduction to design, data analysis, and model building. New York: John Wiley.]]

[8]

Fu, M. C. 2002. Optimization for simulation: theory vs. practice. INFORMS Journal on Computing 14 (3): 192--227.]]

Digital Library

[9]

Glover, F., J. P. Kelly, and M. Laguna. 1999. New advances for wedding optimization and simulation. In Proceedings of the 1999 Winter Simulation Conference, ed. P. A. Farrington, H. B. Nembhard, D. T. Sturrock, and G. W. Evans, 255--260. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

[10]

Hood, S. J. and P. D. Welch. 1992. Experimental design issues in simulation with examples from semiconductor manufacturing. In Proceedings of the 1992 Winter Simulation Conference, ed. J. J. Swain, D. Goldsman, R. C. Crain, and J. R. Wilson, 255--263. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

[11]

Hood, S. J. and P. D. Welch. 1993. Response surface methodology and its application in simulation. In Proceedings of the 1993 Winter Simulation Conference, ed. G. W. Evans, M. Mollaghasemi, E. C. Russell, and W. E. Biles, 115--122. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

[12]

Kelton, W. D. 1997. Statistical analysis of simulation output. In Proceedings of the 1997 Winter Simulation Conference, ed. S. Andradottir, K. J. Healy, D. H. Withers, and B. L. Nelson, 23--30. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

[13]

Kelton, W. D. 2000. Experimental design for simulation. In Proceedings of the 2000 Winter Simulation Conference, ed. J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick, 32--38. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

[14]

Kelton, W. D., R. P. Sadowski, and D. A. Sadowski. 2002. Simulation with Arena. 2nd. ed. New York: McGraw-Hill.]]

Digital Library

[15]

Kleijnen, J.P.C. 1998. Experimental design for sensitivity analysis, optimization, and validation of simulation models. In Handbook of simulation, ed. J. Banks, 173--223. New York: John Wiley.]]

[16]

Law, A. M. and W. D. Kelton. 2000. Simulation modeling and analysis. 3rd ed. New York: McGraw-Hill.]]

Digital Library

[17]

Montgomery, D. C. 1997. Design and analysis of experiments. 4th ed. New York: John Wiley.]]

Digital Library

[18]

Swain, J. J. and P. A. Farrington. 1994. Designing simulation experiments for evaluating manufacturing systems. In Proceedings of the 1994 Winter Simulation Conference, ed. J. D. Tew, M. S. Manivannan, D. A. Sadowski, and A. F. Seila, 69--76. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.]]

Digital Library

Cited By

Ryzhov IZhang QChen Y(2020)Advanced statistical methodsProceedings of the Winter Simulation Conference10.5555/3466184.3466186(1-15)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466186
Holmgren JRamstedt LDavidsson PPersson J(2011)Multi-Agent-Based simulation for analysis of transport policy and infrastructure measuresProceedings of the 14th international conference on Agent Based Simulation for a Sustainable Society and Multi-agent Smart Computing10.1007/978-3-642-35612-4_1(1-15)Online publication date: 14-Nov-2011
https://dl.acm.org/doi/10.1007/978-3-642-35612-4_1
Harmon SYoungblood S(2008)Lessons learned about developing acceptable acceptability criteriaProceedings of the 2008 Summer Computer Simulation Conference10.5555/2367656.2367694(1-7)Online publication date: 16-Jun-2008
https://dl.acm.org/doi/10.5555/2367656.2367694
Show More Cited By

Recommendations

Experimental Designs for Computer Simulation Experiments

Experimental designs have found widespread application in the biological and physical sciences. The purpose in using these designs is not only that they are economical in terms of the number of experimental trials required, but more importantly, they ...
Design and Development of an Experimental Design and Evaluation System
IMCCC '14: Proceedings of the 2014 Fourth International Conference on Instrumentation and Measurement, Computer, Communication and Control

This paper develops an experimental and Evaluation system which accelerates the construction of experimental scheme. At the same time, it helps with the creation of experiment samples and evaluation of the impact of experiment variable attributes. Based ...
Experimental optimization of statistical simulation
WSC '74: Proceedings of the 7th conference on Winter simulation - Volume 2

Although simulation involves solving a mathematical model through experimentation, the literature of simulation does not reflect a broad, sustained interest in the design of simulation experiments. The objective of this paper is to fulfill a part of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSC '03: Proceedings of the 35th conference on Winter simulation: driving innovation

December 2003

2094 pages

ISBN:0780381327

General Chair:
David Ferrin
Naperville, IL
,
Program Chair:
Douglas J. Morrice
The University of Texas at Austin, Austin, TX

Sponsors

IIE: Institute of Industrial Engineers
INFORMS/CS: Institute for Operations Research and the Management Sciences/College on Simulation
ASA: American Statistical Association
ACM: Association for Computing Machinery
SIGSIM: ACM Special Interest Group on Simulation and Modeling
IEEE/CS: Institute of Electrical and Electronics Engineers/Computer Society
NIST: National Institute of Standards and Technology
(SCS): The Society for Modeling and Simulation International
IEEE/SMCS: Institute of Electrical and Electronics Engineers/Systems, Man, and Cybernetics Society

Publisher

Winter Simulation Conference

Publication History

Published: 07 December 2003

Check for updates

Qualifiers

Article

Conference

WSC03

Sponsor:

IIE
INFORMS/CS
ASA
ACM
SIGSIM
IEEE/CS
NIST
(SCS)
IEEE/SMCS

WSC03: Winter Simulation Conference 2003

December 7 - 10, 2003

Louisiana, New Orleans

Acceptance Rates

WSC '03 Paper Acceptance Rate 128 of 189 submissions, 68%;

Overall Acceptance Rate 3,413 of 5,075 submissions, 67%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
791
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ryzhov IZhang QChen Y(2020)Advanced statistical methodsProceedings of the Winter Simulation Conference10.5555/3466184.3466186(1-15)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466186
Holmgren JRamstedt LDavidsson PPersson J(2011)Multi-Agent-Based simulation for analysis of transport policy and infrastructure measuresProceedings of the 14th international conference on Agent Based Simulation for a Sustainable Society and Multi-agent Smart Computing10.1007/978-3-642-35612-4_1(1-15)Online publication date: 14-Nov-2011
https://dl.acm.org/doi/10.1007/978-3-642-35612-4_1
Harmon SYoungblood S(2008)Lessons learned about developing acceptable acceptability criteriaProceedings of the 2008 Summer Computer Simulation Conference10.5555/2367656.2367694(1-7)Online publication date: 16-Jun-2008
https://dl.acm.org/doi/10.5555/2367656.2367694
Lee EWhalen T(2007)Synthetic designsACM SIGMETRICS Performance Evaluation Review10.1145/1269899.125490435:1(191-202)Online publication date: 12-Jun-2007
https://dl.acm.org/doi/10.1145/1269899.1254904
Lee EWhalen TGolubchik LAmmar MHarchol-Balter M(2007)Synthetic designsProceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems10.1145/1254882.1254904(191-202)Online publication date: 12-Jun-2007
https://dl.acm.org/doi/10.1145/1254882.1254904
Han SHalpin D(2005)The use of simulation for productivity estimation based on multiple regression analysisProceedings of the 37th conference on Winter simulation10.5555/1162708.1162970(1492-1499)Online publication date: 4-Dec-2005
https://dl.acm.org/doi/10.5555/1162708.1162970
Kleijnen JSanchez SLucas TCioppa T(2005)State-of-the-Art ReviewINFORMS Journal on Computing10.1287/ijoc.1050.013617:3(263-289)Online publication date: 1-Jul-2005
https://dl.acm.org/doi/10.1287/ijoc.1050.0136
Wang SHalpin D(2004)Simulation experiment for improving construction processesProceedings of the 36th conference on Winter simulation10.5555/1161734.1161965(1252-1259)Online publication date: 5-Dec-2004
https://dl.acm.org/doi/10.5555/1161734.1161965

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten