Article

Free access

Sensitivity analysis and optimization in simulation: design of experiments and case studies

Author:

Jack P. C. KleijnenAuthors Info & Claims

WSC '95: Proceedings of the 27th conference on Winter simulation

Pages 133 - 140

https://doi.org/10.1145/224401.224454

Published: 01 December 1995 Publication History

Abstract

This paper is an advanced tutorial on the use of statistical techniques in sensitivity analysis, including the application of these techniques to optimization and validation of simulation models. Sensitivity analysis is divided into two phases. The first phase is a pilot stage, which consists of screening or searching for the important factors; a simple technique is sequential bifurcation. In the second phase, regression analysis is used to approximate the input/output behavior of the simulation model. This regression analysis gives better results when the simulation experiment is well designed, using classical statistical designs such as fractional factorials. To optimize the simulated system, Response Surface Methodology (RSM) is applied; RSM combines regression analysis, design of experiments, and steepest ascent. To validate a simulation model that lacks input/output data, again regression analysis and design of experiments are applied. Several case studies are summarized; they illustrate how in practice statistical techniques can make simulation studies give more general results, in less time.

References

[1]

Bettonvil, B. and J.P.C. Kleijnen. 1994. Identifying the important factors in simulation models with many factors. Tilburg University.

[2]

Glynn, P.W. and D.L. Iglehart. 1989. Importance sampiing for stochastic simulation. Management Science 35: 1367-1392.

Digital Library

[3]

Ho,Y. and X. Cao. 1991. Perturbation analysis of discrete event systems. Dordrecht: Kluwer.

[4]

Hood, S.J. and P.D. Welch. 1993. Response surface methodology and its application in simulation. In Proceedings of the 1993 Winter Simulation Conference.

Digital Library

[5]

Kleijnen, J.P.C. 1987. Statistical tools for simulation practitioners. New York: Marcel Dekker.

Digital Library

[6]

Kleijnen, J.P.C. 1993. Simulation and optimization in production planning: a case study, Decision Support Systems 9: 269-280.

Digital Library

[7]

Kleijnen, J.P.C. 1995a. Verification and validation of simulation models. European Journal of Operational Research 82:145-162.

[8]

Kleijnen, J.P.C. 1995b. Statistical validation of simulation models: a case-study. European Journal of Operational Research (in press).

[9]

Kleijnen, J.P.C. 1996. Simulation: sensitivity analysis and optimization through regression analysis and experimental design. In Proceedings of NATO Advanced Study Institute on Current Issues and Challenges in the Reliability and Maintenance of Complex Systems, Heidelberg: Springer-Verlag.

[10]

Kleijnen, J.P.C, B. Bettonvil, and W. Van Groenendaal. 1995. Validation of simulation models: regression analysis revisited. Tilburg University.

[11]

Kleijnen, J.P.C., G. Van Ham, and J. Rotmans. 1992. Techniques for sensitivity analysis of simulation models: a case study of the CO2 greenhouse effect. Simulation 58: 410-417.

[12]

Kleijnen J.P.C. and W. Van Groenendaal. 1992. Simulation: a statistical perspective. Chichester (U.K.): Wiley.

Digital Library

[13]

Oren, T.I. 1993. Three simulation experimentation environments: SIMAD, SIMGEST, and E/SLAM. In Proceedings of the 1993 European Simulation Symposium. La Jolla: Society for Computer Simulation.

[14]

Rubinstein, R.Y. and A. Shapiro. 1993. Discrete event systems: sensitivity analysis and stochastic optimization via the score function method, New York: Wiley.

[15]

Van Groenendaal, W. 1994. Investment analysis and DSS for gas transmission on Java. Tilburg (Netherlands): Tilburg University.

[16]

Van Meel, J. 1994. The dynamics of business engineering. Delft (Netherlands): Delft University.

[17]

Zeigler, B. 1976. Theory of modelling and simulation. New York: Wiley Interscience.

Digital Library

Cited By

Uhrmacher AFrazier PHähnle RKlügl FLorig FLudäscher BNenzi LRuiz-Martin CRumpe BSzabo CWainer GWilsdorf P(2024)Context, Composition, Automation, and Communication: The C2AC Roadmap for Modeling and SimulationACM Transactions on Modeling and Computer Simulation10.1145/367322634:4(1-51)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3673226
Brito CBarbosa VLima LRocha JAraújo JLopes LRego PSales MCallou GFé ISilva F(2024)Performance evaluation of a video surveillance system using stochastic petri nets for license plate detection on highwaysJournal of Reliable Intelligent Environments10.1007/s40860-024-00235-xOnline publication date: 14-Aug-2024
https://doi.org/10.1007/s40860-024-00235-x
Lima LSabino ABarbosa VFeitosa LBrito CAraujo JSilva F(2024)Dependability analysis and disaster recovery measures in smart hospital systemsJournal of Reliable Intelligent Environments10.1007/s40860-024-00222-2Online publication date: 13-May-2024
https://doi.org/10.1007/s40860-024-00222-2
Show More Cited By

Index Terms

Sensitivity analysis and optimization in simulation: design of experiments and case studies
1. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Statistical graphics

Recommendations

Simulation analysis
WSC '83: Proceedings of the 15th conference on Winter simulation - Volume 1

The use of a computer simulation model to learn about the system(s) under study must involve an analysis of the results from the simulation program itself. A classification of simulation types is given which provides a framework for a treatment of ...
Coupling optimization and statistical analysis with simulation models
WSC '16: Proceedings of the 2016 Winter Simulation Conference

Simulation optimization has become commonplace in commercial simulation tools, but automated statistical analysis of the impacts of varying input parameters is much less common. In this paper we explore how both optimization and statistical analysis can ...
Simulation sensitivity analysis: A frequency domain approach
WSC '81: Proceedings of the 13th conference on Winter simulation - Volume 2

A procedure is presented for assessing the sensitivity of a discrete event digital simulation model to the values assumed for its input parameters. A frequency domain approach is taken where the input parameters oscillate throughout a run of the model. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSC '95: Proceedings of the 27th conference on Winter simulation

December 1995

1493 pages

ISBN:0780330188

Chairmen:
William R. Lilegdon
Pritsker Corp., West Lafayette, IN
,
David Goldsman
Georgia Institute of Technology, Atlanta
,
Editors:
Christos Alexopoulos
Georgia Institute of Technology, Atlanta
,
Keebom Kang
Naval Postgraduate School, Monterey, CA

Sponsors

IIE: Institute of Industrial Engineers
IEEE-SMCS: Systems, Man & Cybernetics Society
INFORMS/CS: Computer Science TC
ASA: American Statistical Association
ACM: Association for Computing Machinery
SCS: Society for Computer Simulation
SIGSIM: ACM Special Interest Group on Simulation and Modeling
IEEE-CS: Computer Society
NIST: National Institute of Standards & Technology

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 December 1995

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

WSC95

Sponsor:

IIE
IEEE-SMCS
INFORMS/CS
ASA
ACM
SCS
SIGSIM
IEEE-CS
NIST

WSC95: 1995 Winter Simulation Conference

December 3 - 6, 1995

Virginia, Arlington, USA

Acceptance Rates

WSC '95 Paper Acceptance Rate 122 of 183 submissions, 67%;

Overall Acceptance Rate 3,413 of 5,075 submissions, 67%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
1,138
Total Downloads

Downloads (Last 12 months)167
Downloads (Last 6 weeks)17

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Uhrmacher AFrazier PHähnle RKlügl FLorig FLudäscher BNenzi LRuiz-Martin CRumpe BSzabo CWainer GWilsdorf P(2024)Context, Composition, Automation, and Communication: The C2AC Roadmap for Modeling and SimulationACM Transactions on Modeling and Computer Simulation10.1145/367322634:4(1-51)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3673226
Brito CBarbosa VLima LRocha JAraújo JLopes LRego PSales MCallou GFé ISilva F(2024)Performance evaluation of a video surveillance system using stochastic petri nets for license plate detection on highwaysJournal of Reliable Intelligent Environments10.1007/s40860-024-00235-xOnline publication date: 14-Aug-2024
https://doi.org/10.1007/s40860-024-00235-x
Lima LSabino ABarbosa VFeitosa LBrito CAraujo JSilva F(2024)Dependability analysis and disaster recovery measures in smart hospital systemsJournal of Reliable Intelligent Environments10.1007/s40860-024-00222-2Online publication date: 13-May-2024
https://doi.org/10.1007/s40860-024-00222-2
Frichi YAboueljinane LJawab F(2023)Using discrete-event simulation to assess an AHP-based dynamic patient prioritisation policy for elective surgeryJournal of Simulation10.1080/17477778.2023.2284145(1-25)Online publication date: 5-Dec-2023
https://doi.org/10.1080/17477778.2023.2284145
Anderson‐Cook CLu L(2023)Is designed data collection still relevant in the big data era?Quality and Reliability Engineering International10.1002/qre.332639:4(1085-1101)Online publication date: 2-Apr-2023
https://doi.org/10.1002/qre.3326
Santos BSoares ANguyen TMin DLee JSilva F(2021)IoT Sensor Networks in Smart Buildings: A Performance Assessment Using Queuing ModelsSensors10.3390/s2116566021:16(5660)Online publication date: 23-Aug-2021
https://doi.org/10.3390/s21165660
Brito CSilva LCallou GNguyen TMin DLee JSilva F(2021)Offloading Data through Unmanned Aerial Vehicles: A Dependability EvaluationElectronics10.3390/electronics1016191610:16(1916)Online publication date: 10-Aug-2021
https://doi.org/10.3390/electronics10161916
Gonçalves IRodrigues LSilva FNguyen TMin DLee J(2021)Surveillance System in Smart Cities: A Dependability Evaluation Based on Stochastic ModelsElectronics10.3390/electronics1008087610:8(876)Online publication date: 7-Apr-2021
https://doi.org/10.3390/electronics10080876
Santos LCunha BFé IVieira MSilva F(2021)Data Processing on Edge and Cloud: A Performability Evaluation and Sensitivity AnalysisJournal of Network and Systems Management10.1007/s10922-021-09592-x29:3Online publication date: 8-Mar-2021
https://doi.org/10.1007/s10922-021-09592-x
Ruscheinski ABudde KWarnke TWilsdorf PHiller BDombrowsky MUhrmacher AJohansson BJain S(2018)Generating simulation experiments based on model documentations and templatesProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320611(715-726)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320611
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents