research-article

Green Simulation: Reusing the Output of Repeated Experiments

Authors:

Jeremy StaumAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 27, Issue 4

Article No.: 23, Pages 1 - 28

https://doi.org/10.1145/3129130

Published: 27 October 2017 Publication History

Abstract

We introduce a new paradigm in simulation experiment design and analysis, called “green simulation,” for the setting in which experiments are performed repeatedly with the same simulation model. Green simulation means reusing outputs from previous experiments to answer the question currently being asked of the simulation model. As one method for green simulation, we propose estimators that reuse outputs from previous experiments by weighting them with likelihood ratios, when parameters of distributions in the simulation model differ across experiments. We analyze convergence of these estimators as more experiments are repeated, while a stochastic process changes the parameters used in each experiment. As another method for green simulation, we propose an estimator based on stochastic kriging. We find that green simulation can reduce mean squared error by more than an order of magnitude in examples involving catastrophe bond pricing and credit risk evaluation.

References

[1]

Bruce Ankenman, Barry L. Nelson, and Jeremy Staum. 2010. Stochastic kriging for simulation metamodeling. Operat. Res. 58, 2 (2010), 371--382.

Digital Library

[2]

Russell R. Barton, Barry L. Nelson, and Wei Xie. 2014. Quantifying input uncertainty via simulation confidence intervals. INFORMS J. Comput. 26, 1 (2014), 74--87.

Digital Library

[3]

Richard J. Beckman and Michael D. McKay. 1987. Monte Carlo estimation under different distributions using the same simulation. Technometrics 29, 2 (1987), 153--160.

Digital Library

[4]

George D. Birkhoff. 1931. Proof of the ergodic theorem. In Proceedings of the National Academy of Sciences, Vol. 17. National Academy of Sciences, 656--660.

[5]

Eric S. Blake and Ethan J. Gibney. 2011. The Deadliest, Costliest, and Most Intense United States Tropical Cyclones from 1851 to 2010 (and Other Frequently Requested Hurricane Facts). Retrieved July 25, 2017 from http://www.nhc.noaa.gov/dcmi.shtml.

[6]

Joshua D. Coval, Jakub W. Jurek, and Erik Stafford. 2009. Economic catastrophe bonds. Amer. Econ. Rev. 99, 3 (June 2009), 628--666.

[7]

Angelos Dassios and Ji-Wook Jang. 2003. Pricing of catastrophe reinsurance and derivatives using the Cox process with shot noise intensity. Fin. Stochast. 7, 1 (January 2003), 73--95.

[8]

Mingbin Feng and Jeremy Staum. 2015. Green simulation designs for repeated experiments. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, I. Moon, T. M. K. Roeder, C. Macal, and M. D. Rossetti (Eds.). IEEE Press, Piscataway, NJ, 403--413.

[9]

Michael Fu and others. 2015. Handbook of Simulation Optimization. Vol. 216. Springer, New York.

[10]

Paul Glasserman and Xingbo Xu. 2014. Robust risk measurement and model risk. Quant. Fin. 14, 1 (2014), 29--58.

[11]

Tim Hesterberg. 1988. Advances in Importance Sampling. Ph.D. Dissertation. Stanford University.

[12]

Tim Hesterberg. 1995. Weighted average importance sampling and defensive mixture distributions. Technometrics 37, 2 (1995), 185--194.

[13]

Jack P. C. Kleijnen and Reuven Y. Rubinstein. 1996. Optimization and sensitivity analysis of computer simulation models by the score function method. Eur. J. Operat. Res. 88 (1996), 413--427.

[14]

Stuart A. Klugman, Harry H. Panjer, and Gordon E. Willmot. 2012. Loss Models: From Data to Decisions (4 ed.). John Wiley 8 Sons.

[15]

Andrea Laforgia. 1991. Bounds for modified Bessel functions. J. Comput. Appl. Math. 34, 3 (1991), 263--267.

Digital Library

[16]

Pierre L’Ecuyer. 1990. A unified view of the IPA, SF, and LR gradient estimation techniques. Manage. Sci. 36, 11 (1990), 1364--1383.

Digital Library

[17]

Pierre L’Ecuyer. 1993. Two approaches for estimating the gradient in functional form. In Proceedings of the 25th Conference on Winter Simulation. ACM, New York, 338--346.

Digital Library

[18]

Yudell L. Luke. 1972. Inequalities for generalized hypergeometric functions. J. Approx. Theory 5, 1 (1972), 41--65.

[19]

Alvaro Maggiar, Andreas Wächter, Irina S. Dolinskaya, and Jeremy Staum. 2015. A Derivative-Free Algorithm for the Optimization of Functions Smoothed via Gaussian Convolution Using Multiple Importance Sampling. Retrieved May 27, 2017 from http://www.optimization-online.org/DB_HTML/2015/07/5017.html.

[20]

Luca Martino, Victor Elvira, David Luengo, and Jukka Corander. 2015. An adaptive population importance sampler: Learning from uncertainty. IEEE Trans. Signal Process. 63, 16 (2015), 4422--4437.

Digital Library

[21]

Alexander J. McNeil, Rüdiger Frey, and Paul Embrechts. 2005. Quantitative Risk Management: Concepts, Techniques, and Tools. Princeton University Press, Princeton, NJ.

[22]

Robert C. Merton. 1974. On the pricing of corporate debt: The risk structure of interest rates. J. Fin. 29, 2 (1974), 449--470.

[23]

Sean Meyn and Richard L. Tweedie. 2009. Markov Chains and Stochastic Stability (2 ed.). Cambridge University Press, New York, NY.

[24]

Munich Re Geo Risks Research. 2015. Loss Events Worldwide 1980--2014, 10 Costliest Events Ordered by Overall Losses. Technical Report. Munich Re.

[25]

Esa Nummelin. 2004. General Irreducible Markov Chains and Non-Negative Operators. Cambridge Tracts in Mathematics, Vol. 83. Cambridge University Press.

[26]

Art B. Owen and Yi Zhou. 2000. Safe and effective importance sampling. J. Amer. Statist. Assoc. 95, 449 (2000), 135--143.

[27]

Reuven Y. Rubinstein and Alexander Shapiro. 1993. Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method. John Wiley 8 Sons, New Jersey.

[28]

Peter Salemi, Jeremy Staum, and Barry L. Nelson. 2013. Generalized integrated Brownian fields for simulation metamodeling. In Proceedings of the 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World, R. Pasupathy, S.-H. Kim, R. Hill, A. Tolk, and M. E. Kuhl (Eds.). IEEE Press, 543--554.

[29]

Jeremy Staum. 2009. Better simulation metamodeling: The why, what, and how of stochastic kriging. In Proceedings of the 2015 Winter Simulation Conference, M. D. Rossetti, B. Johansson R. R. Hill, A. Dunkin, and R. G. Ingalls (Eds.). IEEE Press, Piscataway, NJ, 119--133.

[30]

Eric Veach. 1997. Robust Monte Carlo Methods for Light Transport Simulation. Ph.D. Dissertation. Stanford University.

Digital Library

[31]

Eric Veach and Leonidas J. Guibas. 1995. Optimally combining sampling techniques for Monte Carlo rendering. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’95). ACM, New York, NY, 419--428.

Digital Library

[32]

Wei Xie, Barry L. Nelson, and Russell R. Barton. 2014. A Bayesian framework for quantifying uncertainty in stochastic simulation. Operat. Res. 62, 6 (2014), 1439--1452.

Digital Library

Cited By

Wu DWang YZhou E(2024)Data-Driven Ranking and Selection Under Input UncertaintyOperations Research10.1287/opre.2022.237572:2(781-795)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1287/opre.2022.2375
Feng BSong E(2024)Efficient Nested Simulation Experiment Design via the Likelihood Ratio MethodINFORMS Journal on Computing10.1287/ijoc.2022.0392Online publication date: 12-Jun-2024
https://doi.org/10.1287/ijoc.2022.0392
Li YKaplan ZNakayama M(2024)Monte Carlo Methods for Economic CapitalINFORMS Journal on Computing10.1287/ijoc.2021.026136:1(266-284)Online publication date: Jan-2024
https://doi.org/10.1287/ijoc.2021.0261
Show More Cited By

Index Terms

Green Simulation: Reusing the Output of Repeated Experiments
1. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis
    2. Simulation types and techniques

Recommendations

Green Simulation with Database Monte Carlo

In a setting in which experiments are performed repeatedly with the same simulation model, green simulation means reusing outputs from previous experiments to answer the question currently being asked of the model. In this article, we address the ...
On the Variance of Single-Run Unbiased Stochastic Derivative Estimators
We analyze the variance of single-run unbiased stochastic derivative estimators. The distribution of a specific conditional expectation characterizes an intrinsic distributional property of the derivative estimators in a given class, which, in turn, ...
Green simulation designs for repeated experiments
WSC '15: Proceedings of the 2015 Winter Simulation Conference

In this article we present the concept of green simulation, which views simulation outputs as scarce resources that should be recycled and reused. Output recycling, if implemented properly, can turn the computational costs in an experiment into ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 27, Issue 4

October 2017

158 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/3155315

Editor:
Adelinde M. Uhrmacher
University of Rostock, Germany

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication Notes

Badge change: Article originally badged under Version 1.0 guidelines https://www.acm.org/publications/policies/artifact-review-badging

Publication History

Published: 27 October 2017

Accepted: 01 July 2017

Revised: 01 May 2017

Received: 01 October 2016

Published in TOMACS Volume 27, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
362
Total Downloads

Downloads (Last 12 months)51
Downloads (Last 6 weeks)4

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu DWang YZhou E(2024)Data-Driven Ranking and Selection Under Input UncertaintyOperations Research10.1287/opre.2022.237572:2(781-795)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1287/opre.2022.2375
Feng BSong E(2024)Efficient Nested Simulation Experiment Design via the Likelihood Ratio MethodINFORMS Journal on Computing10.1287/ijoc.2022.0392Online publication date: 12-Jun-2024
https://doi.org/10.1287/ijoc.2022.0392
Li YKaplan ZNakayama M(2024)Monte Carlo Methods for Economic CapitalINFORMS Journal on Computing10.1287/ijoc.2021.026136:1(266-284)Online publication date: Jan-2024
https://doi.org/10.1287/ijoc.2021.0261
Uhrmacher AFrazier PHähnle RKlügl FLorig FLudäscher BNenzi LRuiz-Martin CRumpe BSzabo CWainer GWilsdorf P(2024)Context, Composition, Automation, and Communication: The C2AC Roadmap for Modeling and SimulationACM Transactions on Modeling and Computer Simulation10.1145/367322634:4(1-51)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3673226
Zhou YAndradóttir SKim S(2024)Selection of the Best in the Presence of Subjective Stochastic ConstraintsACM Transactions on Modeling and Computer Simulation10.1145/366481434:4(1-60)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3664814
Chen QFeng M(2023)Generalized Importance Sampling for Nested SimulationProceedings of the Winter Simulation Conference10.5555/3643142.3643176(409-420)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3643142.3643176
Konrad KLiu Y(2023)Achieving Stable Service-Level Targets in Time-Varying Queueing Systems: A Simulation-Based Offline Learning Staffing AlgorithmProceedings of the Winter Simulation Conference10.5555/3643142.3643169(327-338)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3643142.3643169
Wilsdorf PWolpers AHilton JHaack FUhrmacher A(2023)Automatic Reuse, Adaption, and Execution of Simulation Experiments via Provenance PatternsACM Transactions on Modeling and Computer Simulation10.1145/356492833:1-2(1-27)Online publication date: 28-Feb-2023
https://dl.acm.org/doi/10.1145/3564928
Konrad KLiu Y(2023)Achieving Stable Service-Level Targets in Time-Varying Queueing Systems: A Simulation-Based Offline Learning Staffing Algorithm2023 Winter Simulation Conference (WSC)10.1109/WSC60868.2023.10408273(327-338)Online publication date: 10-Dec-2023
https://doi.org/10.1109/WSC60868.2023.10408273
Chen QBen Feng M(2023)Generalized Importance Sampling for Nested Simulation2023 Winter Simulation Conference (WSC)10.1109/WSC60868.2023.10408062(409-420)Online publication date: 10-Dec-2023
https://doi.org/10.1109/WSC60868.2023.10408062
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents