article

Balanced Explorative and Exploitative Search with Estimation for Simulation Optimization

Authors:

Sigrún Andradóttir,

Andrei A. PrudiusAuthors Info & Claims

INFORMS Journal on Computing, Volume 21, Issue 2

Pages 193 - 208

https://doi.org/10.1287/ijoc.1080.0309

Published: 01 April 2009 Publication History

Abstract

We discuss desirable features that optimization algorithms should possess to exhibit good empirical performance when applied to solve simulation optimization problems possessing little known structure. Our framework emphasizes maintaining an appropriate balance between exploration, exploitation, and estimation. With the exception of estimation, our ideas are also applicable in (unstructured) deterministic optimization. Exploration refers to (globally) searching the entire feasible region for promising solutions, exploitation refers to the (local) search for improved solutions in promising subregions, and estimation refers to obtaining enhanced estimates of the objective function values at promising solutions and of the optimal solution. We also present two new random search methods that possess these desirable features, prove their almost-sure global convergence, and provide preliminary numerical results that suggest that the proposed framework is promising from a practical point of view.

References

[1]

Ahmed, M. A. and Alkhamis, T. M., "Simulation-based optimization using simulated annealing with ranking and selection," Comput. Oper. Res., v29, pp. 383-402, 2002.

[2]

Alrefaei, M. H. and Andradóttir, S., "A simulated annealing algorithm with constant temperature for discrete stochastic optimization," Management Sci., v45, pp. 748-764, 1999.

Digital Library

[3]

Alrefaei, M. H. and Andradóttir, S., "A modification of the stochastic ruler method for discrete stochastic optimization," Eur. J. Oper. Res., v133, pp. 160-182, 2001.

[4]

Alrefaei, M. H. and Andradóttir, S., "Discrete stochastic optimization using variants of the stochastic ruler method," Naval Res. Logist., v52, pp. 344-360, 2005.

[5]

Andradóttir, S., "A method for discrete stochastic optimization," Management Sci., v41, pp. 1946-1961, 1995.

Digital Library

[6]

Andradóttir, S., "A global search method for discrete stochastic optimization," SIAM J. Optim., v6, pp. 513-530, 1996.

[7]

Andradóttir, S. and Banks, J., "Simulation optimization," Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice, John Wiley & Sons, New York, pp. 307-333, 1998.

[8]

Andradóttir, S., "Accelerating the convergence of random search methods for discrete stochastic optimization," ACM Trans. Model. Comput. Simulation, v9, pp. 349-380, 1999.

Digital Library

[9]

Andradóttir, S., "Rate of convergence of random search methods for discrete optimization using steady-state simulation," 2000.

[10]

Andradóttir, S., Henderson, S. G. and Nelson, B. L., "An overview of simulation optimization via random search," Handbooks in Operations Research and Management Science: Simulation, Elsevier Science, Amsterdam, pp. 617-632, 2006a.

[11]

Andradóttir, S., "Simulation optimization with countably infinite feasible regions: Efficiency and convergence," ACM Trans. Model. Comput. Simulation, v16, pp. 357-374, 2006b.

Digital Library

[12]

Banks, J. A., Carson, J. S., Nelson, B. L. and Nicol, D. M., Discrete-Event System Simulation, Prentice Hall, Upper Saddle River, NJ, 2004.

[13]

Chen, H.-S., Chen, C.-H., Dai, L., Yucesan, E., Andradóttir, S., Healy, K. J., Withers, D. H. and Nelson, B. L., "New development of optimal computing budget allocation for discrete event simulation," Proc. 1997 Winter Simulation Conf., Institute of Electrical and Electronics Engineers, Piscataway, NJ, pp. 334-341, 1997.

Digital Library

[14]

Chick, S. E. and Inoue, K., "New two-stage and sequential procedures for selecting the best simulated system," Oper. Res., v49, pp. 732-743, 2001.

Digital Library

[15]

Devroye, L., "A class of optimal performance-directed probabilistic automata," IEEE Trans. Systems, Man, Cybernetics, v6, pp. 777-784, 1976a.

[16]

Devroye, L., "On the convergence of statistical search," IEEE Trans. Systems, Man, Cybernetics, v6, pp. 46-56, 1976b.

[17]

Devroye, L., "Probabilistic search as a strategy selection procedure," IEEE Trans. Systems, Man, Cybernetics, v6, pp. 315-321, 1976c.

[18]

Fox, B. L. and Heine, G. W., "Probabilistic search with overrides," Ann. Appl. Probab., v5, pp. 1087-1094, 1995.

[19]

Fu, M. C., "Optimization for simulation: Theory vs. practice," INFORMS J. Comput., v14, pp. 192-215, 2002.

Digital Library

[20]

Fu, M. C., Glover, F. W., April, J., Kuhn, M. E., Steiger, N. M., Armstrong, F. B. and Joines, J. A., "Simulation optimization: A review, new developments, and applications," Proc. 2005 Winter Simulation Conf., Institute of Electrical and Electronics Engineers, Piscataway, NJ, pp. 83-95, 2005.

Digital Library

[21]

Gelfand, S. B. and Mitter, S. K., "Simulated annealing with noisy or imprecise energy measurements," J. Optim. Theory Appl., v62, pp. 49-62, 1989.

Digital Library

[22]

Gong, W.-B., Ho, Y.-C. and Zhai, W., "Stochastic comparison algorithm for discrete optimization with estimation," SIAM J. Optim., v10, pp. 384-404, 1999.

Digital Library

[23]

Gutjahr, W. J. and Pflug, G. Ch., "Simulated annealing for noisy cost functions," J. Global Optim., v8, pp. 1-13, 1996.

[24]

Homem-de-Mello, T., "Variable-sample methods for stochastic optimization," ACM Trans. Model. Comput. Simulation, v13, pp. 108-133, 2003.

Digital Library

[25]

Hong, L. J. and Nelson, B. L., "The tradeoff between sampling and switching: New sequential procedures for indifference-zone selection," IIE Trans., v37, pp. 623-634, 2005.

[26]

Hong, L. J. and Nelson, B. L., "Discrete optimization via simulation using COMPASS," Oper. Res., v54, pp. 115-129, 2006.

Digital Library

[27]

Kim, S.-H., Nelson, B. L., Henderson, S. G. and Nelson, B. L., "Selecting the best system," Handbooks in Operations Research and Management Science: Simulation, Elsevier Science, Amsterdam, pp. 501-534, 2006.

[28]

Lawler, E. L., Combinatorial Optimization: Networks and Matroids, Holt, Rinehart and Winston, New York, 1976.

[29]

Norkin, V. I., Ermoliev, Y. M. and Ruszczyński, A., "On optimal allocation of indivisibles under uncertainty," Oper. Res., v46, pp. 381-395, 1998a.

Digital Library

[30]

Norkin, V. I., Pflug, G. Ch. and Ruszczyński, A., "A branch and bound method for stochastic global optimization," Math. Programming, v83, pp. 425-450, 1998b.

Digital Library

[31]

Pichitlamken, J. and Nelson, B. L., "A combined procedure for optimization via simulation," ACM Trans. Model. Comput. Simulation, v13, pp. 155-179, 2003.

Digital Library

[32]

Prudius, A. A., Andradóttir, S., Ingalls, R. G., Rossetti, M. D., Smith, J. S. and Peters, B. A., "Simulation optimization using balanced explorative and exploitative search," Proc. 2004 Winter Simulation Conf., Institute of Electrical and Electronics Engineers, Piscataway, NJ, pp. 545-549, 2004.

Digital Library

[33]

Rubinstein, R. Y. and Kroese, D. P., The Cross-Entropy Method, Springer, New York, 2004.

[34]

Shi, L. and Ólafsson, S., "Nested partitions method for stochastic optimization," Methodology Comput. Appl. Probab., v2, pp. 271-291, 2000.

[35]

Wolpert, D. H. and Macready, W. G., "No free lunch theorems for optimization," IEEE Trans. Evolutionary Comput., v1, pp. 67-82, 1997.

Digital Library

[36]

Yakowitz, S. and Lugosi, E., "Random search in the presence of noise, with application to machine learning," SIAM J. Sci. Statist. Comput., v11, pp. 702-712, 1990.

Digital Library

[37]

Yan, D. and Mukai, H., "Stochastic discrete optimization," SIAM J. Control Optim., v30, pp. 594-612, 1992.

Digital Library

Cited By

Zhang QHu J(2022)Actor-Critic–Like Stochastic Adaptive Search for Continuous Simulation OptimizationOperations Research10.1287/opre.2021.221470:6(3519-3537)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1287/opre.2021.2214
Han JKim SPark C(2021)Improved Penalty Function with Memory for Stochastically Constrained Optimization via SimulationACM Transactions on Modeling and Computer Simulation10.1145/346533331:4(1-26)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3465333
Mathesen LYaghoubi SPedrielli GFainekos G(2019)Falsification of Cyber-Physical Systems with Robustness Uncertainty Quantification Through Stochastic optimization with Adaptive Restart2019 IEEE 15th International Conference on Automation Science and Engineering (CASE)10.1109/COASE.2019.8843005(991-997)Online publication date: 22-Aug-2019
https://dl.acm.org/doi/10.1109/COASE.2019.8843005
Show More Cited By

Balanced Explorative and Exploitative Search with Estimation for Simulation Optimization
1. Information systems
  1. Data management systems
    1. Database administration
2. Theory of computation
  1. Design and analysis of algorithms

Recommendations

An Adaptive Particle Swarm Optimizer Using Balanced Explorative and Exploitative Behaviors
SYNASC '08: Proceedings of the 2008 10th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

Particle Swarm Optimization (PSO) has recently emerged as a nature inspired algorithm for real parameter optimization. This article describes a method for improving the final accuracy and the convergence speed of PSO by adding a new coefficient to the ...
Simulation optimization using balanced explorative and exploitative search
WSC '04: Proceedings of the 36th conference on Winter simulation

We present a new random search method for solving simulation optimization problems. Our approach emphasizes the need for maintaining the right balance between exploration and exploitation during various stages of the search. Exploration represents ...
Self-adaptive velocity particle swarm optimization for solving constrained optimization problems

Particle swarm optimization (PSO) is originally developed as an unconstrained optimization technique, therefore lacks an explicit mechanism for handling constraints. When solving constrained optimization problems (COPs) with PSO, the existing research ...

Comments

Information & Contributors

Information

Published In

cover image INFORMS Journal on Computing

INFORMS Journal on Computing Volume 21, Issue 2

Spring 2009

169 pages

ISSN:1526-5528

Issue’s Table of Contents

Publisher

INFORMS

Linthicum, MD, United States

Publication History

Published: 01 April 2009

Accepted: 01 September 2008

Received: 01 September 2006

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang QHu J(2022)Actor-Critic–Like Stochastic Adaptive Search for Continuous Simulation OptimizationOperations Research10.1287/opre.2021.221470:6(3519-3537)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1287/opre.2021.2214
Han JKim SPark C(2021)Improved Penalty Function with Memory for Stochastically Constrained Optimization via SimulationACM Transactions on Modeling and Computer Simulation10.1145/346533331:4(1-26)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3465333
Mathesen LYaghoubi SPedrielli GFainekos G(2019)Falsification of Cyber-Physical Systems with Robustness Uncertainty Quantification Through Stochastic optimization with Adaptive Restart2019 IEEE 15th International Conference on Automation Science and Engineering (CASE)10.1109/COASE.2019.8843005(991-997)Online publication date: 22-Aug-2019
https://dl.acm.org/doi/10.1109/COASE.2019.8843005
Sun WHu ZHong LJohansson BJain S(2018)Gaussian mixture model-based random search for continuous optimization via simulationProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320757(2003-2014)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320757
Hu LAndradóttir S(2018)An Asymptotically Optimal Set Approach for Simulation OptimizationINFORMS Journal on Computing10.1287/ijoc.2018.081131:1(21-39)Online publication date: 19-Dec-2018
https://dl.acm.org/doi/10.1287/ijoc.2018.0811
Mathesen LPedrielli GNg S(2017)Trust region based stochastic optimization with adaptive restartProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242357(1-12)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242357
Park CKim S(2015)Penalty Function with Memory for Discrete Optimization via Simulation with Stochastic ConstraintsOperations Research10.5555/3215594.321560863:5(1195-1212)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.5555/3215594.3215608
Kocuk BAltınel İAras N(2015)Approximating the objective function's gradient using perceptrons for constrained minimization with application in drag reductionComputers and Operations Research10.1016/j.cor.2015.05.01264:C(139-158)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1016/j.cor.2015.05.012
Wang HGong B(2015)Hierarchical stochastic modeling and optimization for petroleum field development under geological uncertaintyComputers and Industrial Engineering10.1016/j.cie.2014.11.00780:C(23-32)Online publication date: 1-Feb-2015
https://dl.acm.org/doi/10.1016/j.cie.2014.11.007
Sun LHong LHu Z(2014)Balancing Exploitation and Exploration in Discrete Optimization via Simulation Through a Gaussian Process-Based SearchOperations Research10.5555/2777421.277743762:6(1416-1438)Online publication date: 1-Dec-2014
https://dl.acm.org/doi/10.5555/2777421.2777437
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents