research-article

Public Access

An Introduction to Multiobjective Simulation Optimization

Authors:

Susan R. Hunter,

Eric A. Applegate,

Oscar Rincón-Guevara,

Carolina Vivas-ValenciaAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 29, Issue 1

Article No.: 7, Pages 1 - 36

https://doi.org/10.1145/3299872

Published: 24 January 2019 Publication History

All formats PDF

Abstract

The multiobjective simulation optimization (MOSO) problem is a nonlinear multiobjective optimization problem in which multiple simultaneous and conflicting objective functions can only be observed with stochastic error. We provide an introduction to MOSO at the advanced tutorial level, aimed at researchers and practitioners who wish to begin working in this emerging area. Our focus is exclusively on MOSO methods that characterize the entire efficient or Pareto-optimal set as the solution to the MOSO problem; later, this set may be used as input to the broader multicriteria decision-making process. Our introduction to MOSO includes an overview of existing theory, methods, and provably convergent algorithms that explicitly control sampling error for (1) MOSO on finite sets, called multiobjective ranking and selection; (2) MOSO with integer-ordered decision variables; and (3) MOSO with continuous decision variables. In the context of integer-ordered and continuous decision variables, we focus on methods that provably converge to a local efficient set under the natural ordering. We also discuss key open questions that remain in this emerging field.

References

[1]

F. B. Abdelaziz. 2012. Solution approaches for the multiobjective stochastic programming. European Journal of Operational Research 216 (2012), 1--16.

[2]

M. Akan, O. Alagoz, B. Ata, F. S. Erenay, and A. Said. 2012. A broader view of designing the liver allocation system. Operations Research 60, 4 (2012), 757--770.

Digital Library

[3]

R. Allmendinger, M. T. M. Emmerich, J. Hakanen, Y. Jin, and E. Rigoni. 2016. Surrogate-assisted multicriteria optimization: Complexities, prospective solutions, and business case. Journal of Multi-Criteria Decision Analysis 24, 1–2 (January--April 2016), 5--24.

[4]

M. H. Alrefaei and A. H. Diabat. 2009. A simulated annealing technique for multi-objective simulation optimization. Applied Mathematics and Computation 215, 8 (2009), 3029--3035.

Digital Library

[5]

S. Amaran, N. V. Sahinidis, B. Sharda, and S. J. Bury. 2016. Simulation optimization: A review of algorithms and applications. Annals of Operations Research 240, 1 (May 2016), 351--380.

[6]

L. Amodeo, C. Prins, and D. R. Sánchez. 2009. Comparison of metaheuristic approaches for multi-objective simulation-based optimization in supply chain inventory management. In Applications of Evolutionary Computing, M. Giacobini, A. Brabazon, S. Cagnoni, G. A. Di Caro, A. Ekárt, A. I. Esparcia-Alcázar, M. Farooq, A. Fink, P. Machado, J. McCormack, F. Neri, M. O’Neill, M. Preuss, F. Rothlauf, E. Tarantino, and S. Yang (Eds.). Springer, Berlin, 798--807.

Digital Library

[7]

T. W. Anderson. 2003. An Introduction to Multivariate Statistical Analysis (3rd ed.). Wiley-Interscience, Hoboken, NJ.

[8]

M. Andersson, H. Grimm, A. Persson, and A. Ng. 2007. A web-based simulation optimization system for industrial scheduling. In Proceedings of the 2007 Winter Simulation Conference, S. G. Henderson, B. Biller, M.-H. Hsieh, J. Shortle, J. D. Tew, and R. R. Barton (Eds.). IEEE, Piscataway, NJ, 1844--1852.

Digital Library

[9]

S. Andradóttir. 2015. A review of random search methods. In Handbook of Simulation Optimization, M. C. Fu (Ed.). International Series in Operations Research 8 Management Science, Vol. 216. Springer, New York, 277--292.

[10]

E. A. Applegate, G. Feldman, S. R. Hunter, and R. Pasupathy. 2018. Multi-objective ranking and selection: Optimal sampling laws and tractable approximations via SCORE. Optimization Online (2018). Retrieved from http://www.optimization-online.org/DB_HTML/2018/07/6750.html.

[11]

C. Audet, J. Bigeon, D. Cartier, S. Le Digabel, and L. Salomon. 2018. Performance indicators in multiobjective optimization. Optimization Online (2018). Retrieved from http://www.optimization-online.org/DB_HTML/2018/10/6887.html.

[12]

C. Audet, G. Savard, and W. Zghal. 2008. Multiobjective optimization through a series of single-objective formulations. SIAM Journal on Optimization 19, 1 (2008), 188--210.

Digital Library

[13]

F. F. Baesler and J. A. Sepulveda. 2000. Simulation optimization: Multi-response simulation optimization using stochastic genetic search within a goal programming framework. In Proceedings of the 2000 Winter Simulation Conference, J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick (Eds.). IEEE, Piscataway, NJ, 788--794.

Digital Library

[14]

F. F. Baesler and J. A. Sepulveda. 2001. Multi-objective simulation optimization for a cancer treatment center. In Proceedings of the 2001 Winter Simulation Conference, B. A. Peters, J. S. Smith, D. J. Medeiros, and M. W. Rohrer (Eds.). IEEE, Piscataway, NJ, 1405--1411.

Digital Library

[15]

D. Batur and F. F. Choobineh. 2010. Mean-variance based ranking and selection. In Proceedings of the 2010 Winter Simulation Conference, B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yücesan (Eds.). IEEE, Piscataway, NJ.

Digital Library

[16]

D. Bertsimas, V. F. Farias, and N. Trichakis. 2013. Fairness, efficiency, and flexibility in organ allocation for kidney transplantation. Operations Research 61, 1 (2013), 73--87.

Digital Library

[17]

L. Bianchi, M. Dorigo, L. M. Gambardella, and W. J. Gutjahr. 2009. A survey on metaheuristics for stochastic combinatorial optimizaiton. Natural Computing 8 (2009), 239--287.

Digital Library

[18]

P. Billingsley. 1995. Probability and Measure (3rd ed.). John Wiley and Sons, New York.

[19]

J. Boesel, B. L. Nelson, and S.-H. Kim. 2003. Using ranking and selection to “clean up” after simulation optimization. Operations Research 51, 5 (2003), 814--825.

Digital Library

[20]

H. Bonnel and C. Collonge. 2014. Stochastic optimization over a Pareto set associated with a stochastic multi-objective optimization problem. Journal of Optimization Theory and Applications 162 (2014), 405--427.

Digital Library

[21]

H. Bonnel and C. Collonge. 2015. Optimization over the Pareto outcome set associated with a convex bi-objective optimization problem: Theoretical results, deterministic algorithm and application to the stochastic case. Journal of Global Optimization 62 (2015), 481--505.

Digital Library

[22]

L. Bottou, F. E. Curtis, and J. Nocedal. 2017. Optimization methods for large-scale machine learning. ArXiv e-prints (June 2017), 95. arXiv:1606.04838. Retrieved from https://arxiv.org/abs/1606.04838

[23]

J. Branke and W. Zhang. 2015. A new myopic sequential sampling algorithm for multi-objective problems. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, I. Moon, T. M. K. Roeder, C. Macal, and M. D. Rossetti (Eds.). IEEE, Piscataway, NJ, 3589--3598.

Digital Library

[24]

J. Branke, W. Zhang, and Y. Tao. 2016. Multiobjective ranking and selection based on hypervolume. In Proceedings of the 2016 Winter Simulation Conference, T. M. K. Roeder, P. I. Frazier, R. Szechtman, E. Zhou, T. Huschka, and S. E. Chick (Eds.). IEEE, Piscataway, NJ, 859--870.

Digital Library

[25]

D. Brockhoff. 2011. Theoretical aspects of evolutionary multiobjective optimization. In Theory of Randomized Search Heuristics: Foundations and Recent Developments, A. Auger and B. Doerr (Eds.). Series on Theoretical Computer Science, Vol. 1. World Scientific, Hackensack, NJ, 101--139.

[26]

D. Büche, P. Stoll, R. Dornberger, and P. Koumoutsakos. 2002. Multiobjective evolutionary algorithm for the optimization of noisy combustion processes. IEEE Transactions on Systems, Man, and Cybernetics -- Part C: Applications and Reviews 32, 4 (2002), 460--473.

Digital Library

[27]

J. C. Butler, D. J. Morrice, and P. Mullarkey. 2001. A multiple attribute utility theory approach to ranking and selection. Management Science 47, 6 (2001), 800--816.

Digital Library

[28]

G. Casella and R. L. Berger. 2002. Statistical Inference (2nd ed.). Duxbury, Pacific Grove, CA.

[29]

T. M. Chan. 2013. Klee’s measure problem made easy. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science. IEEE, Piscataway, NJ, 410--419.

Digital Library

[30]

W. K. V. Chan, A. D’Ambrogio, G. Zacharewicz, N. Mustafee, G. Wainer, and E. Page (Eds.). 2017. Proceedings of the 2017 Winter Simulation Conference. IEEE, Piscataway, NJ. Retrieved from http://meetings2.informs.org/wordpress/wsc2017/tracks/.

[31]

V. Chankong and Y. Y. Haimes. 1983. Multiobjective Decision Making: Theory and Methodology. Elsevier Science Publishing Co., New York.

[32]

C.-H. Chen, Q.-S. Jia, and L. H. Lee (Eds.). 2013. Stochastic Simulation Optimization for Discrete Event Systems: Perturbation Analysis, Ordinal Optimization, and Beyond. World Scientific, Singapore.

[33]

C.-H. Chen and L. H. Lee. 2010. Stochastic Simulation Optimization: An Optimal Computing Budget Allocation. World Scientific, Hackensack, NJ.

Digital Library

[34]

C.-H. Chen, J. Lin, E. Yücesan, and S. E. Chick. 2000. Simulation budget allocation for further enhancing the efficiency of ordinal optimization. Discrete Event Dynamic Systems 10, 3 (2000), 251--270.

Digital Library

[35]

E. J. Chen and W. D. Kelton. 2005. Sequential selection procedures: Using sample means to improve efficiency. European Journal of Operational Research 166, 1 (2005), 133--153.

[36]

E. J. Chen and L. H. Lee. 2009. A multi-objective selection procedure of determining a Pareto set. Computers 8 Operations Research 36 (2009), 1872--1879.

Digital Library

[37]

T.-L. Chen and C.-C. Wang. 2016. Multi-objective simulation optimization for medical capacity allocation in emergency department. Journal of Simulation 10, 1 (2016), 50--68.

[38]

R. Cheng, M. Li, Y. Tian, X. Zhang, S. Yang, Y. Jin, and X. Yao. 2017. A benchmark test suite for evolutionary many-objective optimization. Complex 8 Intelligent Systems 3, 1 (2017), 67--81.

[39]

E. P. Chew, L. H. Lee, S. Teng, and C. H. Koh. 2009. Differentiated service inventory optimization using nested partitions and MOCBA. Computers 8 Operations Research 36, 5 (2009), 1703--1710.

Digital Library

[40]

M. Chica, A. A. Juan Pérez, O. Cordon, and D. Kelton. 2017. Why simheuristics? Benefits, limitations, and best practices when combining metaheuristics with simulation. SSRN (February 2017).

[41]

S. E. Chick, J. Branke, and C. Schmidt. 2010. Sequential sampling to myopically maximize the expected value of information. INFORMS Journal on Computing 22, 1 (2010), 71--80.

Digital Library

[42]

A. Chinchuluun and P. M. Pardalos. 2007. A survey of recent developments in multiobjective optimization. Annals of Operations Research 154 (2007), 29--50.

[43]

S. H. Choi and T. G. Kim. 2018. Pareto set selection for multiobjective stochastic simulation model. IEEE Transactions on Systems, Man, and Cybernetics: Systems (2018).

[44]

C. A. Coello Coello, G. B. Lamont, and D. A. van Veldhuizen. 2007. Evolutionary Algorithms for Solving Multi-Objective Problems. Springer US, Boston, MA. 800 pages.

Digital Library

[45]

K. Cooper and S. R. Hunter. 2018. PyMOSO: Software for multi-objective simulation optimization with R-PERLE and R-MinRLE. Optimization Online (2018). Retrieved from http://www.optimization-online.org/DB_HTML/2018/10/6876. https://github.com/HunterResearch/PyMOSO.

[46]

K. Cooper, S. R. Hunter, and K. Nagaraj. 2017. An epsilon-constraint method for integer-ordered bi-objective simulation optimization. In Proceedings of the 2017 Winter Simulation Conference, W. K. V. Chan, A. D’Ambrogio, G. Zacharewicz, N. Mustafee, G. Wainer, and E. Page (Eds.). IEEE, Piscataway, NJ, 2303--2314.

Digital Library

[47]

K. Cooper, S. R. Hunter, and K. Nagaraj. 2018. Bi-objective simulation optimization on integer lattices using the epsilon-constraint method in a retrospective approximation framework. Optimization Online (2018). Retrieved from http://www.optimization-online.org/DB_HTML/2018/06/6649.html.

[48]

O. Crespo, J. E. Bergez, and F. Garcia. 2010. Multiobjective optimization subject to uncertainty: Application to irrigation strategy management. Computers and Electronics in Agriculture 74 (2010), 145--154.

Digital Library

[49]

I. Das and J. E. Dennis. 1997. A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multicriteria optimization problems. Structural Optimization 14, 1 (1997), 63--69.

[50]

I. Das and J. E. Dennis. 1998. Normal-boundary intersection: A new method for generating the Pareto surface in nonlinear multicriteria optimization problems. SIAM Journal on Optimization 8, 3 (1998), 631--657.

Digital Library

[51]

K. Deb. 1999. Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evolutionary Computation 7, 3 (1999), 205--230.

Digital Library

[52]

K. Deb. 2009. Multi-objective Optimization Using Evolutionary Algorithms. Wiley, Hoboken, NJ.

Digital Library

[53]

K. Deb, L. Thiele, M. Laumanns, and E. Zitzler. 2001. Scalable Test Problems for Evolutionary Multi-objective Optimization. Technical Report. Institut für Technische Informatik und Kommunikationsnetze, ETH Zürich. TIK-Technical Report No. 112.

[54]

K. Deb, L. Thiele, M. Laumanns, and E. Zitzler. 2002. Scalable multi-objective optimization test problems. In Proceedings of the 2002 Congress on Evolutionary Computation, Vol. 1. IEEE, Piscataway, NJ, 825--830.

[55]

S. Deshpande, L. T. Watson, and R. A. Canfield. 2016. Multiobjective optimization using an adaptive weighting scheme. Optimization Methods and Software 31, 1 (2016), 110--133.

Digital Library

[56]

J. A. Désidéri. 2012. Multiple-gradient descent algorithm (MGDA) for multiobjective optimization. Comptes Rendus Mathematique 350, 5--6 (2012), 313--318.

[57]

H. Ding, L. Benyoucef, and X. Xie. 2006. A simulation-based multi-objective genetic algorithm approach for networked enterprises optimization. Engineering Applications of Artificial Intelligence 19 (2006), 609--623.

[58]

M. Drugan and B. Manderick. 2015. Exploration versus exploitation trade-off in infinite horizon Pareto Multi-Armed bandits algorithms. In Proceedings of the International Conference on Agents and Artificial Intelligence (ICAART’15). SCITEPRESS, Portugal, 66--77.

Digital Library

[59]

E. J. Dudewicz, H. Hyakutake, and V. S. Taneja. 1991. The heteroscedastic method II: Simplified sampling. American Journal of Mathematical and Management Sciences 11, 3--4 (1991), 263--276.

[60]

E. J. Dudewicz and V. S. Taneja. 1978. Multivariate ranking and selection without reduction to a univariate problem. In Proceedings of the 1978 Winter Simulation Conference, H. J. Highland, N. R. Nielsen, and L. G. Hull (Eds.). IEEE, Piscataway, NJ, 207--210.

Digital Library

[61]

E. J. Dudewicz and V. S. Taneja. 1981. A multivariate solution of the multivariate ranking and selection problem. Communications in Statistics -- Theory and Methods 10, 18 (1981), 1849--1868.

[62]

E. J. Dudewicz and V. S. Taneja. 1987. The heteroscedastic method: Multivariate implementation. Annals of the Institute of Statistical Mathematics 39 (1987), 177--190.

[63]

C. Dullinger, W. Struckl, and M. Kozek. 2017. Simulation-based multi-objective system optimization of train traction systems. Simulation Modelling Practice and Theory 72 (2017), 104--117.

[64]

M. Ehrgott. 2005. Multicriteria Optimization (2nd ed.). Lecture Notes in Economics and Mathematical Systems, Vol. 491. Springer, Heidelberg.

Digital Library

[65]

M. Ehrgott. 2012. Vilfredo Pareto and multi-objective optimization. In Optimization Stories: 21st International Symposium on Mathematical Programming, M. Grötschel (Ed.). Documenta mathematica, Bielefeld, Germany, 447--453.

[66]

M. Ehrgott, X. Gandibleux, and A. Przybylski. 2016. Exact methods for multi-objective combinatorial optimisation. In Multiple Criteria Decision Analysis: State of the Art Surveys, S. Greco, M. Ehrgott, and J. R. Figueira (Eds.). International Series in Operations Research 8 Management Science, Vol. 233. Springer, New York, 817--850.

[67]

M. Ehrgott and D. Tenfelde-Podehl. 2003. Computation of ideal and nadir values and implications for their use in MCDM methods. European Journal of Operational Research 151 (2003), 119--139.

[68]

M. Ehrgott and M. M. Wiecek. 2005. Multiobjective programming. In Multiple Criteria Decision Analysis: State of the Art Surveys, J. Figueira, S. Greco, and M. Ehrgott (Eds.). International Series in Operations Research 8 Management Science, Vol. 78. Springer, New York.

[69]

G. Eichfelder. 2008. Adaptive Scalarization Methods in Multiobjective Optimization. Springer, Berlin.

[70]

H. Eskandari and G. D. Geiger. 2009. Evolutionary multiobjective optimization in noisy problem environments. Journal on Heuristics 15 (2009), 559--595.

Digital Library

[71]

G. W. Evans, B. Stuckman, and M. Mollaghasemi. 1991. Multicriteria optimization of simulation models. In Proceedings of the 1991 Winter Simulation Conference, B. L. Nelson, W. D. Kelton, and G. M. Clark (Eds.). IEEE, Piscataway, NJ, 894--900.

Digital Library

[72]

S. L. Faulkenberg and M. M. Wiecek. 2010. On the quality of discrete representations in multiple objective programming. Optimization and Engineering 11 (2010), 423--440.

[73]

G. Feldman. 2017. Sampling Laws for Multi-objective Simulation Optimization on Finite Sets. Ph.D. Dissertation. Purdue University.

[74]

G. Feldman and S. R. Hunter. 2018. SCORE allocations for bi-objective ranking and selection. ACM Transactions on Modeling and Computer Simulation 28, 1, Article 7 (January 2018), 28 pages.

Digital Library

[75]

G. Feldman, S. R. Hunter, and R. Pasupathy. 2015. Multi-objective simulation optimization on finite sets: Optimal allocation via scalarization. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, I. Moon, T. M. K. Roeder, C. Macal, and M. D. Rossetti (Eds.). IEEE, Piscataway, NJ, 3610--3621.

Digital Library

[76]

W. H. Feng, N. Kong, and H. Wan. 2013. A simulation study of cadaveric liver allocation with a single-score patient prioritization formula. Journal of Simulation 7, 2 (2013), 109--125.

[77]

J. Fliege, L. M. Graña Drummond, and B. F. Svaiter. 2009. Newton’s method for multiobjective optimization. SIAM Journal on Optimization 20, 2 (2009), 602--626.

Digital Library

[78]

J. Fliege and B. F. Svaiter. 2000. Steepest descent methods for multicriteria optimization. Mathematical Methods of Operations Research 51, 3 (2000), 479--494.

[79]

J. Fliege and R. Werner. 2014. Robust multiobjective optimization 8 applications in portfolio optimization. European Journal of Operational Research 234 (2014), 422--433.

[80]

J. Fliege and H. Xu. 2011. Stochastic multiobjective optimization: Sample average approximation and applications. Journal of Optimization Theory and Applications 151 (2011), 135--162.

Digital Library

[81]

P. I. Frazier and A. M. Kazachkov. 2011. Guessing preferences: A new approach to multi-attribute ranking and selection. In Proceedings of the 2011 Winter Simulation Conference, S. Jain, R. R. Creasey, J. Himmelspach, K. P. White, and M. Fu (Eds.). IEEE, Piscataway, NJ, 4324--4336.

Digital Library

[82]

L. W. Friedman. 1987. Systems simulation. Design and analysis of multivariate response simulations: The state of the art. Behavioral Science 32 (1987), 138--148.

[83]

M. Fu. 1994. Optimization via simulation: A review. Annals of Operations Research 53 (1994), 199--247.

[84]

M. C. Fu. 2002. Optimization for simulation: Theory vs. practice. INFORMS Journal on Computing 14 (2002), 192--215.

[85]

M. C. Fu (Ed.). 2015. Handbook of Simulation Optimization. International Series in Operations Research 8 Management Science, Vol. 216. Springer, New York.

Digital Library

[86]

M. C. Fu, G. Bayraksan, S. G. Henderson, B. L. Nelson, W. B. Powell, I. O. Ryzhov, and B. Thengvall. 2014. Simulation optimization: A panel on the state of the art in research and practice. In Proceedings of the 2014 Winter Simulation Conference, A. Tolk, S. Y. Diallo, I. O. Ryzhov, L. Yilmaz, S. Buckley, and J. A. Miller (Eds.). IEEE, Piscataway, NJ, 3696--3706.

Digital Library

[87]

D. Goldsman. 2015. A practical guide to ranking and selection methods. In TutORials in Operations Research, D. M. Aleman and A. C. Thiele (Eds.). INFORMS, Catonsville, MD, Chapter 6, 89--110.

[88]

S. Gören, A. Baccouche, and H. Pierreval. 2017. A framework to incorporate decision-maker preferences into simulation optimization to support collaborative design. IEEE Transactions on Systems, Man, and Cybernetics: Systems 47, 2 (February 2017), 229--237.

[89]

W. J. Gutjahr. 2005. Two metaheuristics for multiobjective stochastic combinatorial optimization. In Stochastic Algorithms: Foundations and Applications: 3rd International Symposium (SAGA’05), O. B. Lupanov, O. M. Kasim-Zade, A. V. Chaskin, and K. Steinhöfel (Eds.). Springer-Verlag, Berlin, 116--125.

Digital Library

[90]

W. J. Gutjahr. 2009. A provably convergent heuristic for stochastic bicriteria integer programming. Journal on Heuristics 15 (2009), 227--258.

Digital Library

[91]

W. J. Gutjahr. 2011. Recent trends in metaheuristics for stochastic combinatorial optimization. Central European Journal of Computer Science 1, 1 (2011), 58--66.

[92]

W. J. Gutjahr and A. Pichler. 2016. Stochastic multi-objective optimization: A survey on non-scalarizing methods. Annals of Operations Research 236, 2 (January 2016), 475--499.

[93]

F. S. Hashemi, R. Pasupathy, and M. R. Taaffe. 2017. The adaptive sampling gradient method: Optimizing smooth functions with an inexact oracle. Optimization Online (May 2017). Retrieved from http://www.optimization-online.org/DB_HTML/2017/05/6045.html.

[94]

S. G. Henderson and R. Pasupathy. 2018. Simulation Optimization Library. Retrieved from http://www.simopt.org.

[95]

S. Hoffenson, S. Arepally, and P. Y. Papalambros. 2014. A multi-objective optimization framework for assessing military ground vehicle design for safety. Journal of Defense Modeling and Simulation: Applications, Methodology, Technology 11, 1 (2014), 33--46.

[96]

T. Homem-de-Mello and G. Bayraksan. 2015. Stochastic constraints and variance reduction techniques. In Handbook of Simulation Optimization, M. C. Fu (Ed.). International Series in Operations Research 8 Management Science, Vol. 216. Springer, New York, 245--276.

[97]

L. J. Hong and B. L. Nelson. 2006. Discrete optimization via simulation using COMPASS. Operations Research 54, 1 (January--February 2006), 115--129.

Digital Library

[98]

L. J. Hong and B. L. Nelson. 2009. A brief introduction to optimization via simulation. In Proceedings of the 2009 Winter Simulation Conference, M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin, and R. G. Ingalls (Eds.). IEEE, Piscataway, NJ, 75--85.

Digital Library

[99]

L. J. Hong, B. L. Nelson, and J. Xu. 2015. Discrete optimization via simulation. In Handbook of Simulation Optimization, M. C. Fu (Ed.). International Series in Operations Research 8 Management Science, Vol. 216. Springer, New York, 9--44.

[100]

H. Huang. 2016. Discrete-Event Simulation and Optimization to Improve the Performance of a Healthcare System. Ph.D. Dissertation. University of Washington.

[101]

H. Huang and Z. B. Zabinsky. 2014. Multiple objective probabilistic branch and bound for Pareto optimal approximation. In Proceedings of the 2014 Winter Simulation Conference, A. Tolk, S. Y. Diallo, I. O. Ryzhov, L. Yilmaz, S. Buckley, and J. A. Miller (Eds.). IEEE, Piscataway, NJ, 3916--3927.

Digital Library

[102]

S. Huband, P. Hingston, L. Barone, and L. While. 2006. A review of multiobjective test problems and a scalable test problem toolkit. IEEE Transactions on Evolutionary Computation 10, 5 (2006), 477--506.

Digital Library

[103]

S. R. Hunter and B. McClosky. 2016. Maximizing quantitative traits in the mating design problem via simulation-based Pareto estimation. IIE Transactions 48, 6 (2016), 565--578.

[104]

S. R. Hunter and B. L. Nelson. 2017. Parallel ranking and selection. In Advances in Modeling and Simulation: Seminal Research from 50 Years of Winter Simulation Conferences, A. Tolk, J. Fowler, G. Shao, and E. Yücesan (Eds.). Springer International, Switzerland, Chapter 12, 249--275.

[105]

C.-L. Hwang and A. S. M. Masud. 1979. Multiple Objective Decision Making -- Methods and Applications: A State-of-the-Art Survey. Springer-Verlag, Berlin.

[106]

S. H. Jacobson and L. W. Schruben. 1989. Techniques for simulation response optimization. Operations Research Letters 8 (February 1989), 1--9.

Digital Library

[107]

J. Jahn. 2011. Vector Optimization: Theory, Applications, and Extensions (2 ed.). Springer-Verlag, Berlin.

[108]

N. Jian and S. G. Henderson. 2015. An introduction to simulation optimization. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, T. M. K. Roeder, C. Macal, and M.D. Rosetti (Eds.). IEEE, Piscataway, NJ, 1780--1794.

Digital Library

[109]

S. Jiang, Y.-S. Ong, J. Zhang, and L. Feng. 2014. Consistencies and contradictions of performance metrics in multiobjective optimization. IEEE Transactions on Cybernetics 44, 12 (2014), 2391--2404.

[110]

Y. Jin and J. Branke. 2005. Evolutionary optimization in uncertain environments: A survey. IEEE Transactions on Evolutionary Computation 7, 3 (June 2005), 303--317.

Digital Library

[111]

J. A. Joines, D. Gupta, M. A. Gokce, R. E. King, and M. G. Kay. 2002. Supply chain multi-objective simulation optimization. In Proceedings of the 2002 Winter Simulation Conference, E. Yücesan, C. H. Chen, J. L. Snowdon, and J. M. Charnes (Eds.). IEEE, Piscataway, NJ, 1306--1314.

Digital Library

[112]

S. Kim. 2014. A derivative-free trust-region method for biobjective optimization. Retrieved from https://vimeo.com/83787562.

[113]

S. Kim, R. Pasupathy, and S. G. Henderson. 2015. A guide to sample average approximation. In Handbook of Simulation Optimization, M. Fu (Ed.). International Series in Operations Research 8 Management Science, Vol. 216. Springer-Verlag, New York, 207--243.

[114]

S. Kim and J. Ryu. 2011a. The sample average approximation method for multi-objective stochastic optimization. In Proceedings of the 2011 Winter Simulation Conference, S. Jain, R. R. Creasey, J. Himmelspach, K. P. White, and M. Fu (Eds.). IEEE, Piscataway, NJ, 4026--4037.

Digital Library

[115]

S. Kim and J. Ryu. 2011b. A trust-region algorithm for bi-objective stochastic optimization. Procedia Computer Science 4 (May 2011), 1422--1430.

[116]

S.-H. Kim and B. L. Nelson. 2006. Selecting the best system. In Simulation, S. G. Henderson and B. L. Nelson (Eds.). Elsevier, Amsterdam, The Netherlands, 501--534.

[117]

G. Kirlik and S. Sayin. 2013. Computing the nadir point for multiobjective discrete optimization problems. Journal of Global Optimization 62 (2013), 79--99.

Digital Library

[118]

T. Klein, A. Holzkämper, P. Calanca, R. Seppelt, and J. Fuhrer. 2013. Adapting agricultural land management to climate change: A regional multi-objective optimization approach. Landscape Ecology 28 (2013), 2029--2047.

[119]

T. C. Koopmans. 1951. Analysis of production as an efficient combination of activities. In Activity Analysis of Production and Allocation, T. C. Koopmans (Ed.). John Wiley 8 Sons, New York, 33--97.

[120]

H. W. Kuhn and A. W. Tucker. 1951. Nonlinear programming. In Proceedings of the 2nd Berkeley Symposium on Mathematical Statistics and Probability, J. Neyman (Ed.). University of California Press, Berkeley, CA, 481--492. Retrieved from https://projecteuclid.org/euclid.bsmsp/1200500249.

[121]

G. W. Kuriger and F. H. Grant. 2011. A lexicographic Nelder-Mead simulation optimization method to solve multi-criteria problems. Computers 8 Industrial Engineering 60, 4 (2011), 555--565.

Digital Library

[122]

H. Kushner and G. G. Yin. 2003. Stochastic Approximation and Recursive Algorithms and Applications (2nd ed.). Stochastic Modeling and Applied Probability, Vol. 35. Springer-Verlag, New York.

[123]

R. Lacour, K. Klamroth, and C. M. Fonseca. 2017. A box decomposition algorithm to compute the hypervolume indicator. Computers 8 Operations Research 79 (March 2017), 347--360.

[124]

A. M. Law. 2015. Simulation Modeling and Analysis (5th ed.). McGraw Hill Education, New York.

[125]

J. S. Lee. 2014. Advances in Simulation: Validity and Efficiency. Ph.D. Dissertation. Georgia Institute of Technology. Retrieved from http://hdl.handle.net/1853/53457.

[126]

L. H. Lee, E. P. Chew, S. Teng, and Y. Chen. 2008. Multi-objective simulation-based evolutionary algorithm for an aircraft spare parts allocation problem. European Journal of Operational Research 189, 2 (2008), 476--491.

[127]

L. H. Lee, E. P. Chew, S. Teng, and D. Goldsman. 2010. Finding the non-dominated Pareto set for multi-objective simulation models. IIE Transactions 42 (2010), 656--674.

[128]

L. H. Lee, E. P. Chew, S. Teng, and J. Li. 2009. Application of evolutionary algorithms for solving multi-objective simulation optimization problems. In Multi-objective Memetic Algorithms, C.-K. Goh, Y.-S. Ong, and K. C. Tan (Eds.). Springer, Berlin, 91--110.

[129]

L. H. Lee, C. U. Lee, and Y. P. Tan. 2007. A multi-objective genetic algorithm for robust flight scheduling using simulation. European Journal of Operational Research 177, 3 (2007), 1948--1968.

[130]

H. Li, L. H. Lee, E. P. Chew, and P. Lendermann. 2015a. MO-COMPASS: A fast convergent search algorithm for multi-objective discrete optimization via simulation. IIE Transactions 47, 11 (2015), 1153--1169.

[131]

H. Li, Y. Li, G. Pedrielli, L. H. Lee, E. P. Chew, and C.-H. Chen. 2015b. Multi-objective multi-fidelity optimization with ordinal transformation and optimal sampling. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, T. M. K. Roeder, C. Macal, and M. D. Rosetti (Eds.). IEEE, Piscataway, NJ, 3737--3747.

Digital Library

[132]

H. Li, G. Pedrielli, L. H. Lee, and E. P. Chew. 2017. Enhancement of supply chain resilience through inter-echelon information sharing. Flexible Services and Manufacturing Journal 29, 2 (2017), 260--285.

[133]

H. Li, Y. Zhu, Y. Chen, G. Pedrielli, N. A. Pujowidianto, and Y. Chen. 2015c. The object-oriented discrete event simulation modeling: A case study on aircraft spare part management. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, T. M. K. Roeder, C. Macal, and M.D. Rosetti (Eds.). IEEE, Piscataway, NJ, 3514--3525.

Digital Library

[134]

J. Li, W. Liu, G. Pedrielli, L. H. Lee, and E. P. Chew. 2018. Optimal computing budget allocation to select the non-dominated systems -- A large deviations perspective. IEEE Transactions on Automatic Control (2018).

[135]

R.-C. Lin, M. Y. Sir, and K. S. Pasupathy. 2013. Multi-objective simulation optimization using data envelopment analysis and genetic algorithm: Specific application to determining optimal resource levels in surgical services. Omega 41 (2013), 881--892.

[136]

W. Liu, S. Gao, and L. H. Lee. 2017. A multi-objective perspective on robust ranking and selection. In Proceedings of the 2017 Winter Simulation Conference, W. K. V. Chan, A. D’Ambrogio, G. Zacharewicz, N. Mustafee, G. Wainer, and E. Page (Eds.). IEEE, Piscataway, NJ, 2116--2127.

Digital Library

[137]

S. Lucidi, M. Maurici, L. Paulon, F. Rinaldi, and M. Roma. 2016. A simulation-based multi-objective optimization approach for health care service management. IEEE Transactions on Automation Science and Engineering 13, 4 (2016), 1480--1491.

[138]

M. Mahdavi, T. Yang, and R. Jin. 2013. Stochastic convex optimization with multiple objectives. In Advances in Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Curran Associates, Red Hook, NY, 1115--1123. Retrieved from http://papers.nips.cc/paper/4942-stochastic-convex-optimization-with-multiple-objectives.pdf.

Digital Library

[139]

R. T. Marler and J. S. Arora. 2004. Survey of multi-objective optimization methods for engineering. Structural and Multidisciplinary Optimization 26 (2004), 369--395.

[140]

V. Mattila and K. Virtanen. 2014. Maintenance scheduling of a fleet of fighter aircraft through multi-objective simulation-optimization. Simulation: Transactions of the Society for Modeling and Simulation International 90, 9 (2014), 1023--1040.

Digital Library

[141]

V. Mattila and K. Virtanen. 2015. Ranking and selection for multiple performance measures using incomplete preference information. European Journal of Operational Research 242 (2015), 568--579.

[142]

J. R. W. Merrick, D. Morrice, and J. C. Butler. 2015. Using multiattribute utility theory to avoid bad outcomes by focusing on the best systems in ranking and selection. Journal of Simulation 9, 3 (2015), 238--248.

[143]

A. Messac, A. Ismail-Yahaya, and C. A. Mattson. 2003. The normalized normal constraint method for generating the Pareto frontier. Structural and Multidisciplinary Optimization 25, 2 (2003), 86--98.

[144]

K. Miettinen. 1999. Nonlinear Multiobjective Optimization. Kluwer Academic Publishers, Boston.

[145]

B. L. Nelson. 2010. Optimization via simulation over discrete decision variables. In TutORials in Operations Research, J. J. Hasenbein, P. Gray, and H. J. Greenberg (Eds.). INFORMS, Catonsville, MD, Chapter 9, 193--207.

[146]

A. Nguyen, S. Reiter, and P. Rigo. 2014. A review on simulation-based optimization methods applied to building performance analysis. Applied Energy 113 (2014), 1043--1058. Retrieved from

[147]

S. Ólafsson. 2006. Metaheuristics. In Simulation, S. G. Henderson and B. L. Nelson (Eds.). Elsevier, Amsterdam, The Netherlands, 633--654.

[148]

L. Pang, F. Meng, and J. Wang. 2018. Asymptotic convergence of stationary points of stochastic multiobjective programs with parametric variational inequality constraint via SAA approach. Journal of Industrial and Management Optimization (2018).

[149]

A. Pascoletti and P. Serafini. 1984. Scalarizing vector optimization problems. Journal of Optimization Theory and Applications 42, 4 (1984), 499--524.

[150]

R. Pasupathy and S. Ghosh. 2013. Simulation optimization: A concise overview and implementation guide. In TutORials in Operations Research, H. Topaloglu (Ed.). INFORMS, Catonsville, MD, Chapter 7, 122--150.

[151]

R. Pasupathy, P. W. Glynn, S. Ghosh, and F. Hashemi. 2018. On sampling rates in simulation-based recursions. SIAM Journal on Optimization 28, 1 (2018), 45--73.

[152]

R. Pasupathy and S. G. Henderson. 2006. A testbed of simulation-optimization problems. In Proceedings of the 2006 Winter Simulation Conference, L. F. Perrone, F. P. Wieland, J. Liu, B. G. Lawson, D. M. Nicol, and R. M. Fujimoto (Eds.). IEEE, Piscataway, NJ, 255--263.

Digital Library

[153]

R. Pasupathy and S. G. Henderson. 2011. SimOpt: A library of simulation optimization problems. In Proceedings of the 2011 Winter Simulation Conference, S. Jain, R. R. Creasey, J. Himmelspach, K. P. White, and M. Fu (Eds.). IEEE, Piscataway, NJ, 4075--4085.

Digital Library

[154]

R. Pasupathy, S. R. Hunter, N. A. Pujowidianto, L. H. Lee, and C.-H. Chen. 2015. Stochastically constrained ranking and selection via SCORE. ACM Transactions on Modeling and Computer Simulation 25, 1, Article 1 (January 2015), 26 pages.

Digital Library

[155]

R. Pasupathy and S. Kim. 2011. The stochastic root-finding problem: Overview, solutions, and open questions. ACM Transactions on Modeling and Computer Simulation 21, 3, Article 19 (2011), 23 pages.

Digital Library

[156]

R. Pasupathy and B. W. Schmeiser. 2010. Root finding via DARTS: Dynamic adaptive random target shooting. In Proceedings of the 2010 Winter Simulation Conference, B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yücesan (Eds.). IEEE, Piscataway, NJ, 1255--1262. Retrieved from http://www.informs-sim.org/wsc10papers/115.pdf.

Digital Library

[157]

F. Poirion, Q. Mercier, and J.-A. Désidéri. 2017. Descent algorithm for nonsmooth stochastic multiobjective optimization. Computational Optimization and Applications 68, 2 (November 2017 2017), 317--331.

Digital Library

[158]

O. Prakash, K. Srinivasan, and K. P. Sudheer. 2015. Adaptive multi-objective simulation-optimization framework for dynamic flood control operation in a river-reservoir system. Hydrology Research 46, 6 (2015), 893--911.

[159]

P. Reverdy. 2016. Gaussian multi-armed bandit problems with multiple objectives. In American Control Conference (ACC’16). IEEE, Piscataway, NJ, 5263--5269.

[160]

S. L. Rosen, C. M. Harmonosky, and M. T. Traband. 2007. A simulation optimization method that considers uncertainty and multiple performance measures. European Journal of Operational Research 181, 1 (August 2007), 315--330.

[161]

S. L. Rosen, C. M. Harmonosky, and M. T. Traband. 2008. Optimization of systems with multiple performance measures via simulation: Survey and recommendations. Computers 8 Industrial Engineering 54, 2 (March 2008), 327--339.

Digital Library

[162]

R. E. Rosenthal. 1985. Principles of multiobjective optimization. Decision Sciences 16 (1985), 133--152.

[163]

A. Ruszczyński and A. Shapiro (Eds.). 2003. Stochastic Programming. Elsevier, New York.

[164]

J. Ryu and S. Kim. 2014. A derivative-free trust-region method for biobjective optimization. SIAM Journal on Optimization 24, 1 (2014), 334--362.

[165]

S. Sayin. 2000. Measuring the quality of discrete representations of efficient sets in multiple objective mathematical programming. Mathematical Programming, Series A 87 (2000), 543--560.

[166]

B. Schmeiser. 2008. A practitioner, a vender, and a researcher walk into a bar: Trying to explain what researchers do. In Proceedings of the 2008 Winter Simulation Conference, S. J. Mason, R. R. Hill, L. Mönch, O. Rose, T. Jefferson, and J. W. Fowler (Eds.). IEEE, Piscataway, NJ, 2--9.

Digital Library

[167]

A. Shapiro. 2003. Monte Carlo sampling methods. In Stochastic Programming, A. Ruszczyński and A. Shaprio (Eds.). Elsevier, Amsterdam, The Netherlands, 353--425.

[168]

A. Shapiro, D. Dentcheva, and A. Ruszczyński. 2009. Lectures on Stochastic Programming: Modeling and Theory. Society for Industrial and Applied Mathematics, Philadelphia, PA.

Digital Library

[169]

S. Shashaani, F. S. Hashemi, and R. Pasupathy. 2018. ASTRO-DF: A class of adaptive sampling trust-region algorithms for derivative-free simulation optimization. SIAM Journal on Optimization 28, 4 (2018), 3145--3176. Retrieved from https://arxiv.org/pdf/1610.06506v1.pdf.

[170]

F. Siegmund, A. H. C. Ng, and K. Deb. 2016. Hybrid dynamic resampling algorithms for evolutionary multi-objective optimization of invariant-noise problems. In Applications of Evolutionary Computation: 19th European Conference, EvoApplications, Part II, G. Squillero and P. Burelli et al. (Eds.). Springer, New York, 311--216.

[171]

A. Singh and B. S. Minsker. 2008. Uncertainty-based multiobjective optimization of groundwater remediation design. Water Resources Research 44, Article W02404 (2008), 24 pages.

[172]

A. Singhee and P. Castalino. 2010. Pareto sampling: Choosing the right weights by derivative pursuit. In Proceedings of the 47th Design Automation Conference. ACM, New York, 913--916.

Digital Library

[173]

J. Song, Y. Qiu, and Z. Liu. 2016. Integrating optimal simulation budget allocation and genetic algorithm to find the approximate Pareto patient flow distribution. IEEE Transactions on Automation Science and Engineering 13, 1 (2016), 149--159.

[174]

W. Stadler. 1979. A survey of multicriteria optimization or the vector maximization problem, part I: 1776--1960. Journal of Optimization Theory and Applications 29, 1 (September 1979), 1–52.

Digital Library

[175]

I. Steponavičė, S. Ruuska, and K. Miettinen. 2014. A solution process for simulation-based multiobjective design optimization with an application in the paper industry. Computer-Aided Design 47 (2014), 45--58.

Digital Library

[176]

K. Subramanyan, U. Diwekar, and S. E. Zitney. 2011. Stochastic modeling and multi-objective optimization for the APECS system. Computers and Chemical Engineering 35 (2011), 2667--2679.

[177]

E. Tekin and I. Sabuncuoglu. 2004. Simulation optimization: A comprehensive review on theory and applications. IIE Transactions 36, 11 (2004), 1067--1081.

[178]

S. Teng, L. H. Lee, and E. P. Chew. 2010. Integration of indifference-zone with multi-objective computing budget allocation. European Journal of Operational Research 203, 2 (2010), 419--429.

[179]

D. Vanderpooten, L. Weerasena, and M. M. Wiecek. 2017. Covers and approximations in multiobjective optimization. Journal of Global Optimizaiton 67, 3 (March 2017), 601--619.

Digital Library

[180]

M. G. Villarreal-Marroquín, J. M. Castro, and M. Cabrera-Ríos. 2011. A multicriteria simulation optimization method for injection molding. In Proceedings of the 2011 Winter Simulation Conference, S. Jain, R. R. Creasey, J. Himmelspach, K. P. White, and M. Fu (Eds.). IEEE, Piscataway, NJ, 2395--2407.

Digital Library

[181]

M. G. Villarreal-Marroquín, J. D. Svenson, F. Sun, T. J. Santner, A. Dean, and J. M. Castro. 2013. A comparison of two metamodel-based methodologies for multiple criteria simulation optimization using an injection molding case study. Journal of Polymer Engineering 33, 3 (2013), 193--209.

[182]

H. Wang. 2013. Zigzag search for continuous multiobjective optimization. INFORMS Journal on Computing 25, 4 (2013), 654--665.

[183]

H. Wang. 2017. Multi-objective retrospective optimization using stochastic zigzag search. European Journal of Operational Research 263, 3 (December 2017), 946--960.

[184]

H. Wang, R. Pasupathy, and B. W. Schmeiser. 2013. Integer-ordered simulation optimization using R-SPLINE: Retrospective search using piecewise-linear interpolation and neighborhood enumeration. ACM Transactions on Modeling and Computer Simulation 23, 3, Article 17 (2013), 24 pages.

Digital Library

[185]

W. Wang and H. Wan. 2017. Sequential probability ratio test for multiple-objective ranking and selection. In Proceedings of the 2017 Winter Simulation Conference, W. K. V. Chan, A. D’Ambrogio, G. Zacharewicz, N. Mustafee, G. Wainer, and E. Page (Eds.). IEEE, Piscataway, NJ, 1998--2009.

Digital Library

[186]

Y. Wang, L. H. Lee, E. P. Chew, S. S. W. Lam, S. K. Low, M. E. H. Ong, and H. Li. 2015. Multi-objective optimization for a hospital inpatient flow process via discrete event simulation. In Proceedings of the 2015 Winter Simulation Conference, L. Yilmaz, W. K. V. Chan, T. M. K. Roeder, C. Macal, and M. D. Rosetti (Eds.). IEEE, Piscataway, NJ, 3622--3631.

Digital Library

[187]

D. J. White. 1990. A bibliography on the applications of mathematical programming multiple-objective methods. Journal of the Operational Research Society 41, 8 (1990), 669--691.

[188]

M. M. Wiecek, M. Ehrgott, and A. Engau. 2016. Continuous multiobjective programming. In Multiple Criteria Decision Analysis: State of the Art Surveys, S. Greco, M. Ehrgott, and J. R. Figueira (Eds.). International Series in Operations Research 8 Management Science, Vol. 233. Springer New York, New York, 739--815.

[189]

J. Wu and S. Azarm. 2001. Metrics for quality assessment of a multiobjective design optimization solution set. Transactions of the ASME 123 (2001), 18--25.

[190]

J. Xu, E. Huang, C.-H. Chen, and L. H. Lee. 2015. Simulation optimization: A review and exploration in the new era of cloud computing and big data. Asia-Pacific Journal of Operational Research 32, 3, Article 1550019 (2015), 34 pages.

[191]

J. Xu, B. L. Nelson, and L. J. Hong. 2010. Industrial strength COMPASS: A comprehensive algorithm and software for optimization via simulation. ACM Transactions on Modeling and Computer Simulation 20 (2010), 1--29.

Digital Library

[192]

S. Q. Yahyaa and M. M. Drugan. 2015. Correlated Gaussian multi-objective multi-armed bandit across arms algorithm. In IEEE Symposium series on Computational Intelligence. IEEE, Piscataway, NJ, 593--600.

[193]

S. Q. Yahyaa, M. M. Drugan, and B. Manderick. 2014a. Knowledge gradient for multi-objective multi-armed bandit algorithms. In 2014 International Conference on Agents and Artificial Intelligence (ICAART’14). SCITEPRESS, Setubal, Portugal, 74--83.

Digital Library

[194]

S. Q. Yahyaa, M. M. Drugan, and B. Manderick. 2014b. Multivariate normal distribution based multi-armed bandit Pareto algorithm. In The 7th European Conference on Machine Learning and Principles and Practice of Knowledge and Discovery in Databases (EMCL/PKDD’14), PhD Track. Springer, New York, 139--148.

[195]

S. Q. Yahyaa, M. M. Drugan, and B. Manderick. 2014c. The scalarized multi-objective multi-armed bandit problem: An empirical study of its exploration vs. exploitation tradeoff. In 2014 International Joint Conference on Neural Networks (IJCNN’14). IEEE, Piscataway, NJ, 2290--2297.

[196]

S. Q. Yahyaa, M. M. Drugan, and B. Manderick. 2015. Thompson sampling in the adaptive linear scalarized multi objective multi armed bandit. In Proceedings of the International Conference on Agents and Artificial Intelligence (ICAART’15), Vol. 2. SCITEPRESS, Portugal, 55--65.

Digital Library

[197]

H. Yildiz and S. Suri. 2012. On Klee’s measure problem for grounded boxes. In Proceedings of the 28th Annual Symposium on Computational Geometry. ACM, New York, 111--1120.

Digital Library

[198]

M. Yoon and J. Bekker. 2017. Single- and multi-objective ranking and selection procedures in simulation: A historical review. South African Journal of Industrial Engineering 28, 2 (2017), 37--45.

[199]

H. Zhang. 2008. Multi-objective simulation-optimization for earthmoving operations. Automation in Construction 18 (2008), 79--86.

[200]

J. Zhang, Y. Ma, T. Yang, and L. Liu. 2017. Estimation of the Pareto front in stochastic simulation through stochastic Kriging. Simulation Modelling Practice and Theory 79 (2017), 69--86.

[201]

A. Zhou, B. Qu, H. Li, S. Zhao, P. N. Suganthan, and Q. Zhang. 2011. Multiobjective evolutionary algorithms: A survey of the state of the art. Swarm and Evolutionary Computation 1 (2011), 32--49.

[202]

C. Zhou, H. Li, B. K. Lee, and Z. Qiu. 2018. A simulation-based vessel-truck coordination strategy for lighterage terminals. Transportation Research Part C: Emerging Technologies 95 (October 2018), 149--164.

[203]

E. Zitzler. 1999. Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. Ph.D. Dissertation. Swiss Federal Institute of Technology Zurich.

[204]

E. Zitzler, K. Deb, and L. Thiele. 2000. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary Computation 8, 2 (2000), 173--195.

Digital Library

[205]

E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and V. Grunert da Fonseca. 2003. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions on Evolutionary Computation 7, 2 (2003), 117--132.

Digital Library

Cited By

Sabbatella APonti ACandelieri AArchetti F(2024)Bayesian Optimization Using Simulation-Based Multiple Information Sources over Combinatorial StructuresMachine Learning and Knowledge Extraction10.3390/make60401106:4(2232-2247)Online publication date: 5-Oct-2024
https://doi.org/10.3390/make6040110
Uhrmacher AFrazier PHähnle RKlügl FLorig FLudäscher BNenzi LRuiz-Martin CRumpe BSzabo CWainer GWilsdorf P(2024)Context, Composition, Automation, and Communication: The C2AC Roadmap for Modeling and SimulationACM Transactions on Modeling and Computer Simulation10.1145/367322634:4(1-51)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3673226
Zhou YAndradóttir SKim SPark C(2024)Pruning inferior systems using subjective constraints with sequentially added thresholdsSequential Analysis10.1080/07474946.2024.234846443:2(248-274)Online publication date: 5-Jun-2024
https://doi.org/10.1080/07474946.2024.2348464
Show More Cited By

Index Terms

An Introduction to Multiobjective Simulation Optimization

Recommendations

Multiobjective simulation optimization using enhanced evolutionary algorithm approaches
Evolutionary multiobjective optimization
GECCO '11: Proceedings of the 13th annual conference companion on Genetic and evolutionary computation

Many optimization problems are multiobjective in nature in the sense that multiple, conflicting criteria need to be optimized simultaneously. Due to the conflict between objectives, usually, no single optimal solution exists. Instead, the optimum ...
Tutorial on Evolutionary Multiobjective Optimization
GECCO Companion '15: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation

Many optimization problems are multiobjective in nature in the sense that multiple, conflicting criteria need to be optimized simultaneously. Due to the conflict between objectives, usually, no single optimal solution exists. Instead, the optimum ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 29, Issue 1

January 2019

149 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/3309768

Editor:
Adelinde M. Uhrmacher
University of Rostock, Germany

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 January 2019

Accepted: 01 November 2018

Revised: 01 August 2018

Received: 01 March 2017

Published in TOMACS Volume 29, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation (NSF)
Agency for Healthcare Research and Quality (AHRQ)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

44
Total Citations
View Citations
2,545
Total Downloads

Downloads (Last 12 months)574
Downloads (Last 6 weeks)78

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sabbatella APonti ACandelieri AArchetti F(2024)Bayesian Optimization Using Simulation-Based Multiple Information Sources over Combinatorial StructuresMachine Learning and Knowledge Extraction10.3390/make60401106:4(2232-2247)Online publication date: 5-Oct-2024
https://doi.org/10.3390/make6040110
Uhrmacher AFrazier PHähnle RKlügl FLorig FLudäscher BNenzi LRuiz-Martin CRumpe BSzabo CWainer GWilsdorf P(2024)Context, Composition, Automation, and Communication: The C2AC Roadmap for Modeling and SimulationACM Transactions on Modeling and Computer Simulation10.1145/367322634:4(1-51)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3673226
Zhou YAndradóttir SKim SPark C(2024)Pruning inferior systems using subjective constraints with sequentially added thresholdsSequential Analysis10.1080/07474946.2024.234846443:2(248-274)Online publication date: 5-Jun-2024
https://doi.org/10.1080/07474946.2024.2348464
Zhao YChen WYang X(2024)Adaptive Sampling Stochastic Multigradient Algorithm for Stochastic Multiobjective OptimizationJournal of Optimization Theory and Applications10.1007/s10957-023-02334-w200:1(215-241)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10957-023-02334-w
Hunter SOndes B(2023)Properties of Several Performance Indicators for Global Multi-Objective Simulation OptimizationProceedings of the Winter Simulation Conference10.5555/3643142.3643441(3577-3588)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3643142.3643441
Xu JZheng Z(2023)Gradient-Based Simulation Optimization Algorithms via Multi-Resolution System ApproximationsINFORMS Journal on Computing10.1287/ijoc.2023.127935:3(633-651)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1287/ijoc.2023.1279
Karl FPielok TMoosbauer JPfisterer FCoors SBinder MSchneider LThomas JRichter JLang MGarrido-Merchán EBranke JBischl B(2023)Multi-Objective Hyperparameter Optimization in Machine Learning – An OverviewACM Transactions on Evolutionary Learning and Optimization10.1145/3610536Online publication date: 5-Sep-2023
https://dl.acm.org/doi/10.1145/3610536
Eckman DHenderson SShashaani S(2023)Stochastic Constraints: How Feasible Is Feasible?2023 Winter Simulation Conference (WSC)10.1109/WSC60868.2023.10408734(3589-3600)Online publication date: 10-Dec-2023
https://doi.org/10.1109/WSC60868.2023.10408734
Hunter SOndes B(2023)Properties of Several Performance Indicators for Global Multi-Objective Simulation Optimization2023 Winter Simulation Conference (WSC)10.1109/WSC60868.2023.10408213(3577-3588)Online publication date: 10-Dec-2023
https://doi.org/10.1109/WSC60868.2023.10408213
Wang LDemeulemeester E(2023)Simulation optimization in healthcare resource planning: A literature reviewIISE Transactions10.1080/24725854.2022.2147606(1-23)Online publication date: 6-Jan-2023
https://doi.org/10.1080/24725854.2022.2147606
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents