Cited By
View all- Onggo BCurrie CPerdana TFauzi GAchmad AEndyana C(2021)Relief Food Supply Network Simulation2021 Winter Simulation Conference (WSC)10.1109/WSC52266.2021.9715527(1-12)Online publication date: 12-Dec-2021
A Practical Approach to Subset Selection for Multi-objective Optimization via Simulation
Article No.: 20, Pages 1 - 15
Abstract
We describe a practical two-stage algorithm, BootComp, for multi-objective optimization via simulation. Our algorithm finds a subset of good designs that a decision-maker can compare to identify the one that works best when considering all aspects of the system, including those that cannot be modeled. BootComp is designed to be straightforward to implement by a practitioner with basic statistical knowledge in a simulation package that does not support sequential ranking and selection. These requirements restrict us to a two-stage procedure that works with any distributions of the outputs and allows for the use of common random numbers. Comparisons with sequential ranking and selection methods suggest that it performs well, and we also demonstrate its use analyzing a real simulation aiming to determine the optimal ward configuration for a UK hospital.
References
[1]
S. Andradóttir and S.-H. Kim. 2010. Fully sequential procedures for comparing constrained systems via simulation. Nav. Res. Logist. 57 (2010), 403–421.
[2]
R. E. Bechhofer. 1954. A single-sample multiple decision procedure for ranking means of normal populations with known variances. Ann. Math. Statist. 25 (1954), 16–39.
[3]
J. M. Bekki, B. L. Nelson, and J. W. Fowler. 2010. Bootstrapping-based fixed-width confidence intervals for ranking and selection. In Proceedings of the Winter Simulation Conference, B. Johansson et al. (Eds.). IEEE, Piscataway, NJ, 1024–1033.
[4]
J. Branke, S. E. Chick, and C. Schmidt. 2007. Selecting a selection procedure. Manag. Sci. 53 (2007), 1916–1932.
[5]
C.-H. Chen. 1996. A lower bound for the correct subset-selection probability and its application to discrete event simulations.IEEE Trans. Automat. Control 41 (1996), 1227–1231.
[6]
C.-H. Chen, D. He, M. Fu, and L. H. Lee. 2008. Efficient simulation budget allocation for selecting an optimal subset. INFORMS J. Comput. 20 (2008), 579–595.
[7]
M. R. Chernick. 2007. Bootstrap Methods: A Guide for Practitioners and Researchers. John Wiley & Sons, Inc, NY.
[8]
S. E. Chick and K. Inoue. 2001a. New procedures to select the best simulated system using common random numbers. Manag. Sci. 47 (2001), 1133–1149.
[9]
S. E. Chick and K. Inoue. 2001b. New two-stage and sequential procedures for selecting the best simulated system. Oper. Res. 49 (2001), 732–743.
[10]
F. Chingcuanco and C. Osorio. 2013. A procedure to select the best subset among simulated systems using economic opportunity cost. In Proceedings of the Winter Simulation Conference, R. Pasupathy et al. (Eds.). IEEE, Piscataway,NJ, 452–462.
[11]
S. Gao and W. Chen. 2015. A note on the subset selection for simulation optimization. In Proceedings of the Winter Simulation Conference, L. Yilmaz et al. (Eds.). IEEE, Piscataway, NJ, 3768–3776.
[12]
S. Gao and W. Chen. 2016. A new budget allocation framework for selecting top simulated designs. IIE Trans. 48 (2016), 855–863.
[13]
S. S. Gupta. 1965. On some multiple decision (selection and ranking) rules. Technometrics 7 (1965), 225–245.
[14]
L. J. Hong, J. Luo, and B. L. Nelson. 2015. Chance constrained selection of the best. INFORMS J. Comput. 27 (2015), 317–334.
[15]
S. R. Hunter, E. A. Applegate, V. Arora, B. Chong, K. Cooper, O. Rincón-Guevara, and C. Vivas-Valencia. 2019. An introduction to multiobjective simulation optimization. ACM Trans. Model. Comput. Simul. 29, 1 (2019).
[16]
S. R. Hunter and R. Pasupathy. 2013. Optimal sampling laws for stochastically constrained simulation optimization on finite sets. INFORMS J. Comput. 25 (2013), 527–542.
[17]
S.-H. Kim and B. L. Nelson. 2006a. Selecting the best system. In Handbooks in Operations Research and Management Science: Simulation, Shane G. Henderson and Barry L. Nelson (Eds.). Vol. 13. Elsevier B.V., North Holland.
[18]
S.-H. Kim and B. L. Nelson. 2006b. Selecting the best system. In Elsevier Handbooks in Operations Research and Management Science: Simulation, S. G. Henderson and B. L. Nelson (Eds.). Elsevier, Waltham, MA.
[19]
L. W. Koenig and A. M. Law. 1985. A procedure for selecting a subset of size m containing the l best of k independent normal populations. Commun. Statist. - Simul. Computat. 14 (1985), 719–734.
[20]
A. M. Law and W. D. Kelton. 2000. Simulation Modeling & Analysis (3rd ed.). McGraw-Hill, Inc, New York.
[21]
L. H. Lee, E. K. Pen Chew, S. Teng, and D. Goldsman. 2010. Finding the non-dominated Pareto set for multi-objective simulation models. IIE Trans. 42 (2010), 656–674.
[22]
S. Lee and B. L. Nelson. 2014. Bootstrap ranking and selection revisited. In Proceedings of the Winter Simulation Conference, A. Tolk et al. (Eds.). IEEE, Piscataway, NJ, 3857–3868.
[23]
S. Lee and B. L. Nelson. 2015. Computational improvements in bootstrap ranking and selection procedures via multiple comparison with the best. In Proceedings of the Winter Simulation Conference, L. Yilmaz et al. (Eds.). IEEE, Piscataway, NJ, 3758–3767.
[24]
S. Lee and B. L. Nelson. 2016. General-purpose ranking and selection for computer simulation. IIE Trans. 48 (2016), 555–564.
[25]
T. Monks and C. S. M. Currie. 2018. Practical considerations in selecting the best set of simulated systems. In Proceedings of the Winter Simulation Conference, M. Rabe, A. A. Juan, N. Mustafee, A. Skoogh, S. Jain, and B. Johansson (Eds.). IEEE, Piscataway, NJ, 2191–2200.
[26]
T. Monks and C. S. M. Currie. 2020. CLAHRCWessex/BootComp: v1.0.1.
[27]
T. Monks and C. S. M Currie. 2021. CLAHRCWessex/subset-selection-problem: v1.0.1.
[28]
B. L. Nelson and F. J. Matejcik. 1995. Using common random numbers for indifference-zone selection and multiple comparisons in simulation. Manag. Sci. 41 (1995), 1935–1945.
[29]
R. Pasupathy, S. R. Hunter, N. A. Pujowidianto, L. H. Lee, and C.-H. Chen. 2014. Stochastically constrained ranking and selection via SCORE. ACM Trans. Model. Comput. Simul. 25 (2014), 1–26.
[30]
L. Pei, S. Hunter, and B. L. Nelson. 2018. A new framework for parallel ranking & selection using an adaptive standard. In Proceedings of the Winter Simulation Conference, M. Rabe, A. A. Juan, N. Mustafee, A. Skoogh, S. Jain, and B. Johansson (Eds.). IEEE, Piscataway, NJ, 2201–2212.
[31]
M. L. Penn and T. Monks. 2018. Generic Ward Configuration Simulation Model.
[32]
M. L. Penn, T. Monks, A. Kazmierska, and M. Alkoheji. 2018. Designing and redeveloping generic models in healthcare. In Proceedings of the OR Society Simulation Workshop, A. Anagnostou et al. (Eds.). OR Society, Stratford-Upon-Avon, UK.
[33]
D. W. Sullivan and J. R. Wilson. 1989. Restricted subset selection procedures for simulation. Oper. Res. 37(1) (1989), 52–71.
[34]
J. R. Swisher and S. H. Jacobsen. 2002. Evaluating the design of a family practice healthcare clinic using discrete-event simulation. Health Care Manag. Sci. 5 (2002), 75–88.
[35]
Y. Wang, L. Luangkesorn, and L. J. Shuman. 2011. Best-subset selection procedure. In Proceedings of the Winter Simulation Conference, S. Jain et al. (Eds.). IEEE, Piscataway, NJ, 4315–4323.
Index Terms
- A Practical Approach to Subset Selection for Multi-objective Optimization via Simulation
Recommendations
Replicated Computational Results (RCR) Report for“A Practical Approach to Subset Selection for Multi-Objective Optimization via Simulation”
In “A Practical Approach to Subset Selection for Multi-Objective Optimization via Simulation,” Currie and Monks propose an algorithm for multi-objective simulation-based optimization. In contrast to sequential ranking and selection schemes, their ...
A multi-objective selection procedure of determining a Pareto set
Ranking and selection (R&S) procedures have been widely studied and applied in determining the required sample size (i.e., the number of replications or batches) for selecting the best system or a subset containing the best system from a set of k ...
Efficient simulation budget allocation for subset selection using regression metamodels
AbstractThis research considers the ranking and selection (R&S) problem of selecting the optimal subset from a finite set of alternative designs. Given the total simulation budget constraint, we aim to maximize the probability of correctly ...
Comments
Information & Contributors
Information
Published In
October 2021
159 pages
Copyright © 2021 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 August 2021
Accepted: 01 March 2021
Received: 01 September 2020
Revised: 01 June 2020
Published in TOMACS Volume 31, Issue 4
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Funding Sources
- National Institute for Health Research Applied Research Collaboration Wessex
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 782Total Downloads
- Downloads (Last 12 months)213
- Downloads (Last 6 weeks)24
Reflects downloads up to 13 Sep 2024
Other Metrics
Citations
Cited By
View all- Onggo BCurrie CPerdana TFauzi GAchmad AEndyana C(2021)Relief Food Supply Network Simulation2021 Winter Simulation Conference (WSC)10.1109/WSC52266.2021.9715527(1-12)Online publication date: 12-Dec-2021
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in