A tournament framework for the ranking and selection problem
December 2007
Pages 297 - 302
Abstract
A tournament can be broadly defined as a procedure that ranks agents, where they exhibit their performance in a noisy environment. By observing the agents' performance, the organizer computes their ranking and rewards them according to the revealed ranking. The organizer's challenge is therefore to determine the optimal tournament format that identifies the best agent in the most effective fashion. Tournaments thus provide a natural framework for ranking and selection (R&S) via simulation, which represents a set of approaches developed to complement the modeling flexibility of simulation with the efficiency of statistical techniques for effective decision making. In this paper, following the introduction of a general framework to represent various tournament formats and to assess their predictive power, we will report preliminary experimental results on the effectiveness of tournaments in identifying the best simulated system with the desired probability of correct selection in the presence of costs.
References
[1]
Appleton, D. R. 1995. May the best man win? The Statistician 44:529--538.
[2]
Bechofer R. E., T. J. Santner, and D. M. Goldsman. 1995. Design and analysis of experiments for statistical selection, screening, and multiple comparisons. New York, NY: John Wiley & Sons, Inc.
[3]
Berger, J. O. 1980. Statistical decision theory: foundations, concepts, and methods. New York, NY: Springer Verlag.
[4]
Berger, J. O., and J. Deely. 1994. A Bayesian approach to ranking and selection of related means with alternative to AOV methodology. Journal of American Statistics Association 83:364--373.
[5]
Bernardo, J. M., and A. F. M. Smith. 1994. Bayesian theory. New York, NY: Wiley.
[6]
Chen, C. H. 1995. An effective approach to smartly allocate computing budget for discrete event simulation. In Proceedings of the 34th IEEE Conference on Decision and Control, 2598--2605. Piscataway, NJ: IEEE.
[7]
Chen, C. H. 1996. A lower bound for the correct subset-selection probability and its application to discrete event system simulations. IEEE Transactions on Automatic Control 41:1227--1231.
[8]
Chen, C. H., H. C. Chen, and L. Dai. 1996. A gradient approach for smartly allocating computing budget for discrete event simulation. In Proceedings of the 1996 Winter Simulation Conference, ed. J. M. Charnes, D. M. Morrice, D. T. Brunner, and J. J. Swain, 398--405. Piscataway, NJ: IEEE.
[9]
Chen, H. C., C. H. Chen, L. Dai, and E. Yücesan. 1997. New development of optimal computing budget allocation for discrete event simulation. In Proceedings of the 1997 Winter Simulation Conference, ed. S. Andradottir, K. J. Healy, D. H. Withers, and B. L. Nelson, 334--341. Piscataway, NJ: IEEE.
[10]
Chick, S. E. 1997. Selecting the best system: a decision-theoretic approach. In Proceedings of the 1997 Winter Simulation Conference, ed. S. Andradottir, K. J. Healy, D. H. Withers, and B. L. Nelson, 326--333. Piscataway, NJ: IEEE.
[11]
Chick, S. E. and K. Inoue. 2001. New procedures to select the best simulated system using common random numbers. Management Science 47: 1133--1149.
[12]
Chiu, W. K., 1974. The ranking of means of normal populations for a generalized selection goal. Biometrika 61: 579--584.
[13]
Dudewicz, E. J. and S. R. Dalal. 1975. Allocation of observations in ranking and selection with unequal variances. Sankhya B37:28--78.
[14]
Goldsman, D., and B. L. Nelson. 1994. Ranking, selection, and multiple comparison in computer simulation. In Proceedings of the 1994 Winter Simulation Conference, ed. J. D. Tew, S. Manivannan, D. A. Sadowski, and A. F. Seila, 192--199. Piscataway, NJ: IEEE.
[15]
Goldsman, D. and B. L. Nelson. 1998. Comparing systems via simulation. In Handbook of Simulation, ed. J. Banks, 273--306. New York, NY: John Wiley and Sons.
[16]
Gupta, S. S. and J. O. Berger. 1988. Statistical decision theory and related topics IV. New York, NY: Springer Verlag.
[17]
Gupta, S. S. and K. J. Miescke. 1996. Bayesian look ahead one-stage sampling allocations for selection of the best population. Journal of Statistical Planning and Inference 54: 229--244.
[18]
Gupta, S. S. and S. Panchapakesan. 1979. Multiple decision procedures: theory and methodology of selecting and ranking populations. New York, NY: John Wiley.
[19]
Harary F. and L. Moser. 1966. The theory of round robin tournaments. American Mathematical Monthly 73: 231--246.
[20]
Hsu, J. C. 1996. Multiple comparisons theory and methods. London: Chapman & Hall.
[21]
Matejcik, F. J. and B. L. Nelson. 1993. Simultaneous ranking, selection and multiple comparisons for simulation. In Proceeding of the 1993 Winter Simulation Conference, ed. G. W. Evans, M. Mollaghasemi, E. C. Russell, and W. E. Biles, 386--391. Piscataway, NJ: IEEE.
[22]
Matejcik, F. J. and B. L. Nelson. 1995. Two-stage multiple comparisons with the best for computer simulation. Operations Research 43: 633--640.
[23]
Rinott, Y. 1978. On Two-stage selection procedures and related probability inequalities. Communications in Statistics A7: 799--811.
[24]
Rosen, S. 1986. Prizes and incentives in elimination tournaments. The American Economic Review 76: 701--715.
[25]
Ryvkin, D. 2005. The predictive power of composite noisy tournaments. Technical Report, Center for Economic Research and Graduate Education, Charles University and Academy of Sciences, Prague, Czech Republic.
[26]
Szymanski, S. 2003. The economic design of sporting contests: a review. Journal of Economic Literature 41: 1137--1187.
Recommendations
Tournament minors
We say a digraph G is a minor of a digraph H if G can be obtained from a subdigraph of H by repeatedly contracting a strongly-connected subdigraph to a vertex. Here, we show that the class of all tournaments is a well-quasi-order under minor ...
Making a tournament k-arc-strong by reversing or deorienting arcs
The 1st cologne-twente workshop on graphs and combinatorial optimization (CTW 2001)We prove that every tournament T = (V,A) on n ≥ 2k + 1 vertices can be made k-arc-strong by reversing no more than k(k + 1)/2 arcs. This is best possible as the transitive tournament needs this many arcs to be reversed. We show that the number of arcs ...
Another investigation on tournament selection: modelling and visualisation
GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computationTournament selection has been widely used and studied in evolutionary algorithms. To supplement the study of tournament selection, this paper provides several models describing the probabilities that a program of a particular rank is sampled and is ...
Comments
Information & Contributors
Information
Published In
Sponsors
- IIE: Institute of Industrial Engineers
- INFORMS-SIM: Institute for Operations Research and the Management Sciences: Simulation Society
- ASA: American Statistical Association
- IEEE/SMC: Institute of Electrical and Electronics Engineers: Systems, Man, and Cybernetics Society
- SIGSIM: ACM Special Interest Group on Simulation and Modeling
- NIST: National Institute of Standards and Technology
- (SCS): The Society for Modeling and Simulation International
Publisher
IEEE Press
Publication History
Published: 09 December 2007
Check for updates
Qualifiers
- Research-article
Conference
WSC07
Sponsor:
- IIE
- INFORMS-SIM
- ASA
- IEEE/SMC
- SIGSIM
- NIST
- (SCS)
Acceptance Rates
WSC '07 Paper Acceptance Rate 152 of 244 submissions, 62%;
Overall Acceptance Rate 3,413 of 5,075 submissions, 67%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 115Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in