Cited By
View all- Centeno M(1996)An introduction to simulation modelingProceedings of the 28th conference on Winter simulation10.1145/256562.256564(15-22)Online publication date: 8-Nov-1996
Advanced methods for simulation output analysis
Pages 101 - 109
Abstract
This paper reviews statistical methods for analyzing output data from computer simulations of single systems. In particular, it focuses on the problems of choosing initial conditions and estimating steady-state system parameters. The estimation techniques include the replication/deletion approach, the regenerative method, the batch means method, the standardized time series method, the autoregressive method, and the spectral estimation method.
References
[1]
Alexopoulos, C. 1993. Distribution-free confidence intervals for conditionM probabilities and ratios of expectations. Management Science 40(12):1748- 1763.
[2]
Billingsley, P. 1968. Convergence of probability mensures, Wiley, New York.
[3]
Bratley, P., B. L. Fox, and L. E. Schrage. 1987. A guide to simulation, 2d Ed. Springer-Verlag, New York, New York.
[4]
Chance, F., and L. W. Schruben. 1992. Establishing a truncation point in simulation output. Technical Report, School of Operations Research and Industrial Engineering, Cornell University, Ithaca, New York.
[5]
Charnes, J. M. 1989. Statistical analysis of multivariate discrete-event simulation output. Ph.D. Thesis, Department of Operations and Management Science, University of Minnesota, Minneapolis, Minnesota.
[6]
Charnes, J. M. 1990. Power comparisons for the multivariate batch-means method. In Proceedings of the 1990 Winter Simulation Conference, 281-287. IEEE, Piscataway, New Jersey.
[7]
Charnes, J. M. 1991. Multivariate simulation output analysis. In Proceedings of the 1991 Winter Simulation Conference, 187-193. IEEE, Piscataway, New Jersey.
[8]
Chen, R. D., and A. F. Seila. 1987. Multivariate inference in stationary simulation using batch means. In Proceedings of the 1987 Winter Simulation Conference, 302-304. IEEE, Piscataway, New Jersey.
[9]
Chien, C.-H. 1989. Small sample theory for steady state confidence intervals. Technical Report No. 37, Department of Operations Research, Stanford University, Palo Alto, California.
[10]
Chow, Y. S., and H. Robbins. t965. On the asymptotic theory of fixed-width sequential confidence intervals for the mean. Annals of Mathematical Statistics 36:457-462.
[11]
Conway, R. W. 1963. Some tactical problems in digital simulation. Management Science 10:47-61.
[12]
Crane, M. A., and D. L. Iglehart. 1974a. Simulating stable stochastic systems, I: General multiserver queues. Journal of the ACM 21:103-113.
[13]
Crane, M. A., and D. L. Iglehart. 1974b. Simulating stable stochastic systems, II: Markov chains. Journal of the ACM 21:114-123.
[14]
Crane, M. A., and D. L. Iglehart. 1975. Simulating stable stochastic systems IIi: Regenerative processes and discrete-event simulations. Operations Research 23:33-45.
[15]
Crane, M. A., and A. J. Lemoine. 1977. An introduction to the regenerative method for simulation analysis, Springer-Verlag, New York, New York.
[16]
Damerdji, H. 1991. Strong consistency and other properties of the spectral variance estimator. Management Science 37:1424-1440.
[17]
Damerdji, tt. 1994a. Strong consistency of the variance estimator in steady-state simulation output analysis. Mathematics of Operations Research 19:494-512.
[18]
Damerdji, H. 1994b. On the batch means and area variance estimators. In Proceedings of the 1994 Winter Simulation Conference, 340-344. IEEE, Piscataway, New Jersey.
[19]
Fishman, G. S. 1972. Bias considerations in simulation experiments. Operations Research 20:785- 790.
[20]
Fishman, G. S. 1973. Statistical analysis for queueing simulations. Management Science 20:363-369.
[21]
Fishman, G. S. 1974. Estimation of multiserver queueing simulations. Operations Research 22:72- 78.
[22]
Fishman, G. S. 1978a. Grouping observations in digital simulation. Management Science 24:510-521.
[23]
Fishman, G. S. 1978b. Principles of discrete event simulation, John Wiley, New York, New York.
[24]
Fishman, G. S., and L. S. Yarberry. 1993. An implementation of the batch means method. Technical Report UNC/OR/TR/93-1, Department of Operations Research, University of North Carolina, Chapel Hill, North Carolina.
[25]
Gafarian, A.V., C. J. Ancker, and F. Morisaku. 1978. Evaluation of commonly used rules for detecting steady-state in computer simulation. Naval Research Logistics Quarterly 25:511-529.
[26]
Glynn, P. W., and D. L. Iglehart. 1990. Simulation analysis using standardized time series. Mathematics of Operations Research 15:1-16.
[27]
Goldsman, D., M. Meketon, and L. W. Schruben. 1990. Properties of standardized time series weighted area variance estimators. Management Science 36"602-612.
[28]
Goldsman, D., and L. W. Schruben. 1984. Asymptotic properties of some confidence interval estimators for simulation output. Management Science 30:1217-1225.
[29]
Goldsman, D., and L. W. Schruben. 1990. New confidence interval estimators using standardized time series. Management Science 36:393-397.
[30]
Goldsman, D., L. W. Schruben, and J. J. Swain. 1993. Tests for transient means in simulated time series. Technical Report, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia.
[31]
Heidelberger, P., and P. A. W. Lewis. 1984. Quantile estimation in dependent sequences. Operations Research 32:185-209.
[32]
Heidelberger, P., and P. D. Welch. 1981a. A spectral method for confidence interval generation and run length control in simulations. Communications of the A CM 24:233-245.
[33]
Heidelberger, P., and P. D. Welch. 1981b. Adaptive spectral methods for simulation output analysis. IBM Journal of Research and Development 25:860-876.
[34]
Heidelberger, P., and P. D. Welch. 1983. Simulation run length control in the presence of an initial transient. Operations Research 31:1109-1144.
[35]
Iglehart, D. L. 1975. Simulating stable stochastic systems, V: Comparison of ratio estimators. Naval Research Logistics Quarterly 22:553-565.
[36]
Iglehart, D. L. 1976. Simulating stable ~tocha~tic systems, VI: Quantile estimation. Journal of the A CM 23:347-360.
[37]
Iglehart, D. L. 1978. The regenerative method for simulation analysis. In Current Trends in Programming Methodology, Vol. III, eds. K. M. Chandy, and K. M. Yeh, 52-71. Prentice-Hall, Englewood Cliffs, New Jersey.
[38]
Kelton, W. D. 1989. Random initialization methods in simulation. IIE Transactions 21:355-367.
[39]
Kelton, W. D., and A. M. Law. 1983. A new approach for dealing with the startup problem in discrete event simulation. Naval Research Logistics Quarterly 30:641-658.
[40]
Kleijnen, J. P. C. 1974. Statistical techniques in simulation, part I, Marcel Dekker, New York, New York.
[41]
Kleijnen, j. P. C. 1975. Statistical techniques in simulation, part II, Marcel Dekker, New York, New York.
[42]
Law, A. M. 1980. Statistical analysis of the output data from terminating simulations. Naval Research Logistics Quarterly 27:131-143.
[43]
Law, A. M., and J. S. Carson. 1979. A sequential procedure for determining the length of a steady-state simulation. Operations Research 27:1011-1025.
[44]
Law, A. M., and W. D. Kelton. 1984. Confidence intervals for steady-state simulations, I: A survey of fixed sample size procedures. Operations Research 32:1221-1239.
[45]
Law, A. M., and W. D. Kelton. 1991. Simulation modeling and analysis, 2d Ed. McGraw-Hill, New York.
[46]
Meketon, M. S., and B. W. Schmeiser. 1984. Overlapping batch means: Something for nothing? In Proceedings of the 1984 Win ter Simulation Conference, 227-230. IEEE, Piscataway, New Jersey.
[47]
Moore, L. W. 1980. Quantile estimation in regenerative processes. Ph.D. Thesis, Curriculum in Operations Research and Systems Analysis, University of North Carolina, Chapel Hill, North Carolina.
[48]
Nadas, A. 1969. An extension of the theorem of Chow and Robbins on sequential confidence intervals for the mean. Annals of Mathematical Statistics 40:667-671.
[49]
Ockerman, D. H. 1995. Initialization bias tests for stationary stochastic processes based upon standardized time series techniques. Ph.D. Thesis, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia.
[50]
Sargent, R. G., K. Kang, and D. Goldsman. 1992. An investigation of finite-sample behavior of confidence interval estimators. Operations Research 40:898-913.
[51]
Schm~iger, B. W. 1982. Batch size effects in the analysis of simulation output. Operations Research 30:556-568.
[52]
Schmeiser, B. W., and W.-M. T. Song. 1993. Optimal mean-squared-error batch sizes. To appear in Management Science.
[53]
Schriber, T. j., and R. W. Andrews. 1984. An ARMA-based confidence interval for the analysis of simulation output. American J. Math. Management Sci. 4:345-373.
[54]
Schruben, L. W. 1981. Control of initialization bias in multivariate simulation response. Communications of the ACM 24:246-252.
[55]
Schruben, L. W. 1982. Detecting initialization bias in simulation output. Operations Research 30:569- 590.
[56]
Schruben, L. W. 1983. Confidence interval estimation using standardized time series. Operations Research 31:1090-1108.
[57]
Schruben, L. W., H. Singh, and L. Tierney. 1983. Optimal tests for initialization bias in simulation output. Operations Research 31:1167-1178.
[58]
Seila, A. F. 1982a. A batching approach to quantile estimation in regenerative simulations. Management Science 28:573-581.
[59]
Seila, A. F. 1982b. Percentile estimation in discrete event simulation. Simulation 39:193-200.
[60]
Sherman, M. 1995. On batch means in the simulation and statistical communities. In Proceedings of the 1995 Winter Simulation Conference, this volume. IEEE, Piscataway, New Jersey.
[61]
Welch, P. D. 1981. On the problem of the initial transient in steady state simulations. Technical Report, IBM Watson Research Center, Yorktown Heights, New York.
[62]
Welch, P. D. 1983. The statistical analysis of simulation results. In The computer performance modeling handbook, ed. S. Lavenberg, 268-328, Academic Press, New York.
[63]
Welch, P.D. 1987. On the relationship between batch means, overlapping batch means and spectral estimation, In Proceedings of the 1987 Win- ~er Simulation Conference, 320-323. IEEE, Piscataway, New Jersey.
[64]
Wilson, J. R., and A. A. B. Pritsker. 1978a. A survey of research on the simulation startup problem. Simulation 31:55-58.
[65]
Wilson, J. R., and A. A. B. Pritsker. 1978b. Evaluation of startup policies in simulation experiments. Simulation 31:79-89.
Index Terms
- Advanced methods for simulation output analysis
Recommendations
Asymptotic results for batch-variance methods in simulation output analysis
Special issue: Selected papers from the 31st international conference on computers & industrial engineeringWe consider the batching methods for estimating the variance of the sample variance based on steady-state correlated data. Intensive research has been devoted to the problem of estimating the variance of the sample mean, but little to the sample ...
Jackknifed variance estimators for simulation output analysis
WSC '15: Proceedings of the 2015 Winter Simulation ConferenceWe develop new point estimators for the variance parameter of a steady-state simulation process. The estimators are based on jackknifed versions of nonoverlapping batch means, overlapping batch means, and standardized time series variance estimators. ...
Mean-Square Consistency of the Variance Estimator in Steady-State Simulation Output Analysis
In steady-state simulation output analysis, mean-square consistency of the process-variance estimator is important for a number of reasons. One way to construct an asymptotically valid confidence interval around a sample mean is via construction of a ...
Comments
Information & Contributors
Information
Published In
December 1995
1493 pages
ISBN:0780330188
- Chairmen:
- William R. Lilegdon,
- David Goldsman,
- Editors:
- Christos Alexopoulos,
- Keebom Kang
Sponsors
- IIE: Institute of Industrial Engineers
- IEEE-SMCS: Systems, Man & Cybernetics Society
- INFORMS/CS: Computer Science TC
- ASA: American Statistical Association
- ACM: Association for Computing Machinery
- SCS: Society for Computer Simulation
- SIGSIM: ACM Special Interest Group on Simulation and Modeling
- IEEE-CS: Computer Society
- NIST: National Institute of Standards & Technology
Publisher
IEEE Computer Society
United States
Publication History
Published: 01 December 1995
Check for updates
Qualifiers
- Article
Conference
Acceptance Rates
WSC '95 Paper Acceptance Rate 122 of 183 submissions, 67%;
Overall Acceptance Rate 3,413 of 5,075 submissions, 67%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 520Total Downloads
- Downloads (Last 12 months)28
- Downloads (Last 6 weeks)5
Reflects downloads up to 01 Nov 2024
Other Metrics
Citations
Cited By
View all- Centeno M(1996)An introduction to simulation modelingProceedings of the 28th conference on Winter simulation10.1145/256562.256564(15-22)Online publication date: 8-Nov-1996
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in