Cited By
View all- Tafazzoli AWilson JDunkin AIngalls R(2009)N-SkartWinter Simulation Conference10.5555/1995456.1995555(652-662)Online publication date: 13-Dec-2009
Performance evaluation of ASAP3 for steady-state output analysis
Abstract
An experimental performance evaluation of ASAP3 is presented, including several queueing systems with characteristics typically encountered in practical applications of steady-state simulation analysis procedures. Based on the method of nonoverlapping batch means, ASAP3 is a sequential procedure designed to produce a confidence-interval estimator of a steady-state mean response that satisfies user-specified precision and coverage-probability requirements. ASAP3 is compared with its predecessor ASAP and the batch means procedure of Law and Carson (LC) in the following test problems: (a) queue waiting times in the M/M/1/LIFO, M/H2/1, and M/M/1 queues with 80% server utilization; and (b) response (sojourn) times in a central server model of a computer system. Regarding conformance to the given precision and coverage-probability requirements, ASAP3 compared favorably with the ASAP and LC procedures. Regarding the average sample sizes needed to satisfy progressively more stringent precision requirements, ASAP3's efficiency was reasonable for the given test problems.
References
[1]
Alexopoulos, C., and D. Goldsman. 2004. To batch or not to batch? ACM Transactions on Modeling and Computer Simulation 14 (1): 76--114.
[2]
Box, G. E. P., G. M. Jenkins, and G. C. Reinsel. 1994. Time series analysis: Forecasting and control. 3rd ed. Englewood Cliffs, New Jersey: Prentice Hall.
[3]
Chow, Y. S., and H. Robbins. 1965. On the asymptotic theory of fixed-width sequential confidence intervals for the mean. The Annals of Mathematical Statistics 36: 457--462.
[4]
Fishman, G. S. 1998. LABATCH.2: Software for statistical analysis of simulation sample path data. In Proceedings of the 1998 Winter Simulation Conference, ed. D. J. Medeiros, E. F. Watson, J. S. Carson, and M. S. Manivannan, 131--139. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. Available online via <www.informs-cs.org/wsc98papers/017.PDF> {accessed July 15, 2005}.
[5]
Fishman, G. S., and L. S. Yarberry. 1997. An implementation of the batch means method. INFORMS Journal on Computing 9 (3): 296--310.
[6]
Lada, E. K., J. R. Wilson, and N. M. Steiger. 2003. A wavelet-based spectral method for steady-state simulation analysis. In Proceedings of the 2003 Winter Simulation Conference, ed. S. Chick, P. J. Sánchez, D. Ferrin, and D. J. Morrice, 422--430. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. Available online via <www.informs-cs.org/wsc03papers/052.pdf> {accessed July 15, 2005}.
[7]
Law, A. M., Carson, J. S. 1979. A sequential procedure for determining the length of a steady-state simulation. Operations Research 27 (5): 1011--1025.
[8]
Nádas, A. 1969. An extension of a theorem of Chow and Robbins on sequential confidence intervals for the mean. The Annals of Mathematical Statistics 40 (2): 667--671.
[9]
Steiger, N. M. 1999. Improved batching for confidence interval construction in steady-state simulation. Doctoral dissertation, Department of Industrial Engineering, North Carolina State University, Raleigh, North Carolina. Available online via <www.lib.ncsu.edu/etd/public/etd-19231992992670/etd.pdf> {accessed July 15, 2005}.
[10]
Steiger, N. M., E. K. Lada, J. R. Wilson, C. Alexopoulos, D. Goldsman, and F. Zouaoui. 2002. ASAP2: An improved batch means procedure for simulation output analysis. In Proceedings of the 2002 Winter Simulation Conference, ed. E. Yücesan, C.-H. Chen, J. L. Snowdon, and J. M. Charnes, 336--344. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. Available online via <www.informs-cs.org/wsc02papers/043.pdf> {accessed July 15, 2005}.
[11]
Steiger, N. M., E. K. Lada, J. R. Wilson, J. A. Joines, C. Alexopoulos, and D. Goldsman. 2003. ASAP3 software and user's manual. Department of Industrial Engineering, North Carolina State University, Raleigh, North Carolina. Available online via <ftp.ncsu.edu/pub/eos/pub/jwilson/installasap3.exe> {accessed July 15, 2005}.
[12]
Steiger, N. M., E. K. Lada, J. R. Wilson, J. A. Joines, C. Alexopoulos, and D. Goldsman. 2005a. ASAP3: A batch means procedure for steady-state simulation analysis. ACM Transactions on Modeling and Computer Simulation 15 (1): 39--73.
[13]
Steiger, N. M., E. K. Lada, and J. R. Wilson. 2005b. Performance evaluation of ASAP3 for steady-state simulation analysis. IIE Transactions, in review. Available online via <ftp.ncsu.edu/pub/eos/pub/jwilson/iiev15t.pdf> {accessed July 15, 2005}.
[14]
Steiger, N. M., and J. R. Wilson. 1999. Improved batching for confidence interval construction in steady-state simulation. In Proceedings of the 1999 Winter Simulation Conference, ed. P. A. Farrington, H. B. Nembhard, D. T. Sturrock, and G. W. Evans, 442--451. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. Available online via <www.informs-cs.org/wsc99papers/061.PDF> {accessed July 15, 2005}.
[15]
Steiger, N. M., and J. R. Wilson. 2000. Experimental performance evaluation of batch-means procedures for simulation output analysis. In Proceedings of the 2000 Winter Simulation Conference, ed. R. R. Barton, J. A. Joines, K. Kang, and P. A. Fishwick, 627--636. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. Available online via <www.informs-cs.org/wsc00papers/084.PDF> {accessed July 15, 2005}.
[16]
Steiger, N. M., and J. R. Wilson. 2001. Convergence properties of the batch means method for simulation output analysis. INFORMS Journal on Computing 13 (4): 277--293.
[17]
Steiger, N. M., and J. R. Wilson. 2002a. An improved batch means procedure for simulation output analysis. Management Science 48 (12): 1569--1586.
[18]
Steiger, N. M., and J. R. Wilson. 2002b. ASAP software and user's manual. Department of Industrial Engineering, North Carolina State University, Raleigh, North Carolina. Available online via <ftp.ncsu.edu/pub/eos/pub/jwilson/installasap.exe> {accessed July 15, 2005}.
[19]
Steiger, N. M., E. K. Lada, J. R. Wilson, J. A. Joines, C. Alexopoulos, and D. Goldsman. 2004. Steady-state simulation analysis using ASAP3. In Proceedings of the 2004 Winter Simulation Conference, ed. R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters, 672--680. Available online via <www.informs-sim.org/wsc04papers/081.pdf> {accessed July 15, 2005}.
[20]
Stuart, A., and J. K. Ord. 1994. Kendall's advanced theory of statistics, Volume 1: Distribution theory. 6th ed. London: Edward Arnold.
- Performance evaluation of ASAP3 for steady-state output analysis
Recommendations
Steady-state analysis of a discrete-time batch arrival queue with working vacations
This paper analyzes a discrete-time batch arrival queue with working vacations. In a Geo^X/G/1 system, the server works at a lower speed during the vacation period which becomes a lower speed operation period. This model is more appropriate for the ...
Steady-state simulation analysis using ASAP3
WSC '04: Proceedings of the 36th conference on Winter simulationWe discuss ASAP3, a refinement of the batch means algorithms ASAP and ASAP2. ASAP3 is a sequential procedure designed to produce a confidence-interval estimator for the expected response of a steady-state simulation that satisfies user-specified ...
How Hard are Steady-State Queueing Simulations?
Special Issue on Don IglehartSome queueing systems require tremendously long simulation runlengths to obtain accurate estimators of certain steady-state performance measures when the servers are heavily utilized. However, this is not uniformly the case. We analyze a number of ...
Comments
Information & Contributors
Information
Published In
December 2005
2769 pages
ISBN:0780395190
Sponsors
Publisher
Winter Simulation Conference
Publication History
Published: 04 December 2005
Check for updates
Qualifiers
- Article
Acceptance Rates
WSC '05 Paper Acceptance Rate 209 of 316 submissions, 66%;
Overall Acceptance Rate 3,413 of 5,075 submissions, 67%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 67Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 26 Jan 2025
Other Metrics
Citations
Cited By
View all- Tafazzoli AWilson JDunkin AIngalls R(2009)N-SkartWinter Simulation Conference10.5555/1995456.1995555(652-662)Online publication date: 13-Dec-2009
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in