Article

Free access

Principles and techniques of simulation validation, verification, and testing

Author:

Osman BalciAuthors Info & Claims

WSC '95: Proceedings of the 27th conference on Winter simulation

Pages 147 - 154

https://doi.org/10.1145/224401.224456

Published: 01 December 1995 Publication History

Abstract

Sufficient experience has been gained over the last decade in simulation validation, verification, and testing (VV&T) to establish basic principles about its characteristics. This paper presents 15 principles of simulation VV&T. These principles help the researchers, practitioners and managers better understand what model VV&T is all about. They serve to provide the underpinnings for the VV&T techniques that can be used throughout the life cycle of a simulation study. This paper also surveys current software VV&T techniques and current simulation VV&T techniques. Understanding and applying these principles and employing proper testing techniques throughout the life cycle of a simulation study are key factors in increasing the probability of success in a simulation study.

References

[1]

Balci, O. 1986. Requirements for model development environments. Computers & Operations Research 13:53-67.

Digital Library

[2]

Balci, O. 1994. Validation, verification, and testing techniques throughout the life cycle of a simulation study. Annals of Operations Research 53:121-173.

[3]

Balci, O. 1996. Principles of simulation model validation, verification, and testing. International Journal in Computer Simulation, to appear.

Digital Library

[4]

Balci, O., A. I. Bertelrud, C. M. Esterbrook, and R. E. Nance. 1995. A picture-based object-oriented visual simulation environment. In Proceedings of the 1995 Winter Simulation Conference, ed. C. Alexopoulos, K. Kang, W. Lilegdon, and D. Goldsman. IEEE, Pi~cataway, New Jersey.

Digital Library

[5]

Balci, O. and R. E. Nance. 1985. Formulated problem verification as an explicit requirement of model credibility. Simulation 45:76-86.

[6]

Balci, O. and R. E. Nance. 1987. Simulation model development environments: a research prototype. Journal of Operational Research Society 38:753- 763.

[7]

Balci, O. and R. E. Nance. 1992. The simulation model development environment: an overview. In Proceedings of the 1992 Winter Simulation Conference, ed. J. J. Swain, D. Goldsman, R. C. Crain, and J. R. Wilson, 726-736. IEEE, Piscataway, New Jersey.

Digital Library

[8]

Balci, O. and R. G. Sargent. 1981. A methodology for cost-risk analysis in the statistical validation of simulation models. Communications of the ACM 24:190-197.

Digital Library

[9]

Hetzel, W. 1984. The complete guide to software testing. QED Information Sciences, Wellesley, Massachusetts.

Digital Library

[10]

Johnson, M. E. and M. Mollaghasemi. 1994. Simulation input data modeling. Annals of Operations Research 53:47-75.

[11]

Nance, R. E. 1994. Conical methodology: an evolutionary convergence of systems and software engineering. Annals of Operations Research 53:1-45.

[12]

Nance, R. E. and C. M. Overstreet. 1987. Diagnostic assistance using digraph representations of discrete event simulation model specifications. Transactions of the SCS 4:33-57.

[13]

0ren, T. I. 1981. Concepts and criteria to assess acceptability of simulation studies: a frame of reference. Communications of the A CM 24:180-189.

Digital Library

[14]

0ren, T. I. 1986. Artificial intelligence in quality assurance of simulation studies. In Modelling and simulation methodology in the artificial intelligence era, ed. M. S. Elzas, T. I. Oren, and B. P. Zeigler, 267- 278. North Holland, Amsterdam, Holland.

[15]

0ren, T. I. 1987. Quality assurance paradigms for artificial intelligence in modelling and simulation. Simulation 48:149-151.

Digital Library

[16]

Paul, R. j. 1989. Visual simulation: seeing is believing? In Impacts of recent computer advances on operations research, ed. R. Sharda, B. L. Golden, E. Wasil, O. Balci, and W. Stewart, 422-432. Elsevier Science Publishing, New York, New York.

[17]

Schach, S. R. 1993. Software engineering. 2nd ed. Irwin, Homewood, Illinois.

Digital Library

[18]

Schlesinger, S., et al. 1979. Terminology for model credibility. Simulation 32:103-104.

[19]

Shannon, R. E. 1975. Systems simulation: the art and science. Prentice-Hall, Englewood Cliffs, New Jersey.

[20]

U.S. GAd. 1976. Report to the congress: ways to improve management of federally funded computerized models. LCD-75-111, U.S. General Accounting Office, Washington, DC.

[21]

U.S. GAd. 1979. Guidelines for model evaluation. PAD-79-17, U.S. General Accounting Office, Washington, DC.

[22]

U.S. GAd. 1987. DOD Simulations: improved assessment procedures would increase the credibility of results. GAO/PEMD-88-3, U.S. General Accounting Office, Washington, DC.

[23]

Watson, C. E. 1976. The problems of problem solving. Business Horizons 19:88-94.

[24]

Whitner, R. B. and O. Balci. 1989. Guidelines for selecting and using simulation model verification techniques. In Proceedings of the 1989 Winter Simulation Conference, ed. E. A. MacNair, K. J. Musselman, and P. Heidelberger, 559-568. IEEE, Piscataway, New Jersey.

Digital Library

Cited By

Hamed MTuran HElsawah S(2024)Balancing supply diversification and environmental impacts: A system dynamics approach to de-risk rare earths supply chainResources Policy10.1016/j.resourpol.2024.10503892(105038)Online publication date: May-2024
https://doi.org/10.1016/j.resourpol.2024.105038
Souza PFerreto TCalheiros R(2023)EdgeSimPy: Python-based modeling and simulation of edge computing resource management policiesFuture Generation Computer Systems10.1016/j.future.2023.06.013148(446-459)Online publication date: Nov-2023
https://doi.org/10.1016/j.future.2023.06.013
Tüzün AHackenberg G(2023)Quality Assurance of Digital Twins: An Experience Report in the Automotive IndustryQuality of Information and Communications Technology10.1007/978-3-031-43703-8_2(15-29)Online publication date: 13-Sep-2023
https://doi.org/10.1007/978-3-031-43703-8_2
Show More Cited By

Index Terms

Principles and techniques of simulation validation, verification, and testing

Recommendations

Principles of Simulation Model Validation, Verification, and Testing
Validation, Verification, and Testing Techniques Throughout the Life Cycle of a Simulation Study
Selecting verification and validation techniques for simulation projects: a planning and tailoring strategy
WSC '13: Proceedings of the 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World

Conducting verification and validation (V&V) of modeling and simulation (M&S) requires systematic and structured application of different V&V techniques throughout the M&S life cycle. Whether an existing technique is appropriate to a particular V&V ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSC '95: Proceedings of the 27th conference on Winter simulation

December 1995

1493 pages

ISBN:0780330188

Chairmen:
William R. Lilegdon
Pritsker Corp., West Lafayette, IN
,
David Goldsman
Georgia Institute of Technology, Atlanta
,
Editors:
Christos Alexopoulos
Georgia Institute of Technology, Atlanta
,
Keebom Kang
Naval Postgraduate School, Monterey, CA

Sponsors

IIE: Institute of Industrial Engineers
IEEE-SMCS: Systems, Man & Cybernetics Society
INFORMS/CS: Computer Science TC
ASA: American Statistical Association
ACM: Association for Computing Machinery
SCS: Society for Computer Simulation
SIGSIM: ACM Special Interest Group on Simulation and Modeling
IEEE-CS: Computer Society
NIST: National Institute of Standards & Technology

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 December 1995

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

WSC95

Sponsor:

IIE
IEEE-SMCS
INFORMS/CS
ASA
ACM
SCS
SIGSIM
IEEE-CS
NIST

WSC95: 1995 Winter Simulation Conference

December 3 - 6, 1995

Virginia, Arlington, USA

Acceptance Rates

WSC '95 Paper Acceptance Rate 122 of 183 submissions, 67%;

Overall Acceptance Rate 3,413 of 5,075 submissions, 67%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
2,614
Total Downloads

Downloads (Last 12 months)429
Downloads (Last 6 weeks)35

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hamed MTuran HElsawah S(2024)Balancing supply diversification and environmental impacts: A system dynamics approach to de-risk rare earths supply chainResources Policy10.1016/j.resourpol.2024.10503892(105038)Online publication date: May-2024
https://doi.org/10.1016/j.resourpol.2024.105038
Souza PFerreto TCalheiros R(2023)EdgeSimPy: Python-based modeling and simulation of edge computing resource management policiesFuture Generation Computer Systems10.1016/j.future.2023.06.013148(446-459)Online publication date: Nov-2023
https://doi.org/10.1016/j.future.2023.06.013
Tüzün AHackenberg G(2023)Quality Assurance of Digital Twins: An Experience Report in the Automotive IndustryQuality of Information and Communications Technology10.1007/978-3-031-43703-8_2(15-29)Online publication date: 13-Sep-2023
https://doi.org/10.1007/978-3-031-43703-8_2
Bruzzone AMassei MFrosolini M(2022)Redesign of Supply Chain in Fashion Industry based on Strategic EngineeringProcedia Computer Science10.1016/j.procs.2022.01.392200(1913-1918)Online publication date: 2022
https://doi.org/10.1016/j.procs.2022.01.392
Pasley HBrown HHolzworth DWhish JBell LHuth N(2022)How to build a crop model. A reviewAgronomy for Sustainable Development10.1007/s13593-022-00854-943:1Online publication date: 29-Dec-2022
https://doi.org/10.1007/s13593-022-00854-9
TEKİNDAL MTEKİNDAL M(2021)Validity and Reliability of Basic Depression Scale For TurkeyTemel Depresyon Ölçeği’nin Türkçe Geçerlik ve GüvenirliğiBatı Karadeniz Tıp Dergisi10.29058/mjwbs.9066295:3(452-463)Online publication date: 25-Dec-2021
https://doi.org/10.29058/mjwbs.906629
Abu-Taieh E(2019)Introductory Chapter: Simulation and ModelingSimulation Modelling Practice and Theory10.5772/intechopen.83663Online publication date: 20-Feb-2019
https://doi.org/10.5772/intechopen.83663
Schamai WAlbarello NHelle PBuffoni LFritzson P(2017)TOWARDS THE AUTOMATION OF MODEL-BASED DESIGN VERIFICATIONINSIGHT10.1002/inst.1214320:1(42-48)Online publication date: 6-Apr-2017
https://doi.org/10.1002/inst.12143
Ramos DMoreno UAlmeida L(2016)Method for Design and Performance Evaluation of Ad Hoc Networked Mobile Robotic Systems using OMNET++IFAC-PapersOnLine10.1016/j.ifacol.2016.11.14449:30(144-149)Online publication date: 2016
https://doi.org/10.1016/j.ifacol.2016.11.144
Kasaie PKelton W(2015)Guidelines for design and analysis in agent-based simulation studiesProceedings of the 2015 Winter Simulation Conference10.5555/2888619.2888638(183-193)Online publication date: 6-Dec-2015
https://dl.acm.org/doi/10.5555/2888619.2888638
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents