research-article

Formal specification of hypotheses for assisting computer simulation studies

Authors:

Colja A. Becker,

Ingo J. TimmAuthors Info & Claims

TMS/DEVS '17: Proceedings of the Symposium on Theory of Modeling & Simulation

Article No.: 18, Pages 1 - 12

Published: 23 April 2017 Publication History

Abstract

The aim of computer simulation studies is to answer research questions by means of experiments. For providing reliable evidence, the procedure of the study needs to be aligned with the question and the steps of the study need to be adjusted and combined accordingly. Supporting this process is challenging as the identification, customization, and combination of adequate tools and techniques for the systematic design of simulation experiments with respect to a hypothesis is not trivial. Hence, for providing computer-aided assistance, a language for the specification of hypotheses with respect to the credible and reproducible testing of research questions in computer simulation is needed. In this paper, we propose an approach for formally specifying hypotheses that allows for automated hypothesis testing. Based on specified hypotheses, we demonstrate the assistance of simulation studies in terms of model parametrization and analysis of results with respect to the statistically sound evaluation of hypotheses.

References

[1]

Bajpai, N. 2009. Business Statistics. Pearson.

[2]

Banks, J., J. Carson, B. Nelson, and D. Nicol. 2013. Discrete-Event System Simulation, Volume 5. Pearson.

[3]

Berndt, J. O. 2013. "Self-organizing Logistics Process Control: An Agent-based Approach". In Agents and Artificial Intelligence, pp. 397--412. Springer.

[4]

Better, M., F. Glover, and M. Laguna. 2007. "Advances in analytics: Integrating dynamic data mining with simulation optimization". IBM Journal of Research and Development vol. 51 (3.4), pp. 477--487.

Digital Library

[5]

Bley, H., C. Franke, C. Wuttke, and A. Gross. 2000. "Automation of simulation studies". In Proceedings of the 2nd CIRP International Seminar on Intelligent Computation in Manufacturing, pp. 89--94. Citeseer.

[6]

Bogon, T., I. J. Timm, U. Jessen, M. Schmitz, S. Wenzel, A. D. Lattner, D. Paraskevopoulos, and S. Spieckermann. 2012. "Towards assisted input and output data analysis in manufacturing simulation: the EDASim approach". In Proceedings of the Winter Simulation Conference, pp. 257. Winter Simulation Conference.

Digital Library

[7]

Buchanan, B. G., and E. A. Feigenbaum. 1978. "DENDRAL and Meta-DENDRAL: Their Applications Dimension.". Technical report, DTIC Document.

Digital Library

[8]

Croucher, A. 2011. "PyTOUGH: A Python scripting library for automating TOUGH2 simulations". In Proceedings of the New Zealand Geothermal Workshop, Volume 21, pp. 1--6.

[9]

Daum, T., and R. G. Sargent. 2001. "Experimental frames in a modern modeling and simulation system". Iie Transactions vol. 33 (3), pp. 181--192.

[10]

Degroot, A. 1969. Methodology: Foundations of Inference and Research in the Behavioral Sciences. Psychological Studies. Walter de Gruyter GmbH & Company KG.

[11]

Diallo, S. Y., J. J. Padilla, I. Bozkurt, and A. Tolk. 2013. "Modeling and simulation as a theory building paradigm". In Ontology, Epistemology, and Teleology for M&S, pp. 193--206. Springer.

[12]

Ewing, G., K. Pawlikowski, and D. McNickle. 1999. Akaroa-2: Exploiting network computing by distributing stochastic simulation. SCSI Press.

[13]

Freedman, D., R. Pisani, and R. Purves. 2007. Statistics. W.W. Norton & Company.

[14]

Ganter, B., and S. O. Kuznetsov. 2000. "Formalizing hypotheses with concepts". In International Conference on Conceptual Structures, pp. 342--356. Springer.

Digital Library

[15]

Gilbert, N., and K. Troitzsch. 2005. Simulation for the social scientist. McGraw-Hill Education (UK).

Digital Library

[16]

Gonçalves, B., and F. Porto. 2015. "Managing scientific hypotheses as data with support for predictive analytics". Computing in Science & Engineering vol. 17 (5), pp. 35--43.

[17]

Griffin, T. G., S. Petrovic, A. Poplawski, and B. Premore. 2002. SOS: Scripts for Organizing 'Speriments. SSF Research Network. http://www.ssfnet.org/sos/, accessed Jan. 2017.

[18]

Hanneman, R., A. Kposowa, and M. Riddle. 2012. Basic Statistics for Social Research. Wiley.

[19]

Himmelspach, J., and A. M. Uhrmacher. 2007. "Plug'n simulate". In 40th Annual Simulation Symposium, pp. 137--143. IEEE Computer Society.

Digital Library

[20]

Hoad, K., S. Robinson, and R. Davies. 2010. "Automated selection of the number of replications for a discrete-event simulation". Journal of the Operational Research Society vol. 61 (11), pp. 1632--1644.

[21]

Hudert, S., C. Niemann, and T. Eymann. 2010. "On computer simulation as a component in information systems research". In Design Science Research in Information Systems, pp. 167--179. Springer.

Digital Library

[22]

King, R. D., J. Rowland, S. G. Oliver, M. Young, W. Aubrey, E. Byrne, M. Liakata, M. Markham, P. Pir, L. N. Soldatova et al. 2009. "The automation of science". Science vol. 324 (5923), pp. 85--89.

[23]

Kleijnen, J. P. 2005. "Supply chain simulation tools and techniques: a survey". International Journal of Simulation and Process Modelling vol. 1 (1-2), pp. 82--89.

[24]

Kothari, C. 2004. Research Methodology: Methods and Techniques. New Age International (P) Limited.

[25]

Lattner, A. D., T. Bogon, and I. J. Timm. 2011. "Convergence Classification and Replication Prediction for Simulation Studies". In Int. Conf. Agents and Artificial Intelligence, pp. 255--268. Springer.

[26]

Lattner, A. D., H. Pitsch, I. J. Timm, S. Spieckermann, and S. Wenzel. 2011. "AssistSim--Towards Automation of Simulation Studies in Logistics". SNE vol. 21 (3--4), pp. 119--128.

[27]

Law, A. M. 2014. Simulation modeling and analysis, Volume 5. McGraw-Hill New York.

[28]

Lee, D.-E., and D. Arditi. 2006. "Automated statistical analysis in stochastic project scheduling simulation". Journal of Construction Engineering and Management vol. 132 (3), pp. 268--277.

[29]

Millman, E., D. Arora, and S. W. Neville. 2011. "STARS: A framework for statistically rigorous simulation-based network research". In Advanced Information Networking and Applications (WAINA), 2011 IEEE Workshops of International Conference on, pp. 733--739. IEEE.

Digital Library

[30]

Nikolai, C., and G. Madey. 2009. "Tools of the trade: A survey of various agent based modeling platforms". Journal of Artificial Societies and Social Simulation vol. 12 (2), pp. 2.

[31]

Ören, T. I., B. P. Zeigler, and M. S. Elzas. 1984. Simulation and Model-based Methodologies: An Integrative View, Volume 10. NATO ASI Series F: Computer and System Sciences.

[32]

Perrone, L. F., C. S. Main, and B. C. Ward. 2012. "Safe: simulation automation framework for experiments". In Proceedings of the Winter Simulation Conference, pp. 249. Winter Simulation Conference.

Digital Library

[33]

Recker, J. 2013. Information Systems Research as a Science, pp. 11--21. Springer.

[34]

Robinson, S. 2005. "Automated analysis of simulation output data". In Proceedings of the Winter Simulation Conference, pp. 763--770. IEEE.

Digital Library

[35]

Schreiber, G. 2008. "Knowledge Engineering". In Handbook of Knowledge Representation, edited by F. van Harmelen, V. Lifschitz, and B. Porter, Volume 1 of Found. of Art. Intelligence, pp. 929--946. Elsevier.

[36]

Schruben, L., and D. Singham. 2011. "Agent based simulation output analysis". In Proceedings of the Winter Simulation Conference, pp. 540--548. IEEE.

Digital Library

[37]

Simon, R. 1989. "Optimal two-stage designs for phase II clinical trials". Controlled clinical trials vol. 10 (1), pp. 1--10.

[38]

Soldatova, L. N., and R. D. King. 2006. "An ontology of scientific experiments". Journal of the Royal Society Interface vol. 3 (11), pp. 795--803.

[39]

Steiger, N. M., and J. R. Wilson. 2002. "An improved batch means procedure for simulation output analysis". Management Science vol. 48 (12), pp. 1569--1586.

Digital Library

[40]

Taylor, I. J., E. Deelman, D. B. Gannon, and M. Shields. 2014. Workflows for e-Science: scientific workflows for grids. Springer Publishing Company, Incorporated.

Digital Library

[41]

Teran-Somohano, A., O. Dayibaş, L. Yilmaz, and A. Smith. 2014. "Toward a model-driven engineering framework for reproducible simulation experiment lifecycle management". In Proceedings of the Winter Simulation Conference, pp. 2726--2737. IEEE.

Digital Library

[42]

Teran-Somohano, A., A. E. Smith, J. Ledet, L. Yilmaz, and H. Oğuztüzün. 2015. "A model-driven engineering approach to simulation experiment design and execution". In Proceedings of the Winter Simulation Conference, pp. 2632--2643. IEEE.

Digital Library

[43]

Tietjen, G. 2012. A Topical Dictionary of Statistics. Springer US.

[44]

Timm, I. J., and F. Lorig. 2015. "A survey on methodological aspects of computer simulation as research technique". In Proceedings of the Winter Simulation Conference, pp. 2704--2715. IEEE.

Digital Library

[45]

Tran, N., C. Baral, V. J. Nagaraj, and L. Joshi. 2005. "Knowledge-based integrative framework for hypothesis formation in biochemical networks". In Data Integration in the Life Sciences, pp. 121--136. Springer.

Digital Library

[46]

Triola, M. 2013. Elementary Statistics. Pearson Education, Limited.

[47]

Van Zundert, J., S. Antonijevic, A. Beaulieu, K. van Dalen-Oskam, D. Zeldenrust, and T. L. Andrews. 2012. "Cultures of Formalisation: Towards an Encounter between Humanities and Computing". In Understanding digital humanities, pp. 279--294. Springer.

[48]

Verburg, J. M., C. Grassberger, S. Dowdell, J. Schuemann, J. Seco, and H. Paganetti. 2016. "Automated Monte Carlo Simulation of Proton Therapy Treatment Plans". Technology in Cancer Research & Treatment vol. 15 (6), pp. NP35--NP46.

[49]

Wagner, T., A. Gellrich, C. Schwenke, K. Kabitzsch, and G. Schneider. 2014. "Automated planning and creation of simulation experiments with a domain specific ontology for semiconductor manufacturing AMHS". In Proceedings of the Winter Simulation Conference, pp. 2628--2639. IEEE Press.

Digital Library

[50]

Yilmaz, L., S. Chakladar, and K. Doud. 2016. "The Goal-Hypothesis-Experiment framework: A generative cognitive domain architecture for simulation experiment management". In Winter Simulation Conference (WSC), 2016, pp. 1001--1012. IEEE.

Digital Library

[51]

Zeigler, B. P., H. Praehofer, and T. G. Kim. 2000. Theory of modeling and simulation: integrating discrete event and continuous complex dynamic systems. Academic press.

Digital Library

Cited By

Uhrmacher AFrazier PHähnle RKlügl FLorig FLudäscher BNenzi LRuiz-Martin CRumpe BSzabo CWainer GWilsdorf P(2024)Context, Composition, Automation, and Communication: The C2AC Roadmap for Modeling and SimulationACM Transactions on Modeling and Computer Simulation10.1145/367322634:4(1-51)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3673226
Berndt JRodermund STimm IJohansson BJain S(2018)Social contagion of fertilityProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320638(953-964)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320638
Lorig FLebherz DBerndt JTimm I(2017)HYpothesis-driven experiment design in computer simulation studiesProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242290(1-12)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242290

Recommendations

Formal Specification and Design Time Testing

It is shown how design time testing can be used in conjunction with formal specification. Emphasis is placed on the benefits of using an executable specification language OBJ, of having a design controlled by requirements specification, and of adherence ...
A Formal Framework for ASTRAL Intralevel Proof Obligations

ASTRAL is a formal specification language for real-time systems. It is intended to support formal software development, and therefore has been formally defined. This paper focuses on how to formally prove the mathematical correctness of ASTRAL ...
Towards a Formal Basis for the Formal Development Method and the Ina Jo Specification Language
Special issue on computer security and privacy

In carrying out SDC's Formal Development Method, one writes a specification of a system under design in the Ina Jo™ specification language and proves that the specification meets the requirements of the system. This paper develops an abstract machine ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

TMS/DEVS '17: Proceedings of the Symposium on Theory of Modeling & Simulation

April 2017

234 pages

Editors:
Fernando Barros
University of Coimbra, Portugal
,
Xiaolin Hu
Georgia State University
,
Joachim Denil
University of Antwerp, Belgium
,
Rhys Goldstein
Autodesk Research, Canada

In-Cooperation

SIGSIM: ACM Special Interest Group on Simulation and Modeling

Publisher

Society for Computer Simulation International

San Diego, CA, United States

Publication History

Published: 23 April 2017

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SpringSim '17

SpringSim '17: Spring Simulation Multi-Conference

April 23 - 26, 2017

Virginia, Virginia Beach

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
50
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Uhrmacher AFrazier PHähnle RKlügl FLorig FLudäscher BNenzi LRuiz-Martin CRumpe BSzabo CWainer GWilsdorf P(2024)Context, Composition, Automation, and Communication: The C2AC Roadmap for Modeling and SimulationACM Transactions on Modeling and Computer Simulation10.1145/367322634:4(1-51)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3673226
Berndt JRodermund STimm IJohansson BJain S(2018)Social contagion of fertilityProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320638(953-964)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320638
Lorig FLebherz DBerndt JTimm I(2017)HYpothesis-driven experiment design in computer simulation studiesProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242290(1-12)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242290

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents