Software paradigms, assessment types and non-functional requirements in model-based integration testing: a systematic literature review

Context: In modern systems, like cyber-physical systems, where software and physical services are interacting, safety, security or performance play an important role. In order to guarantee the correct interoperability of such systems, with respect to functional and non-functional requirements, integration testing is an effective measure to achieve this. Model-based testing moreover not only enables early definition and validation, but also test automation. This makes it a good choice to overcome urgent challenges of integration testing. Objective: Many publications on model-based integration testing (MBIT) approaches can be found. Nevertheless, a study giving a systematic overview on the underlying software paradigms, measures for guiding the integration testing process as well as non-functional requirements they are suitable for, is missing. The aim of this paper is to find and synthesize the relevant primary studies to gain a comprehensive understanding of the current state of model-based integration testing. Method: For synthesizing the relevant studies, we conducted a systematic literature review (SLR) according to the guidelines of Kitchenham. Results: The systematic search and selection retrieved 83 relevant studies from which data has been extracted. Our review identified three assessment criteria for guiding the testing process, namely static metrics, dynamic metrics and stochastic &random. In addition it shows that just a small fraction considers non-functional requirements. Most approaches are for component-oriented systems. Conclusion: Results from the SLR show that there are two major research gaps. First, there is an accumulated need for approaches in the MBIT field that support non-functional requirements, as they are gaining importance. Second, means for steering the integration testing process, especially together with automation, need to evolve.