research-article

Traceability and SysML design slices to support safety inspections: A controlled experiment

Authors:

Davide Falessi,

Mehrdad Sabetzadeh,

Tao YueAuthors Info & Claims

ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 23, Issue 1

Article No.: 9, Pages 1 - 43

https://doi.org/10.1145/2559978

Published: 20 February 2014 Publication History

Abstract

Certifying safety-critical software and ensuring its safety requires checking the conformance between safety requirements and design. Increasingly, the development of safety-critical software relies on modeling, and the System Modeling Language (SysML) is now commonly used in many industry sectors. Inspecting safety conformance by comparing design models against safety requirements requires safety inspectors to browse through large models and is consequently time consuming and error-prone. To address this, we have devised a mechanism to establish traceability between (functional) safety requirements and SysML design models to extract design slices (model fragments) that filter out irrelevant details but keep enough context information for the slices to be easy to inspect and understand. In this article, we report on a controlled experiment assessing the impact of the traceability and slicing mechanism on inspectors' conformance decisions and effort. Results show a significant decrease in effort and an increase in decisions' correctness and level of certainty.

References

[1]

M. Abadi and L. Lamport. 1991. The existence of refinement mappings. Theor. Comput. Sci. 82, 2.

Digital Library

[2]

G. Adair. 1984. The hawthorne effect: A reconsideration of the methodological artifact. J. Appl. Psychol. 69, 2.

[3]

E. R. Babbie. 2010. The Basics of Social Research 5^th Ed. Wadsworth Publishing.

[4]

T. Baguley. 2006. Understanding statistical power in the context of applied research. Appl. Ergonom. 35, 2.

[5]

V. Basili, G. Caldiera, and D. Rombach. 1994. Goal/question/metric paradigm. In Encyclopedia Software Engineering 1, John Wiley and Sons, 528--532.

[6]

V. R. Basili, F. Shull, and F. Lanubile. 1999. Building knowledge through families of experiments. IEEE Trans. Softw. Engin. 25, 4. 456--473.

Digital Library

[7]

S. Biffl, A. Aurum, B. Bohem, H. Erdogmus, and P. Grünbacher. 2006. Value-Based Software Engineering. Springer.

Digital Library

[8]

R. Binder. 1999. Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Longman Publishing Company.

Digital Library

[9]

D. Binkley and K. Gallagher. 1996. Program slicing. Adv. Comput. 43.

[10]

D. Binkley and M. Harman. 2003. A survey of empirical results on program slicing. Adv. Comput. 62.

[11]

L. Briand, T. Coq, S. Nejati, R. K. Panesar-Walawege, and M. Sabetzadeh. 2009a. Characterizing the chain of evidence for software safety cases: A conceptual model based on the iec 61508 standard. http://modelme.simula.no/.

[12]

L. Briand, Y. Labiche, and T. Yue. 2009b. Automated traceability analysis for uml model refinements. Inf. Softw. Technol. 51, 2. 512--527.

Digital Library

[13]

L. C. Briand, Y. Labiche, L. O'Sullivan, and M. M. Sowka. 2006. Automated impact analysis of uml models. J. Syst. Softw. 79, 3, 339--352.

Digital Library

[14]

B. Brykczynski. 1999. A survey of software inspection checklists. SIGSOFT Softw. Engin. Not. 24, 1, 82.

Digital Library

[15]

J. E. Burge, J. M. Carroll, R. McCall, and I. Mistrk. 2008. Rationale-Based Software Engineering. Springer.

Digital Library

[16]

G. Cantone, Z. Abdulnabi, A. Lomartire, and G. Calavaro. 2003. Effectiveness of code reading and functional testing with event-driven object-oriented software. In Empiracal Methods and Studies in Software Engineering, Springer, 166--192.

[17]

J. Cleland-Huang, R. Settimi, O. Benkhadra, E. Berezhanskaya, and S. Christina. 2005. Goal-centric traceability for managing non-functional requirements. In Proceedings of the 27^th International Conference on Software Engineering (ICSE'05).

Digital Library

[18]

P. Clements, R. Kazman, and M. Klein. 2002. Evaluating Software Architecture: Methods and Case Studies. Addison-Wesley.

Digital Library

[19]

J. Coehn. 1992. A power prime. Psychol. Bull. 112, 1.

[20]

M. Dwyer, J. Hatcliff, M. Hoosier, V. Ranganath, A. Robby, and T. Wallentine. 2006. Evaluating the effectiveness of slicing for model reduction of concurrent object-oriented programs. In Proceedings of the 12^th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS'06). 73--89.

Digital Library

[21]

A. Egyed, S. Biffl, M. Heindl, and P. Grunbacher. 2005. Determining the cost-quality trade-off for automated software traceability. In Proceedings of the 20^th IEEE/ACM International Conference on Automated Software Engineering.

Digital Library

[22]

D. Falessi, M. Ali Babar, G. Cantone, and P. Kruchten. 2010. Applying empirical software engineering to software architecture: Challenges and lessons learned. Empirical Softw. Engin. 15, 3, 250--276.

Digital Library

[23]

D. Falessi, M. Becker, and G. Cantone. 2006a. Design decision rationale: Experiences and steps ahead towards systematic use. ACM SIGSOFT Softw. Engin. Not. 31, 5.

Digital Library

[24]

D. Falessi, L. Briand, G. Cantone, R. Capilla, and P. Kruchten. 2012a. The value of design rationale information. ACM Trans. Softw. Engin. Methodol. 22, 3.

Digital Library

[25]

D. Falessi, L. C. Briand, and G. Cantone. 2009. The impact of automated support for linking equivalent requirements based on similarity measures. Tech. rep. 2009-08, Simula Research Laboratory.

[26]

D. Falessi and G. Cantone. 2006. Exploring feasibility of software defects orthogonal classification. In Software and Data Technologies, Springer, 136--152.

[27]

D. Falessi, G. Cantone, and M. Becker. 2006b. Documenting design decision rationale to improve individual and team design decision making: An experimental evaluation. In Proceedings of the 5^th ACM/IEEE International Symposium on Empirical Software Engineering.

Digital Library

[28]

D. Falessi, G. Cantone, and G. Canfora. 2011a. Empirical principles and an industrial case study in retrieving equivalent requirements via natural language processing techniques. IEEE Trans. Softw. Engin. 39, 1, 1--30.

Digital Library

[29]

D. Falessi, G. Cantone, and G. Canfora. 2013. Empirical principles and an industrial case study in retrieving equivalent requirements via natural language processing techniques. IEEE Trans. Softw. Engin. 39, 1, 18--44.

Digital Library

[30]

D. Falessi, G. Cantone, and C. Grande. 2007. A comparison of structured analysis and object oriented analysis: An experimental study. In Proceedings of the International Conference on Software and Data Technologies.

[31]

D. Falessi, G. Cantone, and P. Kruchten. 2008. Value-based design decision rationale documentation: Principles and empirical feasibility study. In Proceeding of the 7^th Working IEEE/IFIP Conference on Software Architecture (WICSA'08).

Digital Library

[32]

D. Falessi, S. Nejati, M. Sabetzadeh, L. C. Briand, and A. Messina. 2011b. SafeSlice: A model slicing and design safety inspection tool for sysml. In Proceedings of the 19^th ACM SIGSOFT Symposium on the Foundations of Software Engineering and the 13^th European Software Engineeing Conference (ESEC/FSE'11).

Digital Library

[33]

D. Falessi, M. Sabetzadeh, L. Briand, E. Turella, T. Coq, and R. K. Panesar-Walawege. 2012b. Planning for safety standards compliance: A model-based tool-supported approach. IEEE Softw. 29, 3, 64--70.

Digital Library

[34]

P. Haumer, M. Jarke, K. Pohl, and K. Weidenhaupt. 2000. Improving reviews of conceptual models by extended traceability to captured system usage. Interact. Comput. 13, 1, 77--95.

[35]

J. H. Hayes and A. Dekhtyar. 2005. A framework for comparing requirements tracing experiments. Int. J. Softw. Engin. Knowl. Engin. 15, 5.

[36]

M. Heindl and S. Biffl. 2005. A case study on value-based requirements tracing. In Proceedings of the 10^th European Software Engineering Conference. ACM Press, New York.

Digital Library

[37]

C. Hofmeister, R. Nord, and D. Soni. 1999. Applied Software Architecture. Addison-Wesley.

Digital Library

[38]

IEC. 2005. Functional safety of electrical / electronic / programmable electronic safety-related systems (IEC 61508). International Electrotechnical Commission.

[39]

IEEE. 1994. 1044-1993 IEEE standard classification for software anomalies. http://standards.ieee.org/findstds/standard/1044-2009.html.

[40]

IEEE. 2000. IEEE std 1471:2000, recommended practice for architectural description of software- intensive systems. http://standards.ieee.org/findstds/standard/1471-2000.html.

[41]

H. Kagdi, J. Maletic, and A. Sutton. 2005. Context-free slicing of uml class models. In Proceedings of the 21^st IEEE International Conference on Software Maintenance. IEEE Computer Society.

Digital Library

[42]

R. Kazman and L. Bass. 2002. Making architecture reviews work in the real world. IEEE Softw. 19, 1, 67--73.

Digital Library

[43]

B. Kitchenham and S. L. Pfleeger. 2002. Principles of survey research: Part 3: Constructing a survey instrument. ACM SIGSOFT Softw. Engin. Not. 27, 2.

Digital Library

[44]

T. Klykken. 2009. A case study using SysML for safety-critical systems. Master thesis, University of Oslo. http://people.svv.lu/nejati/tonje.pdf.

[45]

A. V. Knethen. 2002. Change-oriented requirements traceability: Support for evolution of embedded systems. In Proceedings of the International Conference on Software Maintenance (ICSM'02). IEEE Computer Society.

Digital Library

[46]

A. V. Knethen and M. Grund. 2003. QuaTrace: A tool environment for (semi-) automatic impact analysis based on traces. In Proceedings of the International Conference on Software Maintenance. IEEE Computer Society.

Digital Library

[47]

B. Korel, I. Singh, L. Tahat, and B. Vaysburg. 2003. Slicing of state-based models. In Proceedings of the International Conference on Software Maintenance. IEEE Computer Society.

Digital Library

[48]

P. Kruchten. 2003. The Rational Unified Process: An Introduction 3rd Ed. Addison-Wesley Professional.

Digital Library

[49]

P. Kruchten, P. Lago, and H. Van Vliet. 2006. Building up and reasoning about architectural knowledge. In Proceedings of the 2^nd International Conference on the Quality of Software Architectures. Lecture Notes in Computer Science, vol. 4214, Springer.

Digital Library

[50]

P. Lago, P. Avgeriou, and R. Hilliard. 2010. Guest editors' introduction: Software architecture: Framing stakeholders' concerns. IEEE Softw. 27, 20--24.

Digital Library

[51]

P. Letelier. 2002. A framework for requirements traceability in uml-based projects. In Proceedings of the 1st International Workshop on Traceability in Emerging Forms of Software Engineering.

[52]

C. Lewerentz and T. Lindner. 1995. Formal Development of Reactive Systems - Case Study Production Cell. Springer.

Digital Library

[53]

R. Likert. 1932. A technique for the measurement of attitudes. Arch. Psychol. 22, 140, 1--55.

[54]

C. Manning, P. Raghavan, and H. Schtze. 2008. Introduction to Information Retrieval. Cambridge University Press.

Digital Library

[55]

D. Montgomery. 2006. Design and Analysis of Experiments. John Wiley and Sons.

Digital Library

[56]

S. Nair, J. de la Vara, M. Sabetzadeh, and L. Briand. 2013. Classification, structuring, and assessment of evidence for safety: A systematic literature review. In Proceedings of the 6^th IEEE International Conference on Software Testing, Verification and Validation.

Digital Library

[57]

S. Nejati, M. Sabetzadeh, D. Falessi, L. Briand, and T. Coq. 2012. A SysML-based approach to traceability management and design slicing in support of safety certification: Framework, tool support, and case studies. Inf. Softw. Technol. 54, 6.

Digital Library

[58]

B. Nuseibeh, J. J. Kramer, and A. A. Finkelstein. 2003. ViewPoints: Meaningful relationships are difficult&excl; In Proceedings of the 25^th International Conference on Software Engineering.

Digital Library

[59]

OMG. 2008. SysML specification (formal/2008-11-01). http://www.omg.org/spec/SysML/1.1/.

[60]

OMG. 2009. UML 2.2 superstructure specification (formal/2009-02-04). http://www.omg.org/spec/UML/2.2/.

[61]

A. Oppenheim. 1992. Questionnaire Design, Interviewing and Attitude Measurement. Continuum International Publishing.

[62]

B. Ramesh and M. Jarke. 2001. Toward reference models for requirements traceability. IEEE Trans. Softw. Engin. 27, 1, 58--93.

Digital Library

[63]

R. Rosenthal. 1966. Experimenter Effects in Behavioral Research. Appleton Century Crofts.

[64]

N. Rozanski and E. Woods. 2005. Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives. Addison-Wesley.

Digital Library

[65]

M. Sabetzadeh, S. Nejati, L. Briand, and A.-H. Mills. 2011. Using SysML for modeling of safety-critical software-hardware interfaces: Guidelines and industry experience. In Proceedings of the IEEE 13^th International Symposium on High-Assurance Systems Engineering (HASE'11).

Digital Library

[66]

W. Schafer and H. Wehrheim. 2007. The challenges of building advanced mechatronic systems. In Proceedings of the Conference on the Future of Software Engineering. IEEE Computer Society.

Digital Library

[67]

Software Engineering Institute, CMU. 2006. Ultra-large-scale systems: The software challenge of the future. http://www.sei.cmu.edu/library/assets/ULS_Book20062.pdf.

[68]

T. Thelin, P. Runeson, and C. Wohlin. 2003. An experimental comparison of usage-based and checklist-based reading. IEEE Trans. Softw. Engin. 29, 8.

Digital Library

[69]

L. Thomas. 1997. Retrospective power analysis. Conserv. Biol. 11, 1, 276--280.

[70]

F. Tip. 1994. A survey of program slicing techniques. J. Program. Lang. 3, 121--189.

[71]

A. Van Lamsweerde. 2008. Requirements Engineering - From System Goals to UML Models to Software Specifications. Wiley.

Digital Library

[72]

M. Weiser. 1984. Program slicing. IEEE Trans. Softw. Engin. 10, 4, 6.

Digital Library

[73]

C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslen. 2000. Experimentation in Software Engineering: An Introduction. Springer.

Digital Library

[74]

F. Yates. 1934. Contingency tables involving small numbers and the χ 2 test. J. Royal Statist. Soc. 1, 2.

Cited By

Kosch TWelsch RChuang LSchmidt A(2023)The Placebo Effect of Artificial Intelligence in Human–Computer InteractionACM Transactions on Computer-Human Interaction10.1145/352922529:6(1-32)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3529225
Wang QLiu JZhang JDou HSun HChen HChen XHe J(2023)Ont4Sys: Ontology-based tool of Semantic Representation and Verification for Traceability Models2023 27th International Conference on Engineering of Complex Computer Systems (ICECCS)10.1109/ICECCS59891.2023.00024(126-135)Online publication date: 14-Jun-2023
https://doi.org/10.1109/ICECCS59891.2023.00024
Wakimoto MMorisaki S(2022)A Metric for Questions and Discussions Identifying Concerns in Software ReviewsSoftware10.3390/software10300161:3(364-380)Online publication date: 5-Sep-2022
https://doi.org/10.3390/software1030016
Show More Cited By

Index Terms

Traceability and SysML design slices to support safety inspections: A controlled experiment
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

SysML as a Tool for Requirements Traceability in Mechatronic Design
ICMRE 2018: Proceedings of the 2018 4th International Conference on Mechatronics and Robotics Engineering

Modern mechatronic systems have to deal with the ever increasing requirements due to a higher number of demands and their complexity. Those demands are summarized in the requirements list. Classic design methodologies do not directly relate a system or ...
Integrating requirements analysis and safety analysis
RE '95: Proceedings of the Second IEEE International Symposium on Requirements Engineering

Summary form only given. In developing software for safety critical systems, it is necessary to carry out both requirements analysis and safety analysis. During requirements analysis, the behavioural and functional requirements of the system's software ...
Strategic Traceability for Safety-Critical Projects

To support any claim that a product is safe for its intended use, manufacturers must establish traceability within that product's development life cycle. Unfortunately, traceability information submitted to regulators and third parties is often weak, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Software Engineering and Methodology

ACM Transactions on Software Engineering and Methodology Volume 23, Issue 1

February 2014

354 pages

ISSN:1049-331X

EISSN:1557-7392

DOI:10.1145/2582050

Editor:
David S. Rosenblum

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 February 2014

Accepted: 01 April 2013

Revised: 01 April 2013

Received: 01 May 2012

Published in TOSEM Volume 23, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Norges Forskningsråd
Fonds National de la Recherche Luxembourg
DNV under the ModelME! project

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
785
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)5

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kosch TWelsch RChuang LSchmidt A(2023)The Placebo Effect of Artificial Intelligence in Human–Computer InteractionACM Transactions on Computer-Human Interaction10.1145/352922529:6(1-32)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3529225
Wang QLiu JZhang JDou HSun HChen HChen XHe J(2023)Ont4Sys: Ontology-based tool of Semantic Representation and Verification for Traceability Models2023 27th International Conference on Engineering of Complex Computer Systems (ICECCS)10.1109/ICECCS59891.2023.00024(126-135)Online publication date: 14-Jun-2023
https://doi.org/10.1109/ICECCS59891.2023.00024
Wakimoto MMorisaki S(2022)A Metric for Questions and Discussions Identifying Concerns in Software ReviewsSoftware10.3390/software10300161:3(364-380)Online publication date: 5-Sep-2022
https://doi.org/10.3390/software1030016
Cleland-Huang JAgrawal AVierhauser MMayr-Dorn C(2021)Visualizing Change in Agile Safety-Critical SystemsIEEE Software10.1109/MS.2020.300010438:3(43-51)Online publication date: May-2021
https://doi.org/10.1109/MS.2020.3000104
Alenazi MNiu NSavolainen JRothermel GBae D(2020)A novel approach to tracing safety requirements and state-based design modelsProceedings of the ACM/IEEE 42nd International Conference on Software Engineering10.1145/3377811.3380332(848-860)Online publication date: 27-Jun-2020
https://dl.acm.org/doi/10.1145/3377811.3380332
Wang FYang ZHuang ZLiu CZhou YBodeveix JFilali M(2020)An Approach to Generate the Traceability Between Restricted Natural Language Requirements and AADL ModelsIEEE Transactions on Reliability10.1109/TR.2019.293607269:1(154-173)Online publication date: Mar-2020
https://doi.org/10.1109/TR.2019.2936072
Kochanthara SRood NCleophas LDajsuren Yden Brand M(2020)Semi-automatic Architectural Suggestions for the Functional Safety of Cooperative Driving Systems2020 IEEE International Conference on Software Architecture Companion (ICSA-C)10.1109/ICSA-C50368.2020.00018(55-58)Online publication date: Mar-2020
https://doi.org/10.1109/ICSA-C50368.2020.00018
Hübner PPaech B(2020)Interaction-based creation and maintenance of continuously usable trace links between requirements and source codeEmpirical Software Engineering10.1007/s10664-020-09831-w25:5(4350-4377)Online publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1007/s10664-020-09831-w
Alenazi MNiu NSavolainen J(2019)SysML Modeling Mistakes and Their Impacts on Requirements2019 IEEE 27th International Requirements Engineering Conference Workshops (REW)10.1109/REW.2019.00010(14-23)Online publication date: Sep-2019
https://doi.org/10.1109/REW.2019.00010
Agrawal AKhoshmanesh SVierhauser MRahimi MCleland-Huang JLutz RAtlee JBultan TWhittle J(2019)Leveraging artifact trees to evolve and reuse safety casesProceedings of the 41st International Conference on Software Engineering10.1109/ICSE.2019.00124(1222-1233)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSE.2019.00124
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents