research-article

On the measurement of software complexity for plc industrial control systems using TIQVA

Authors:

Eduard EnoiuAuthors Info & Claims

SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing

Pages 1556 - 1565

https://doi.org/10.1145/3341105.3373914

Published: 30 March 2020 Publication History

Abstract

In the safety-critical domain (e.g. transportation, nuclear, aerospace and automotive), large-scale embedded systems implemented using Programmable Logic Controllers (PLCs) are widely used to provide supervisory control. Software complexity metrics, such as code size and cyclomatic complexity, have been used in the software engineering community for predicting quality metrics such as maintainability, bug proneness and robustness. However, since there is no available approach and tool support for measuring software complexity of PLC programs, we developed a tool called TIQVA in an effort to measure complexity for this type of software. We show how to measure different software complexity metrics such as lines of code, cyclomatic complexity, and information flow for a popular PLC programming language named Function Block Diagram (FBD). We evaluate the tool using data provided by Bombardier Transportation from a Train Control Management System (TCMS). In addition, we report some empirical and industrial evidence showing how TIQVA can be used to provide some experimental evidence to support the use of these metrics to estimate testing effort for an industrial control software. The results from this evaluation indicate that other specific dimensions of PLC programs (e.g., function block relationships, block coupling and timing) could be used to improve the measurement of complexity for industrial embedded software.

References

[1]

[n. d.]. Eclipse - The Eclipse Foundation open source community website. https://eclipse.org/. (Accessed on 05/15/2017).

[2]

[n. d.]. Home page of Beremiz. http://www.beremiz.org/. (Accessed on 05/15/2017).

[3]

[n. d.]. Index - KW-Software | EN. http://www.kw-software.com/en/. (Accessed on 05/15/2017).

[4]

[n. d.]. IntelliJ IDEA the Java IDE. https://www.jetbrains.com/idea/. (Accessed on 05/15/2017).

[5]

[n. d.]. Kendall Rank Coefficient | R Tutorial. http://www.r-tutor.com/gpu-computing/correlation/kendall-rank-coefficient. (Accessed on 04/27/2017).

[6]

[n. d.]. Maven âĂŞ Welcome to Apache Maven. https://maven.apache.org/. (Accessed on 05/15/2017).

[7]

[n. d.]. scikit-learn: machine learning in Python - scikit-learn 0.18.1 documentation. http://scikit-learn.org/stable/index.html. (Accessed on 04/29/2017).

[8]

Alain Abran. 2010. Software metrics and software metrology. John Wiley & Sons.

Digital Library

[9]

Victor R Basili and Barry T Perricone. 1984. Software errors and complexity: an empirical investigation0. Commun. ACM 27, 1 (1984), 42--52.

Digital Library

[10]

William Bolton. 2015. Programmable logic controllers. Newnes.

[11]

William Bolton. 2015. Programmable logic controllers. Newnes.

[12]

David N Card and Robert L Glass. 1990. Measuring software design quality. Prentice-Hall, Inc.

[13]

CENELEC. 2001. 50128: Railway Application: Communications, Signaling and Processing Systems, Software For Railway Control and Protection Systems. In Standard Official Document. European Committee for Electrotechnical Standardization.

[14]

Jacob Cohen. 1988. Statistical power analysis for the behavioral sciences. 2nd.

[15]

Reidar Conradi and Alf Inge Wang. 2003. Empirical methods and studies in software engineering: experiences from ESERNET. Vol. 2765. Springer.

[16]

NE Fenton. 1996. eeger, Software Metrics, A Rigorous and Practical Approach.

[17]

John E Gaffney. 1984. Estimating the number of faults in code. IEEE Transactions on Software Engineering 4 (1984), 459--464.

Digital Library

[18]

Maurice Howard Halstead. 1977. Elements of software science. Vol. 7. Elsevier New York.

Digital Library

[19]

Sallie Henry and Dennis Kafura. 1981. Software structure metrics based on information flow. IEEE transactions on Software Engineering 5 (1981), 510--518.

Digital Library

[20]

Karl-Heinz John and Michael Tiegelkamp. 2010. IEC 61131-3: programming industrial automation systems: concepts and programming languages, requirements for programming systems, decision-making aids. Springer Science & Business Media.

[21]

Dennis Kafura and Geereddy R. Reddy. 1987. The use of software complexity metrics in software maintenance. IEEE Transactions on Software Engineering 3 (1987), 335--343.

Digital Library

[22]

Maurice George Kendall. 1948. Rank correlation methods. (1948).

[23]

Kostas Kevrekidis, Stijn Albers, Peter JM Sonnemans, and Guillaume M Stollman. 2009. Software complexity and testing effectiveness: An empirical study. In Reliability and Maintainability Symposium. IEEE, 539--543.

[24]

Andrei N Kolmogorov. 1963. On tables of random numbers. Sankhyā: The Indian Journal of Statistics, Series A (1963), 369--376.

[25]

Lov Kumar, Raoul Jetley, and Ashish Sureka. 2016. Source code metrics for programmable logic controller (PLC) ladder diagram (LD) visual programming language. In International Workshop on Emerging Trends in Software Metrics. IEEE, 15--21.

Digital Library

[26]

Hareton KN Leung and Lee White. 1991. A cost model to compare regression test strategies. In Proceedings. Conference on Software Maintenance. IEEE, 201--208.

[27]

Thomas J McCabe. 1976. A complexity measure. IEEE Transactions on software Engineering 4 (1976), 308--320.

Digital Library

[28]

Thomas J McCabe and Charles W Butler. 1989. Design complexity measurement and testing. Commun. ACM 32, 12 (1989), 1415--1425.

Digital Library

[29]

Carma L McClure. 1978. A model for program complexity analysis. In Proceedings of the 3rd international conference on Software engineering. IEEE Press, 149--157.

[30]

Tom Mens. 2016. Research trends in structural software complexity. arXiv preprint arXiv:1608.01533 (2016).

[31]

Douglas C Montgomery, Elizabeth A Peck, and G Geoffrey Vining. 2015. Introduction to linear regression analysis. John Wiley & Sons.

Digital Library

[32]

Adnan Muslija. 2019. Tiqva - FBD complexity measurement tool. https://github.com/amuslija/fbd-complexity-tool

[33]

Marta Olszewska, Yanja Dajsuren, Harald Altinger, Alexander Serebrenik, Marina Waldén, and Mark GJ van den Brand. 2016. Tailoring complexity metrics for simulink models. In Proceedings of the 10th European Conference on Software Architecture Workshops. ACM, 5.

[34]

Thomas J Ostrand, Elaine J Weyuker, and Robert M Bell. 2005. Predicting the location and number of faults in large software systems. IEEE Transactions on Software Engineering 31, 4 (2005), 340--355.

Digital Library

[35]

Shari Lawrence Pfleeger, Ross Jeffery, Bill Curtis, and Barbara Kitchenham. 1997. Status Report on Software Measurement. IEEE Software 14, 2 (1997), 33--43.

Digital Library

[36]

Mehwish Riaz, Emilia Mendes, and Ewan Tempero. 2009. A systematic review of software maintainability prediction and metrics. In Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement. IEEE Computer Society, 367--377.

Digital Library

[37]

Claude E Shannon. 1951. Prediction and entropy of printed English. Bell Labs Technical Journal 30, 1 (1951), 50--64.

[38]

Martin Shepperd. 1988. A critique of cyclomatic complexity as a software metric. Software Engineering Journal 3, 2 (1988), 30--36.

Digital Library

[39]

Keith Stouffer, Joe Falco, and Karen Scarfone. 2011. Guide to industrial control systems (ICS) security. NIST special publication 800, 82 (2011), 16--16.

Digital Library

[40]

Arthur Henry Watson, Dolores R Wallace, and Thomas J McCabe. 1996. Structured testing: A testing methodology using the cyclomatic complexity metric. Vol. 500. US Department of Commerce, Technology Administration, National Institute of Standards and Technology.

[41]

Elaine J. Weyuker. 1988. Evaluating software complexity measures. IEEE transactions on Software Engineering 14, 9 (1988), 1357--1365.

Digital Library

Cited By

Fischer JVogel-Heuser BSchneider HLanger NFelger MBengel M(2021)Measuring the Overall Complexity of Graphical and Textual IEC 61131-3 Control SoftwareIEEE Robotics and Automation Letters10.1109/LRA.2021.30848866:3(5784-5791)Online publication date: Jul-2021
https://doi.org/10.1109/LRA.2021.3084886

Index Terms

On the measurement of software complexity for plc industrial control systems using TIQVA

Index terms have been assigned to the content through auto-classification.

Recommendations

Software complexity measurement using multiple criteria

One of the challenges in software development is measuring and estimating software complexity at various levels of development. It is difficult to manage a software project without having well-developed measuring techniques. The term 'software ...
Software metrics: using measurement theory to describe the properties and scales of static software complexity metrics

Over the last decade many software metrics have been introduced by researchers and many software tools have been developed using software metrics to measure the "quality" of programs. These metrics for measuring productivity, reliability, ...
The Research on Software Metrics and Software Complexity Metrics
IFCSTA '09: Proceedings of the 2009 International Forum on Computer Science-Technology and Applications - Volume 01

The process of software development, including documentation, design, program, test, and maintenance can be measured statistically. Therefore the quality of software can be monitored efficiently. Software metrics is very important in research of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing

March 2020

2348 pages

ISBN:9781450368667

DOI:10.1145/3341105

Conference Chairs:
Chih-Cheng Hung
Kennesaw State University
,
Tomas Cerny
Baylor University
,
Program Chairs:
Dongwan Shin
New Mexico Tech
,
Alessio Bechini
University of Pisa, Italy

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Vinnova
ECSEL

Conference

SAC '20

Sponsor:

SIGAPP

SAC '20: The 35th ACM/SIGAPP Symposium on Applied Computing

March 30 - April 3, 2020

Brno, Czech Republic

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
156
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)4

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fischer JVogel-Heuser BSchneider HLanger NFelger MBengel M(2021)Measuring the Overall Complexity of Graphical and Textual IEC 61131-3 Control SoftwareIEEE Robotics and Automation Letters10.1109/LRA.2021.30848866:3(5784-5791)Online publication date: Jul-2021
https://doi.org/10.1109/LRA.2021.3084886

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents