Article

Systematic testing for control applications

MEMOCODE '10: Proceedings of the Eighth ACM/IEEE International Conference on Formal Methods and Models for Codesign

Pages 1 - 10

https://doi.org/10.1109/MEMCOD.2010.5558629

Published: 01 July 2010 Publication History

Abstract

Software controllers for physical processes are at the core of many safety-critical systems such as avionics, automotive engine control, and process control. Despite their importance, the design and implementation of software controllers remains an art form; dependability is generally poor, and the cost of verifying systems is prohibitive. We illustrate the potential of applying program analysis tools on problems in controller design and implementation by focusing on concolic execution, a technique for systematic testing for software. In particular, we demonstrate how a concolic execution tool can be modified to automatically analyze controller implementations and (a) produce test cases achieving a coverage goal, (b) synthesize ranges for controller variables that can be used to allocate bits in a fixed-point implementation, and (c) verify robustness of an implementation under input uncertainties. We have implemented these algorithms on top of the Splat test generation tool and have carried out preliminary experiments on control software that demonstrates feasibility of the techniques.

References

[1]

B. Blanchet, P. Cousot, R. Cousot, J. Feret, L. Mauborgne, A. Miné, D. Monniaux, and X. Rival, "A Static Analyzer for Large Safety-Critical Software," in Proc. PLDI, 2003, pp. 196-207.

Digital Library

[2]

D. Delmas, E. Goubault, S. Putot, J. Souyris, K. Tekkal, and F. Védrine, "Towards an Industrial Use of FLUCTUAT on Safety-Critical Avionics Software," in Proc. FMICS, 2009, pp. 53-69.

Digital Library

[3]

R. Xu, "Symbolic Execution Algorithms for Test Generation," Ph.D. dissertation, University of California, Los Angeles, 2009.

Digital Library

[4]

P. Godefroid, N. Klarlund, and K. Sen, "DART: Directed Automated Random Testing." in Proc. PLDI, 2005, pp. 213-223.

Digital Library

[5]

K. Sen, D. Marinov, and G. Agha, "CUTE: A Cconcolic Unit Testing Engine for C," in Proc. FSE, 2005, pp. 263-272.

Digital Library

[6]

RTCA SC-167/EUROCAE WG-12, "DO178B/ED12B - Software Considerations in Airborne Systems and Equipment Certification, Washington D.C., RTCA Inc." 1992.

[7]

R. Majumdar and I. Saha, "Symbolic Robustness Analysis," in Proc. Real-Time Systems Symposium, 2009, pp. 355-364.

Digital Library

[8]

R. Moore, Interval Analysis. Prentice Hall, 1966.

[9]

The MathWorks, "Simulink 7.5 Demos," http://www.mathworks.com/products/simulink/demos.html.

[10]

P. Godefroid, M. Levin, and D. Molnar, "Active Property Checking," in Proc. EMSOFT 08, 2008, pp. 207-216.

Digital Library

[11]

C. Cadar, V. Ganesh, P. Pawlowski, D. Dill, and D. Engler, "EXE: Automatically Generating Inputs of Death," in Proc. CCS, 2006, pp. 322-335.

Digital Library

[12]

A. Brillout, D. Kroening, and T. Wahl, "Mixed Abstractions for Floating-point Arithmetic," in Proc. FMCAD 09, 2009, pp. 69-76.

[13]

M. Fränzle and C. Herde, "HySAT: An Efficient Proof Engine for Bounded Model Checking of Hybrid Systems," Formal Methods in System Design, vol. 30, no. 3, pp. 179-198, 2007.

Digital Library

[14]

G. Fainekos, S. Sankaranarayanan, F. Ivancic, and A. Gupta, "Robustness of Model-Based Simulations," in Proc. Real-Time Systems Symposium, 2009, pp. 345-354.

Digital Library

[15]

P. Godefroid, "Compositional Dynamic Test Generation," in Proc. POPL, 2007, pp. 47-54.

Digital Library

[16]

R. Xu, P. Godefroid, and R. Majumdar, "Testing for Buffer Overflows with Length Abstraction," in Proc. ISSTA, 2008, pp. 27-38.

Digital Library

[17]

G. C. Necula, S. McPeak, S. P. Rahul, and W. Weimer, "CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs," in Proc. CC, 2002, pp. 213-228.

Digital Library

[18]

SRI International, "Yices: An SMT Solver," http://yices.csl.sri.com/.

[19]

F. Rouillier and N. Revol, "MPFI 1.0," http://perso.enslyon.fr/nathalie.revol/mpfi.html.

[20]

LINDO Systems Inc., "LINDO API 6.0 - Powerful Library of Optimization Solvers and Mathematical Programming Tools," http://www.lindo.com/.

[21]

A. B. Kinsman and N. Nicolici, "Finite Precision Bit-width Allocation Using SAT-Modulo Theory," in Proc. DATE, 2009, pp. 1106-1111.

Digital Library

[22]

P. McLane, L. Peppard, and K. Sundareswaran, "Decentralized Feedback Controls for the Brakeless Operation of Multilocomotive Powered Trains," IEEE Transactions on Automatic Control, vol. 21, no. 3, pp. 358-363, 1976.

[23]

The MathWorks, "Real-Time Workshop 7.5," http://www.mathworks.com/products/rtw/.

[24]

Y. Li, K. H. Ang, and G. Chong, "PID Control System Analysis and Design," IEEE Control Systems Magazine, pp. 559-576, 2006.

[25]

D. Liberzon, Switching in Systems and Control. Birkhäuser, 2003.

[26]

"The Interproc Analyzer," http://popart.inrialpes.fr/interproc/interprocweb.cgi.

[27]

O. Bouissou, E. Goubault, S. Putot, K. Tekkal, and F. Védrine, "HybridFluctuat: A Static Analyzer of Numerical Programs within a Continuous Environment," in Proc. CAV, 2009, pp. 620-626.

Digital Library

[28]

P. Godefroid and J. Kinder, "Proving Memory Safety of Floating-Point Computations by Combining Static and Dynamic Program Analysis," in Proc. ISSTA, 2010.

Digital Library

[29]

A. Kanade, R. Alur, F. Ivancic, S. Ramesh, S. Sankaranarayanan, and K. C. Shashidhar, "Generating and Analyzing Symbolic Traces of Simulink/Stateflow Models," in Proc. CAV, 2009, pp. 430-445.

Digital Library

[30]

M. Satpathy, A. Yeolekar, and S. Ramesh, "Randomized Directed Testing (REDIRECT) for Simulink/Stateflow Models," in Proc. EMSOFT, 2008, pp. 217-226.

Digital Library

[31]

S. Xia, B. D. Vito, and C. Muñoz, "Towards Automated Test Generation for Engineering Applications," in Proc. ASE, 2005, pp. 283-286.

Digital Library

[32]

S. Xia, B. D. Vito, and C. Muñoz, "Predicate Abstraction of Programs with Non-linear Computation," in Proc. ATVA, 2006, pp. 352-368.

Digital Library

[33]

B. Botella and C. Gotlieb, A. ans Michel, "Symbolic Execution of Floating-point Computations," Softw. Test., Verif. Reliab., vol. 16, pp. 97-121, 2006.

Digital Library

[34]

D. Lee, A. A. Gaffar, R. C. C. Cheung, O. Mencer, W. Luk, and G. A. Constantinides, "Accuracy-Guaranteed Bit-width Optimization," IEEE Trans. on CAD of Integrated Circuits and Systems, vol. 25, no. 10, pp. 1990-2000, 2006.

Digital Library

[35]

J. A. López, C. Carreras, and O. Nieto-Taladriz, "Improved Interval-based Characterization of Fixed-point LTI Systems with Feedback Loops," IEEE Trans. on CAD of Integrated Circuits and Systems, vol. 26, no. 11, pp. 1923-1932, 2007.

Digital Library

[36]

W. G. Osborne, R. C. C. Cheung, J. G. F. Coutinho, W. Luk, and O. Mencer, "Automatic Accuracy-Guaranteed Bit-width Optimization for Fixed and Floating-point Systems," in Proc. FPL, 2007, pp. 617-620.

[37]

P. Belanovic and M. Rupp, "Automated Floating-point to Fixed-point Conversion with the Fixify Environment," in Proc. International Workshop on Rapid System Prototyping, 2005, pp. 172-178.

Digital Library

[38]

A. Mallik, D. Sinha, P. Banerjee, and H. Zhou, "Low-Power Optimization by Smart Bit-width Allocation in a SystemC-based ASIC Design Environment," IEEE Trans. on CAD of Integrated Circuits and Systems, vol. 26, no. 3, pp. 447-455, 2007.

Digital Library

[39]

J. Stolfi and L. H. Figueiredo, "Self-validated Numerical Methods and Applications," in Monograph for 21st Brazilian Mathematics Colloquium, Rio de Janeiro: IMPA, 1997.

[40]

C. F. Fang, R. A. Rutenbar, and T. Chen, "Fast, Accurate Static Analysis for Fixed-point Finite-precision Effects in DSP Designs," in Proc. ICCAD, 2003, pp. 275-282.

Digital Library

[41]

B. Jeannet and A. Miné, "Apron: A Library of Numerical Abstract Domains for Static Analysis," in Proc. CAV, 2009, pp. 661-667.

Digital Library

[42]

S. Chaudhuri, S. Gulwani, and R. Lublinerman, "Continuity Analysis of Programs," in Proc. POPL, 2010, pp. 57-70.

Digital Library

Cited By

Darulova EKuncak V(2017)Towards a Compiler for RealsACM Transactions on Programming Languages and Systems10.1145/301442639:2(1-28)Online publication date: 10-Mar-2017
https://dl.acm.org/doi/10.1145/3014426
Darulova EKuncak V(2014)Sound compilation of realsACM SIGPLAN Notices10.1145/2578855.253587449:1(235-248)Online publication date: 8-Jan-2014
https://dl.acm.org/doi/10.1145/2578855.2535874
Darulova EKuncak VJagannathan SSewell P(2014)Sound compilation of realsProceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages10.1145/2535838.2535874(235-248)Online publication date: 11-Jan-2014
https://dl.acm.org/doi/10.1145/2535838.2535874
Show More Cited By

Systematic testing for control applications
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

Systematic analysis and design of fuzzy logic controllers for process control
Lazy Systematic Unit Testing: JWalk versus JUnit
TAICPART-MUTATION '07: Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION

Lazy systematic unit testing with JWalk is compared against regression testing with JUnit, the leading agile testing tool. JWalk produced exhaustive test sets more quickly and recalculated full state and transition coverage, when testing modified or ...
Systematic Design Method of Controller using Discrete Fuzzy PI Control and Particle Swarm Optimization
2018 IEEE 14th International Conference on Automation Science and Engineering (CASE)
This paper presents a systematic design method of a controller using the discrete fuzzy PI control and the particle swarm optimization (PSO). Unlike a conventional PI controller, the discrete fuzzy PI controller has variable gains according to its input ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MEMOCODE '10: Proceedings of the Eighth ACM/IEEE International Conference on Formal Methods and Models for Codesign

July 2010

218 pages

ISBN:9781424478866

Copyright © 2010.

Sponsors

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 July 2010

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 34 of 82 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
12
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Darulova EKuncak V(2017)Towards a Compiler for RealsACM Transactions on Programming Languages and Systems10.1145/301442639:2(1-28)Online publication date: 10-Mar-2017
https://dl.acm.org/doi/10.1145/3014426
Darulova EKuncak V(2014)Sound compilation of realsACM SIGPLAN Notices10.1145/2578855.253587449:1(235-248)Online publication date: 8-Jan-2014
https://dl.acm.org/doi/10.1145/2578855.2535874
Darulova EKuncak VJagannathan SSewell P(2014)Sound compilation of realsProceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages10.1145/2535838.2535874(235-248)Online publication date: 11-Jan-2014
https://dl.acm.org/doi/10.1145/2535838.2535874
(2014)Formal property verification in a conformance testing frameworkProceedings of the Twelfth ACM/IEEE Conference on Formal Methods and Models for Codesign10.1109/MEMCOD.2014.6961854(155-164)Online publication date: 1-Oct-2014
https://dl.acm.org/doi/10.1109/MEMCOD.2014.6961854
Mohalik SGadkari AYeolekar AShashidhar KRamesh S(2014)Automatic test case generation from Simulink/Stateflow models using model checkingSoftware Testing, Verification & Reliability10.1002/stvr.148924:2(155-180)Online publication date: 1-Mar-2014
https://dl.acm.org/doi/10.1002/stvr.1489
Barr EVo TLe VSu Z(2013)Automatic detection of floating-point exceptionsACM SIGPLAN Notices10.1145/2480359.242913348:1(549-560)Online publication date: 23-Jan-2013
https://dl.acm.org/doi/10.1145/2480359.2429133
Barr EVo TLe VSu ZGiacobazzi RCousot R(2013)Automatic detection of floating-point exceptionsProceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages10.1145/2429069.2429133(549-560)Online publication date: 23-Jan-2013
https://dl.acm.org/doi/10.1145/2429069.2429133
Zhu ZMisailovic SKelner JRinard MField JHicks M(2012)Randomized accuracy-aware program transformations for efficient approximate computationsProceedings of the 39th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages10.1145/2103656.2103710(441-454)Online publication date: 25-Jan-2012
https://dl.acm.org/doi/10.1145/2103656.2103710
Zhu ZMisailovic SKelner JRinard M(2012)Randomized accuracy-aware program transformations for efficient approximate computationsACM SIGPLAN Notices10.1145/2103621.210371047:1(441-454)Online publication date: 25-Jan-2012
https://dl.acm.org/doi/10.1145/2103621.2103710
Chaudhuri SGulwani SLublinerman RNavidpour SGyimóthy TZeller A(2011)Proving programs robustProceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering10.1145/2025113.2025131(102-112)Online publication date: 5-Sep-2011
https://dl.acm.org/doi/10.1145/2025113.2025131

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents