research-article

High Performance Software Testing on SIMD Machines

Authors:

Edward W. Krauser,

Aditya P. Mathur,

Vernon J. RegoAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 17, Issue 5

Pages 403 - 423

https://doi.org/10.1109/32.90444

Published: 01 May 1991 Publication History

Abstract

A method for high-performance, software testing, called mutant unification, is described. The method is designed to support program mutation on parallel machines based on the single instruction multiple data stream (SIMD) paradigm. Several parameters that affect the performance of unification have been identified and their effect on the time to completion of a mutation test cycle and speedup has been studied. Program mutation analysis provides an effective means for determining the reliability of large software systems and a systematic method for measuring the adequacy of test data. However, it is likely that testing large software systems using mutation is computation bound and prohibitive on traditional sequential machines. Current, implementations of mutation tools are unacceptably slow and are only suitable for testing relatively small programs. The proposed unification method provides a practical alternative to the current approaches. The method also opens up a new application domain for SIMD machines.

References

[1]

{1} A. T. Acree, R. A. DeMillo, T. A. Budd, and F. G. Sayward, "Mutation analysis," Georgia Inst. Technol., Atlanta, Tech. Rep. GIT-ICS-79/08, 1979.]]

[2]

{2} H. Agrawal, R.A. DeMillo, B. Hathaway, W. Hsu, W. Hsu, E. W. Krauser, R.J. Martin, A.P. Mathur, and E.H. Spafford, "Design of mutant operators for the C programming language," Software Eng. Research Center, Purdue Univ., West Lafayette, IN, Tech. Rep. SERCTR-41-P, 1989.]]

[3]

{3} A.V. Aho, R. Sethi, and J.D. Ullman, Compilers: Principles, Techniques, and Tools. Reading, MA: Addison-Wesley, 1986.]]

Digital Library

[4]

{4} G. H. Barnes, R. Brown, M. Kato, D. J. Kuck, D. L. Slotnick, and R. A. Stokes, "The ILLIAC IV computer," IEEE Trans. Comput., vol. C-17, pp. 746-757, Aug. 1968.]]

[5]

{5} K. E. Batcher, "Design of a massively parallel processor," IEEE Trans. Comput., vol. C-29, no. 9, pp. 836-840, 1980.]]

[6]

{6} B. Beizer, Software Testing Techniques. New York: Van Nostrand Reinhold, 1983.]]

Digital Library

[7]

{7} R. Bellman, R.E. Kalaba, and J.A. Lockett, Numerical Inversion of Laplace Transform. New York: Elsevier, 1966.]]

[8]

{8} B. J. Choi, R.A. DeMillo, E. W. Krauser, A. P. Mathur, R. J. Martin, A. J. Offutt, H. Pan, and E. H. Spafford, "The Mothra toolset," in Proc. Hawaii Int. Conf. on System Sciences, Jan. 3-6, 1989.]]

[9]

{9} B. J. Choi, A. P. Mathur, and B. Pattison, "Architecture of P. Mothra: A tool for mutation based testing on the hypercube," in Proc. 3rd Symp. Software Testing, Analysis, and Verification, Key West, FL, Dec. 1989.]]

[10]

{10} R.A. DeMillo, W. M. McCracken, R. J. Martin, and J. F. Passafume, Software Testing and Evaluation. Menlo Park, CA: Benjamin/Cummings, 1987.]]

Digital Library

[11]

{11} R. A. DeMillo, R. J. Lipton, and F. G. Sayward, "Hints on test data selection: Help for the practicing programmer," Computer, vol. 11, no. 4, Apr. 1978.]]

[12]

{12} R. A. DeMillo and A. P. Mathur, "On the use of software artifacts to evaluate the effectiveness of mutation analysis for detecting errors in production software," in Proc. 13th Minnowbrook Workshop Software Engineering, Blue Mountain Lake, NY, July 24-27, 1990.]]

[13]

{13} W. R. Elmendorf, "Cause-effect graphs in functional testing," ACM Comput. Surveys, vol. 8, no. 3, Sept. 1976.]]

[14]

{14} J. P. Hayes, T. N. Mudge, Q. F. Stout, S. Colley, and J. Palmer, "A microprocessor based hypercube supercomputer," IEEE Micro., pp. 6-17, Oct. 1986.]]

Digital Library

[15]

{15} W. D. Hillis, The Connection Machine. Cambridge, MA: MIT Press, 1985.]]

Digital Library

[16]

{16} W. D. Hillis and G. L. Steele, Jr., "Data parallel algorithms," Commun. ACM, vol. 29, no. 12, pp. 1170-1183, Dec. 1986.]]

Digital Library

[17]

{17} J. Hawgood, Numerical Methods in Algol. New York: McGraw-Hill, 1965.]]

[18]

{18} R. W. Hockney and C. Jesshope, Parallel Computers: Architecture, Programming and Algorithms. Bristol, UK: Adam Hilger, 1981.]]

Digital Library

[19]

{19} K. Hwang and F. A. Briggs, Computer Architecture and Parallel Processing . New York: McGraw-Hill, 1984.]]

Digital Library

[20]

{20} W.E. Howden, "A survey of static analysis methods," in Tutorial: Software Testing and Validation Techniques. New York: IEEE, 1978.]]

[21]

{21} W.E. Howden, "Weak mutation testing and completeness of test sets," IEEE Trans. Software Eng., vol. SE-8, July 1982.]]

Digital Library

[22]

{22} W.E. Howden, Functional Program Testing and Analysis. New York: McGraw-Hill, 1987.]]

Digital Library

[23]

{23} D.E. Knuth, T<inf>E</inf>X : The Program. Reading, MA: Addison-Wesley, 1986.]]

Digital Library

[24]

{24} D.E. Knuth, "The errors of T<inf>E</inf>X," Software Practice and Experience, vol. 19, no. 7, pp. 607-685, July 1989.]]

Digital Library

[25]

{25} E. W. Krauser and A. P. Mathur, "Program testing on a massively parallel transputer-based system," in Proc. ISMM Int. Symp. Mini and Microcomputers and their Applications, Austin, TX, Nov. 1986, pp. 67-71.]]

[26]

{26} E. W. Krauser, A. P. Mathur, and V. J. Rego, "High performance testing on SIMD machines," Software Eng. Res. Center, Purdue Univ., West Lafayette, IN, Tech. Rep. SERC-TR-17-P (revised), 1990.]]

Digital Library

[27]

{27} D. J. Kuck et al., "The effects of program restructuring, algorithm change, and architecture choice on program performance," in Proc. Int. Conf. Parallel Processing, 1984, pp. 129-138.]]

[28]

{28} R. J. Martin, "Mothra uses mutation analysis," IEEE Software (Tools Fair Issue), p. 56, May 1990.]]

[29]

{29} A. P. Mathur and E. Galiano, "Inducing vectorization: A formal analysis," in Proc. 3rd Int. Conf. Supercomputing, vol. III, Boston, MA, 1988, pp. 455-463.]]

[30]

{30} A. P. Mathur and E. W. Krauser, "Modeling mutation on a vector processor," in Proc. Int. Conf. Software Engineering, Singapore, Apr. 1988.]]

Digital Library

[31]

{31} A. J. Offutt, "Automatic test data generation," Software Eng. Res. Center, Purdue Univ., West Lafayette, IN, Tech. Rep. SERC-TR-25-P, 1986.]]

[32]

{32} V. J. Rego and A. P. Mathur, "Concurrency enhancement through program unification: A performance analysis," J. Parallel Distributed Comput., vol. 8, pp. 210-217, Mar. 1990.]]

Digital Library

[33]

{33} V. J. Rego and A. P. Mathur, "Exploiting parallelism across program execution: A unification technique and its analysis," IEEE Trans. Parallel Distributed Syst., vol. 1, pp. 399-414, Oct. 1990.]]

Digital Library

[34]

{34} H. J. Siegel, L. J. Siegel, F. C. Kemmerer, P.T. Mueller Jr., H.E. Smalley, and S. D. Smith, "PASM: Partitionable SIMD/MIMD system for image processing and pattern recognition," IEEE Trans. Comput., vol. C-30, pp. 934-947, Dec. 1981.]]

[35]

{35} System Architecture and Implementation, MasPar Computer Corp., Sunnyvale, CA, 1990.]]

[36]

{36} P. J. Walsh, "A measure of test case completeness," Ph.D. dissert., State Univ. New York, Binghamton, 1985.]]

Digital Library

[37]

{37} N. Wirth, Algorithms+Data Structures=Program. Englewood Cliffs, NJ: Prentice-Hall, 1976.]]

Digital Library

[38]

{38} M.R. Woodward and K. Halewood, "From weak to strong, dead or alive? An analysis of some mutation testing issues," in Proc. 2nd Workshop Software Testing, Verification, and Analysis, Banff, AB, Canada, July 1988.]]

Cited By

Cañizares PNúñez AFilgueira Rde Lara J(2023)Parallel mutation testing for large scale systemsCluster Computing10.1007/s10586-023-04074-y27:2(2071-2097)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1007/s10586-023-04074-y
Liu ZYang XLiu YZhao YLi X(2021)ImReMuDFScientific Programming10.1155/2021/75438962021Online publication date: 21-Oct-2021
https://dl.acm.org/doi/10.1155/2021/7543896
Zhang JZhang LHarman MHao DJia YZhang L(2019)Predictive Mutation TestingIEEE Transactions on Software Engineering10.1109/TSE.2018.280949645:9(898-918)Online publication date: 16-Sep-2019
https://dl.acm.org/doi/10.1109/TSE.2018.2809496
Show More Cited By

Index Terms

High Performance Software Testing on SIMD Machines
1. Computer systems organization
  1. Architectures
    1. Parallel architectures
      1. Single instruction, multiple data
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Assessing Testing Tools in Research and Education

An evaluation of three software engineering tools based on their use in research and educational environments is presented. The three testing tools are Mothra, a mutation-testing tool, Asset, a dataflow testing tool, and ATAC, a dataflow testing tool. ...
Priority based data flow testing
ICSM '95: Proceedings of the International Conference on Software Maintenance

Software testing is an expensive component of software development and maintenance. For data flow testing, test cases must be found to test the def-use pairs in a program. Since some of the def-use pairs identified through static analysis may be ...
Test Selection Based on Communicating Nondeterministic Finite-State Machines Using a Generalized Wp-Method

Presents a method of generating test sequences for concurrent programs and communication protocols that are modeled as communicating nondeterministic finite-state machines (CNFSMs). A conformance relation, called trace-equivalence, is defined within ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 17, Issue 5

May 1991

104 pages

ISSN:0098-5589

Editor:
Victor R. Basili
Univ. Maryland, College Park

Issue’s Table of Contents

Copyright © Copyright © 1991 IEEE. All Rights Reserved.

Publisher

IEEE Press

Publication History

Published: 01 May 1991

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cañizares PNúñez AFilgueira Rde Lara J(2023)Parallel mutation testing for large scale systemsCluster Computing10.1007/s10586-023-04074-y27:2(2071-2097)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1007/s10586-023-04074-y
Liu ZYang XLiu YZhao YLi X(2021)ImReMuDFScientific Programming10.1155/2021/75438962021Online publication date: 21-Oct-2021
https://dl.acm.org/doi/10.1155/2021/7543896
Zhang JZhang LHarman MHao DJia YZhang L(2019)Predictive Mutation TestingIEEE Transactions on Software Engineering10.1109/TSE.2018.280949645:9(898-918)Online publication date: 16-Sep-2019
https://dl.acm.org/doi/10.1109/TSE.2018.2809496
Qin XLiu STao Z(2019)A New Mutant Generation Algorithm Based on Basic Path Coverage for Mutant ReductionStructured Object-Oriented Formal Language and Method10.1007/978-3-030-41418-4_13(167-186)Online publication date: 5-Nov-2019
https://dl.acm.org/doi/10.1007/978-3-030-41418-4_13
Wang BXiong YShi YZhang LHao DBultan TSen K(2017)Faster mutation analysis via equivalence modulo statesProceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3092703.3092714(295-306)Online publication date: 10-Jul-2017
https://dl.acm.org/doi/10.1145/3092703.3092714
Zhang JWang ZZhang LHao DZang LCheng SZhang LZeller ARoychoudhury A(2016)Predictive mutation testingProceedings of the 25th International Symposium on Software Testing and Analysis10.1145/2931037.2931038(342-353)Online publication date: 18-Jul-2016
https://dl.acm.org/doi/10.1145/2931037.2931038
Gopinath RJensen CGroce ADillon LVisser WWilliams L(2016)Topsy-TurvyProceedings of the 38th International Conference on Software Engineering Companion10.1145/2889160.2892655(740-743)Online publication date: 14-May-2016
https://dl.acm.org/doi/10.1145/2889160.2892655
Gopinath RJensen CGroce ADillon LVisser WWilliams L(2016)Topsy-TurvyProceedings of the 38th International Conference on Software Engineering Companion10.1145/2884781.2892655(740-743)Online publication date: 14-May-2016
https://dl.acm.org/doi/10.1145/2884781.2892655
Caizares PMerayo MNez A(2016)EMINENTProcedia Computer Science10.1016/j.procs.2016.05.29880:C(63-73)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1016/j.procs.2016.05.298
Mateo PUsaola M(2015)Reducing mutation costs through uncovered mutantsSoftware Testing, Verification & Reliability10.1002/stvr.153425:5-7(464-489)Online publication date: 1-Aug-2015
https://dl.acm.org/doi/10.1002/stvr.1534
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents