Article

An effective testing method for end-user programmers

Authors:

Zhi Quan ZhouAuthors Info & Claims

WEUSE I: Proceedings of the first workshop on End-user software engineering

Pages 1 - 5

https://doi.org/10.1145/1083231.1083236

Published: 21 May 2005 Publication History

Abstract

End-user programmers do not have extensive knowledge of various software testing methodologies used by professional testers. While they are creating the vast majority of software today, errors are pervasive in the programs due to the lack of testing techniques readily adoptable by end-user programmers. In this article we argue that the technique of metamorphic testing is both practical and effective for end-user programmers.

References

[1]

P. E. Ammann and J. C. Knight. Data diversity: an approach to software fault tolerance. IEEE Transactions on Computers, 37(4):418--425, 1988.

Digital Library

[2]

B. Beizer. Software Testing Techniques. Van Nostrand Reinhold, New York, 1990.

Digital Library

[3]

M. Blum and S. Kannan. Designing programs that check their work. In Proceedings of the 31st Annual ACM Symposium on Theory of Computing (STOC'89), pages 86--97. ACM Press, New York, 1989.

Digital Library

[4]

M. Blum and S. Kannan. Designing programs that check their work. Journal of the ACM, 42(1):269--291, 1995.

Digital Library

[5]

M. Blum, M. Luby, and R. Rubinfeld. Self-testing/correcting with applications to numerical problems. Journal of Computer and System Sciences, 47(3):549--595, 1993.

Digital Library

[6]

B. Boehm, C. Abts, A. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece. Software Cost Estimation with Cocomo II. Prentice Hall PTR, Upper Saddle River, NJ, 2000.

Digital Library

[7]

M. Burnett, C. Cook, and G. Rothermel. End-user software engineering. Communications of the ACM, 47(9):53--58, 2004.

Digital Library

[8]

T. Y. Chen, S. C. Cheung, and S. M. Yiu. Metamorphic testing: a new approach for generating next test cases. Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong, 1998.

[9]

T. Y. Chen, J. Feng, and T. H. Tse. Metamorphic testing of programs on partial differential equations: a case study. In Proceedings of the 26th Annual International Computer Software and Applications Conference (COMPSAC 2002), pages 327--333. IEEE Computer Society Press, Los Alamitos, California, 2002.

Digital Library

[10]

T. Y. Chen, D. H. Huang, T. H. Tse, and Z. Q. Zhou, Case studies on the selection of useful relations in metamorphic testing. In Proceedings of the 4th Ibero-American Symposium on Software Engineering and Knowledge Engineering (JIISIC 2004), pages 569--583, Madrid, Spain, 2004. Polytechnic University of Madrid.

[11]

T. Y. Chen, T. H. Tse, and Z. Q. Zhou. Semi-proving: an integrated method based on global symbolic evaluation and metamorphic testing. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2002), pages 191--195. ACM Press, New York, 2002.

Digital Library

[12]

T. Y. Chen, T. H. Tse, and Z. Q. Zhou. Fault-based testing without the need of oracles. Information and Software Technology, 45(1):1--9, 2003.

Digital Library

[13]

W. J. Cody, Jr and W. Waite. Software Manual for the Elementary Functions. Prentice Hall, Englewood Cliffs, New Jersey, 1980.

Digital Library

[14]

C. F. Gerald and P. O. Wheatley. Applied Numerical Analysis. Addison Wesley, Reading, Massachusetts, 1999.

[15]

A. Gotlieb. Exploiting symmetries to test programs. In Proceedings of the 14th International Symposium on Software Reliability Engineering (ISSRE 2003), pages 365--374, 2003.

Digital Library

[16]

A. Gotlieb and B. Botella. Automated metamorphic testing. In Proceedings of the 27th Annual International Computer Software and Applications Conference (COMPSAC 2003), pages 34--40. IEEE Computer Society Press, Los Alamitos, California, 2003.

Digital Library

[17]

G. J. Myers. The Art of Software Testing. Wiley, New York, 1979.

Digital Library

[18]

R. Panko. Finding spreadsheet errors. Information Week, Issue 529, page 100, May 1995.

[19]

G. Rothermel, M. Burnett, L. Li, C. Dupuis, and A. Sheretov. A methodology for testing spreadsheets. ACM Transactions on Software Engineering and Methodology, 10(1):110--147, 2001.

Digital Library

[20]

E. J. Weyuker. On testing non-testable programs. The Computer Journal, 25(4):465--470, 1982.

Cited By

Zhang YTowey DPike M(2024)Enabling Effective Metamorphic- Relation Generation by Novice Testers: A Pilot Study2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00384(2393-2398)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00384
Zhang YTowey DPike MCheng Han JQuan Zhou ZYin CWang QXie C(2024)Scenario‐Driven Metamorphic Testing for Autonomous Driving SimulatorsSoftware Testing, Verification and Reliability10.1002/stvr.189234:7Online publication date: 23-Jul-2024
https://doi.org/10.1002/stvr.1892
Yan BYecies BZhou ZXie XPoon PPullum L(2019)Metamorphic relations for data validationProceedings of the 4th International Workshop on Metamorphic Testing10.1109/MET.2019.00018(70-75)Online publication date: 26-May-2019
https://dl.acm.org/doi/10.1109/MET.2019.00018
Show More Cited By

Index Terms

An effective testing method for end-user programmers
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

An effective testing method for end-user programmers

End-user programmers do not have extensive knowledge of various software testing methodologies used by professional testers. While they are creating the vast majority of software today, errors are pervasive in the programs due to the lack of testing ...
Metamorphic Testing: A Simple Approach to Alleviate the Oracle Problem
SOSE '10: Proceedings of the 2010 Fifth IEEE International Symposium on Service Oriented System Engineering

The oracle problem is very common in the testing of service-oriented systems. Metamorphic testing has been proposed to alleviate the oracle problem in software testing. This talk aims at presenting the state of the art in metamorphic testing. It ...
On Testing Effectiveness of Metamorphic Relations: A Case Study
SSIRI '11: Proceedings of the 2011 Fifth International Conference on Secure Software Integration and Reliability Improvement

One fundamental challenge for software testing is the oracle problem, which means that either there does not exist a mechanism (called oracle) to verify the test output given any possible program input, or it is very expensive, if not impossible, to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WEUSE I: Proceedings of the first workshop on End-user software engineering

May 2005

65 pages

ISBN:1595931317

DOI:10.1145/1083231

ACM SIGSOFT Software Engineering Notes Volume 30, Issue 4
July 2005
1514 pages
ISSN:0163-5948
DOI:10.1145/1082983
Issue’s Table of Contents

Copyright © 2005 Copyright is held by the owner/author(s).

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 May 2005

Check for updates

Author Tags

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 20 of 21 submissions, 95%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
869
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YTowey DPike M(2024)Enabling Effective Metamorphic- Relation Generation by Novice Testers: A Pilot Study2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00384(2393-2398)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00384
Zhang YTowey DPike MCheng Han JQuan Zhou ZYin CWang QXie C(2024)Scenario‐Driven Metamorphic Testing for Autonomous Driving SimulatorsSoftware Testing, Verification and Reliability10.1002/stvr.189234:7Online publication date: 23-Jul-2024
https://doi.org/10.1002/stvr.1892
Yan BYecies BZhou ZXie XPoon PPullum L(2019)Metamorphic relations for data validationProceedings of the 4th International Workshop on Metamorphic Testing10.1109/MET.2019.00018(70-75)Online publication date: 26-May-2019
https://dl.acm.org/doi/10.1109/MET.2019.00018
Johnston OJarman DBerry JZhou ZChen TXie XPoon PPullum L(2019)Metamorphic relations for detection of performance anomaliesProceedings of the 4th International Workshop on Metamorphic Testing10.1109/MET.2019.00017(63-69)Online publication date: 26-May-2019
https://dl.acm.org/doi/10.1109/MET.2019.00017
Yang STowey DZhou Z(2019)Metamorphic Exploration of an Unsupervised Clustering Program2019 IEEE/ACM 4th International Workshop on Metamorphic Testing (MET)10.1109/MET.2019.00015(48-54)Online publication date: May-2019
https://doi.org/10.1109/MET.2019.00015
Wu CSun LZhou ZXie XPoon PPullum L(2019)The impact of a dotProceedings of the 4th International Workshop on Metamorphic Testing10.1109/MET.2019.00011(17-23)Online publication date: 26-May-2019
https://dl.acm.org/doi/10.1109/MET.2019.00011
Kanewala UYueh Chen T(2019)Metamorphic Testing: A Simple Yet Effective Approach for Testing Scientific SoftwareComputing in Science & Engineering10.1109/MCSE.2018.287536821:1(66-72)Online publication date: 1-Jan-2019
https://doi.org/10.1109/MCSE.2018.2875368
Chen TKuo FLiu HPoon PTowey DTse TZhou Z(2018)Metamorphic TestingACM Computing Surveys10.1145/314356151:1(1-27)Online publication date: 4-Jan-2018
https://dl.acm.org/doi/10.1145/3143561
Segura SFraser GSanchez ARuiz-Cortes A(2016)A Survey on Metamorphic TestingIEEE Transactions on Software Engineering10.1109/TSE.2016.253287542:9(805-824)Online publication date: 1-Sep-2016
https://doi.org/10.1109/TSE.2016.2532875
Zhou ZXiang SChen T(2016)Metamorphic Testing for Software Quality Assessment: A Study of Search EnginesIEEE Transactions on Software Engineering10.1109/TSE.2015.247800142:3(264-284)Online publication date: 1-Mar-2016
https://doi.org/10.1109/TSE.2015.2478001
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents