Article

Usage-based statistical testing of web applications

Authors:

Emilia MendesAuthors Info & Claims

ICWE '06: Proceedings of the 6th international conference on Web engineering

Pages 17 - 24

https://doi.org/10.1145/1145581.1145585

Published: 11 July 2006 Publication History

Abstract

The large growth in the number of large Web applications currently developed brings concerns related to their quality, and more specifically their testing and reliability. Web application testing is still in its infancy and relies mostly upon traditional software coverage testing processes, which are highly impractical. Recent studies have looked at usage-based statistical models for testing Web applications. One of these studies, conducted by Kallepalli and Tian [8], used Unified Markov Models (UMMs), built from Web server access logs, as basis for test case selection. In addition, server error logs were also used to measure a Web application's reliability, and consequently to investigate the effectiveness of UMMs as a suitable testing mechanism. This paper describes two experiments that replicated Kallepalli and Tian's work. Our results showed that, in contrast to [8], multiple set UMMs were needed for trustworthy test case generation. In addition, our reliability assessment corroborated results from [8], confirming that UMMs seem to be a suitable testing mechanism to use to test Web applications.

References

[1]

Carroll, C. T. The cost of poor testing: A U.S. government study (part 2). EDPACS, The EDP Audit, Control and Security Newslstter. 31, 2, 1--16, 2003.

[2]

Cheung, R. C. A user-oriented software reliability model. IEEE Trans. Software Eng., vol. SE-6, Mar. 1980.

Digital Library

[3]

Chillarege, R. Software Testing Best Practices. IBM Research Technical Report. http://www.chillarege.com/authwork/papers1990s/TestingBestPractice.pdf (last accessed) 2004.

[4]

Currit, P. A., M. Dyer, H. D. Mills. Certifying the correctness of software. IEEE Trans. Software Eng., vol. SE-12, no. 1, pp. 3--11, Jan. 1986.

Digital Library

[5]

Elbaum, S., S. Karre, G. Rothermel. Improving web application testing with user session data. Proc. of the 25th Int'l Conf. on Software Engineering, pp. 49--59, May, 2003.

Digital Library

[6]

Gouraud, S. D., A. Denise, M. C. Gaudel, B. Marre. A new way of automating statistical testing methods. IEEE Software, 2001.

Digital Library

[7]

Hamlet, R. Testing software for software reliability. Tech. Rep. TR 91.2, rev. 1, Dept. of Comput. Sci., Portland State Univ., Portland, OR, USA, Mar. 1992.

[8]

Kallepalli, C., J. Tian. Measuring and modeling usage and reliability for statistical web testing. IEEE Trans. Software Eng. vol. 27, no. 11, pp. 1023--1036, 2001.

Digital Library

[9]

Karlin, S., H.M. Taylor. A first course in stochastic processes, second ed. New York: Academic Press, 1975.

[10]

Li, Z., J. Tian. Testing the suitability of Markov chains as Web usage models. Proc. of the 27th Annual int'l. computer software and applications conf., pp. 356--362, 2003.

Digital Library

[11]

Liu, C.H., D. C. Kung, P. Hsia. Object-based data flow testing of web applications. Proc. of the First Asia-Pacific Conf. on Quality Software, pp. 7--16, 2000.

Digital Library

[12]

Miller, K. W., L. J. Snell, R. E. Nooman, S. K. Park, D. M. Nicol, B. W. Murrill, J. M. Vias. Estimation the probability of failure when testing reveals no failure. IEEE Trans. Software Eng., vol. 18, pp. 33--43, Jan. 1992.

Digital Library

[13]

Musa, J.D. Software Reliability Engineering. McGraw-Hill, 1998.

Digital Library

[14]

Musa, J. D. A theory of software reliability and its application. IEEE Trans, Software Eng., vol. SE-1, pp. 312--321, Aug.1975.

Digital Library

[15]

Musa, J. D., A. Iannino, K. Okumoto. Software reliability. McGraw-Hill Pubilshing Co., 1990.

[16]

Musa, J. D. Operational profiles in software reliability engineering. IEEE Software. vol. 10, no. 2, pp. 14--32, 1993.

Digital Library

[17]

Nelson, E. Estimating software reliability from test data. Microelectronics and Reliability, vol. 17, no. 1, pp. 67--73, 1978.

[18]

Nguyen, H. Q. Testing applications on the Web. New York: John Wiley & Sons, Inc, 2001.

Digital Library

[19]

Pamas, D. L., A. J. Van Schouwen, S. P. Kwan. An evaluation of safety-critical software. Commun ACM, vol. 23, pp. 636--648. June 1990.

Digital Library

[20]

Rook, P. Software Reliability Handbook, London, New York, 1990.

Digital Library

[21]

Shooman, M. l. Software Engineering: Design, Reliability, and Management. New York: McGraw-Hill, 1983.

Digital Library

[22]

Subraya, B. M., S. V. Subrahmanya. Object driven performance testing of Web applications. First Asia-Pacific Conf. on Proc. of Quality Software, pp. 17--26, no. 30-31, Oct. 2000.

Digital Library

[23]

Tian, J. and E. Lin. Unified Markov Models for Software Testing, Performance Evaluation, and Reliability Analysis. 4th ISSAT Int'l Conf. on Reliability and Quality in Design, Seattle, Washington, August, 1998.

[24]

Tian, J., L. Ma, Z. Li, A. G. Koru. A hierarchical strategy for testing web-based applications and ensuring their reliability. Proc. of the 27th Annual International Computer Software and Applications Conference, 2003.

Digital Library

[25]

Tian, J., S, Rudraraju, Z, Li, Evaluating Web software reliability based on workload and failure data extracted from server logs, IEEE transactions on software engineering. Vol. 30, no.11, November 2004.

Digital Library

[26]

Tonella, P., F. Ricca. Dynamic model extraction and statistical analysis of web applications. Proc. of the Fourth Int'l Workshop on Web Site Evolution, 2002.

Digital Library

[27]

Trammell, C. Quantifying the reliability of software: Statistical testing based on a usage model, Proc. of the second IEEE int'l symposium on software engineering standards, Canada, Aug. 1995.

Digital Library

[28]

Wen, R. B. URL-Driven automated testing. Proc. Of the second Asia-Pacific Conference on Quality software, 2001.

Digital Library

[29]

Whittaker, J. A., M. G. Thomason. A Markov chain model for statistical software testing. IEEE Trans. Software Eng., vol. 20, no. 10, pp. 812--824, 1994.

Digital Library

[30]

Wu, Y., J. Offutt. Modeling and testing web-based applications. Technical Report ISE-TR-02-08, 2002.

[31]

Yang, J. T., J. L. Huang, F. J. Wang, W. C. Chu. An object-oriented architecture supporting Web Application testing. 23rd Int'l Computer Software and Applications Conference, pp:122--127, 27-29, Oct.1999.

Digital Library

Cited By

Bertolino AMiranda BPietrantuono RRusso S(2021)Adaptive Test Case Allocation, Selection and Generation Using Coverage Spectrum and Operational ProfileIEEE Transactions on Software Engineering10.1109/TSE.2019.290618747:5(881-898)Online publication date: 1-May-2021
https://doi.org/10.1109/TSE.2019.2906187
Akbulut SGebreyesus YMishra AYazici A(2019)Systematic Mapping on Quality in Web Application Testing2019 1st International Informatics and Software Engineering Conference (UBMYK)10.1109/UBMYK48245.2019.8965472(1-5)Online publication date: Nov-2019
https://doi.org/10.1109/UBMYK48245.2019.8965472
Emam SMiller J(2015)Test Case Prioritization Using Extended DigraphsACM Transactions on Software Engineering and Methodology10.1145/278920925:1(1-41)Online publication date: 2-Dec-2015
https://dl.acm.org/doi/10.1145/2789209
Show More Cited By

Index Terms

Usage-based statistical testing of web applications

Recommendations

Prioritizing User-Session-Based Test Cases for Web Applications Testing
ICST '08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation

Web applications have rapidly become a critical part of business for many organizations. However, increased usage of web applications has not been reciprocated with corresponding increases in reliability. Unique characteristics, such as quick turnaround ...
Improving the effectiveness of test suite reduction for user-session-based testing of web applications

Context: Test suite reduction is the problem of creating and executing a set of test cases that are smaller in size but equivalent in effectiveness to an original test suite. However, reduced suites can still be large and executing all the tests in a ...
Testing Web-based applications: The state of the art and future trends

Software testing is a difficult task and testing Web-based applications may be even more difficult, due to the peculiarities of such applications. In the last years, several problems in the field of Web-based applications testing have been addressed by ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICWE '06: Proceedings of the 6th international conference on Web engineering

July 2006

384 pages

ISBN:1595933522

DOI:10.1145/1145581

General Chairs:
Dave Wolber
University of San Francisco, USA
,
Neil Calder
SLAC, USA
,
Program Chairs:
Chris Brooks
University of San Francisco, USA
,
Athula Ginige
University of Western Sydney, Australia

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
3,141
Total Downloads

Downloads (Last 12 months)85
Downloads (Last 6 weeks)3

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bertolino AMiranda BPietrantuono RRusso S(2021)Adaptive Test Case Allocation, Selection and Generation Using Coverage Spectrum and Operational ProfileIEEE Transactions on Software Engineering10.1109/TSE.2019.290618747:5(881-898)Online publication date: 1-May-2021
https://doi.org/10.1109/TSE.2019.2906187
Akbulut SGebreyesus YMishra AYazici A(2019)Systematic Mapping on Quality in Web Application Testing2019 1st International Informatics and Software Engineering Conference (UBMYK)10.1109/UBMYK48245.2019.8965472(1-5)Online publication date: Nov-2019
https://doi.org/10.1109/UBMYK48245.2019.8965472
Emam SMiller J(2015)Test Case Prioritization Using Extended DigraphsACM Transactions on Software Engineering and Methodology10.1145/278920925:1(1-41)Online publication date: 2-Dec-2015
https://dl.acm.org/doi/10.1145/2789209
Chen CMiao HChen Y(2013)Concept analysis based approach to statistical web testingTheories of Programming and Formal Methods10.5555/2554641.2554648(100-117)Online publication date: 1-Jan-2013
https://dl.acm.org/doi/10.5555/2554641.2554648
Chen CChen YMiao HWang H(2013)Usage-pattern based Statistical Web Testing and Reliability MeasurementProcedia Computer Science10.1016/j.procs.2013.09.02021(140-147)Online publication date: 2013
https://doi.org/10.1016/j.procs.2013.09.020
(2013)A systematic mapping study of web application testingInformation and Software Technology10.1016/j.infsof.2013.02.00655:8(1374-1396)Online publication date: 1-Aug-2013
https://dl.acm.org/doi/10.1016/j.infsof.2013.02.006
Chen CMiao HChen Y(2013)Concept Analysis Based Approach to Statistical Web TestingTheories of Programming and Formal Methods10.1007/978-3-642-39698-4_7(100-117)Online publication date: 2013
https://doi.org/10.1007/978-3-642-39698-4_7
Tian J(2011)An Emerging Experience Factory to Support High-Quality Applications Based on Software Components and Services (Invited Paper)Journal of Software10.4304/jsw.6.2.289-2976:2Online publication date: 18-Feb-2011
https://doi.org/10.4304/jsw.6.2.289-297
Fraser GZeller A(2011)Exploiting Common Object Usage in Test Case GenerationProceedings of the 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation10.1109/ICST.2011.53(80-89)Online publication date: 21-Mar-2011
https://dl.acm.org/doi/10.1109/ICST.2011.53
Halfond WChoudhary SOrso A(2011)Improving penetration testing through static and dynamic analysisSoftware Testing, Verification & Reliability10.1002/stvr.45021:3(195-214)Online publication date: 1-Sep-2011
https://dl.acm.org/doi/10.1002/stvr.450
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents