Abstract
Web applications are exposed to frequent changes both in requirements and involved technologies. At the same time, there is a continuously growing demand for quality and trust and such a fast evolution and quality constraints claim for mechanisms and techniques for automated testing. Web application automated testing often involves random crawlers to navigate the application under test and automatically explore its structure. However, owing to the specific challenges of the modern Web systems, automatic crawlers may leave large portions of the application unexplored. In this paper, we propose the use of structural metrics to predict whether an automatic crawler with given crawling capabilities will be sufficient or not to achieve high coverage of the application under test. In this work, we define a taxonomy of such capabilities and we determine which combination of them is expected to give the highest reward in terms of coverage increase. Our proposal is supported by an experiment in which 19 web applications have been analyzed.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Alshahwan, N., Harman, M.: Automated session data repair for web application regression testing. In: ICST ’08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, pp. 298–307. IEEE Computer Society, Washington (2008)
Alshahwan, N., Harman, M., Marchetto, A., Tonella, P.: Improving web application testing using testability measure. In: IEEE International Symposium on Web Systems Evolution (WSE), IEEE Computer Society, Edmonton (September 2009)
Anneliese J.O., Andrews A., Alexander R.T.: Testing web applications by modeling with fsms. Softw. Syst. Model. 4, 326–345 (2005)
Artzi, S., Kiezun, A., Dolby, J., Tip, F., Dig, D., Paradkar, A., Ernst, M.D.: Finding bugs in dynamic web applications. In: ISSTA ’08: Proceedings of the 2008 International Symposium on Software Testing and Analysis, pp. 261–272. ACM, New York (2008)
Bellettini, C., Marchetto, A., Trentini, A.: TestUml: user-metrics driven web applications testing. In: SAC ’05: Proceedings of the 2005 ACM Symposium on Applied Computing, pp. 1694–1698. ACM, New York (2005)
Bruntink M., van Deursen A.: An empirical study into class testability. J. Syst. Softw. 79(9), 1219–1232 (2006)
Conallen, J.: Modeling web applications with uml. In: White Paper (1999)
Conallen J.: Building Web Applications with UML. Addison-Wesley, Reading (2000)
Dhyani D., Ng W.K., Bhowmick S.S.: A survey of web metrics. ACM Comput. Surv. 34(4), 469–503 (2002)
Douglis, F., Feldmann, A., Krishnamurthy, B., Mogul, J.: Rate of change and other metrics: a live study of the world wide web. In: USITS’97: Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems. USENIX Association, Berkeley (1997)
Elbaum, S., Karre, S., Rothermel, G.: Improving Web application testing with user session data. In: Proceedings of the 25th International Conference on Software Engineering (ICSE), pp. 49–59. IEEE Computer Society, Portland (May 2003)
Elbaum S., Rothermel G., Karre S., Fisher M. II: Leveraging user-session data to support web application testing. IEEE Trans. Softw. Eng. 31(3), 187–202 (2005)
Fisher, M. II, Elbaum, S., Rothermel, G.: Dynamic characterization of web application interfaces. In: FASE’07: Proceedings of the Fundamental Approaches to Software Engineering, pp. 260–275. Springer, Berlin (2007)
Ghosheh, E., Qaddour, J., Kuofie, M., Black, S.: A comparative analysis of maintainability approaches for web applications. In: AICCSA ’06: Proceedings of the IEEE International Conference on Computer Systems and Applications, pp. 1155–1158, IEEE Computer Society, Washington (2006)
Girardi C., Ricca F., Tonella P.: Web crawlers compared. Int. J. Web Inform. Syst. 2, 85–94 (2006)
Halfond, W.G., Anand, S., Orso, A.: Precise interface identification to improve testing and analysis of web applications. In: ISSTA ’09: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, pp. 285–296. ACM, New York (2009)
Halfond, W.G.J., Orso, A.: Improving test case generation for web applications using automated interface discovery. In: ESEC-FSE ’07: Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, pp. 145–154. ACM, New York (2007)
Halfond, W.G.J., Orso, A.: Automated identification of parameter mismatches in web applications. In: SIGSOFT ’08/FSE-16: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 181–191. ACM, New York (2008)
Harman, M.: Open problems in testability transformation (keynote). In: 1st International Workshop on Search Based Testing (SBT 2008), Lillehammer (2008)
Harman M., Hu L., Hierons R.M., Wegener J., Sthamer H., Baresel A., Roper M.: Testability transformation. IEEE Trans. Softw. Eng. 30(1), 3–16 (2004)
Heydon A., Najork M.: Mercator: a scalable, extensible web crawler. World Wide Web 2(4), 219–229 (1999)
Hieatt E., Mee R., Faster G.: Testing the web application engineering internet. IEEE Softw. 19(2), 60–65 (2002)
Jungmayr, S.: Testability measurement and software dependencies. In: Proceedings of the 12th International Workshop on Software Measurement, pp. 179–202. Magdeburg,Shaker Publication, Aachen (2002)
Kan S.H.: Metrics and models in software quality engineering. Addison-Wesley, Reading (2003)
Kim K.-H., Kim Y.-G.: Process reverse engineering for bpr: a form-based approach. J. Inform. Manage. 23(4), 187–200 (1998)
Lakhotia, K., McMinn, P., Harman, M.: Automated test data generation for coverage: Haven’t we solved this problem yet? In: 4th Testing Academia and Industry Conference—Practice and Research Techniques (TAIC PART’09), pp. 95–104. IEEE Computer Society, Washington (2009)
Marchetto, A., Tonella, P., Ricca, F.: State-based testing of ajax web applications. In: International Conference on Software Testing Verification and Validation (ICST), IEEE Computer Society Lillehammer (April 2008)
McCabe T.: A complexity measure. IEEE Trans. Softw. Eng. 2(4), 308–320 (1976)
Mesbah, A., van Deursen, A.: Invariant-based automatic testing of ajax user interfaces. In: 31st International Conference on Software Engineering (ICSE). IEEE Computer Society (May 2009)
Miller R.C., Bharat K.: Sphinx: a framework for creating personal, site-specific web crawlers. Comput. Netw. ISDN Syst. 30(1–7), 119–130 (1998)
Palmer J.W.: Web site usability, design, and performance metrics. Info. Syst. Res. 13(2), 151–167 (2002)
Ricca, F., Tonella, P.: Analysis and testing of web applications. In: ICSE ’01: Proceedings of the 23rd International Conference on Software Engineering, pp. 25–34. IEEE Computer Society, Washington (2001)
Sampath, S., Bryce, R.C., Viswanath, G., Kandimalla, V., Koru, A.G.: Prioritizing user-session-based test cases for web applications testing. In: ICST ’08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, pp. 141–150. IEEE Computer Society, Washington (2008)
Sampath, S., Sprenkle, S., Gibson, E., Pollock, L., Souter, A.: Analyzing clusters of web application user sessions. In: WODA ’05: Proceedings of the Third International Workshop on Dynamic Analysis, pp. 1–7. ACM, New York (2005)
Sampath S., Sprenkle S., Gibson E., Pollock L., Greenwald A. Souter: Applying concept analysis to user-session-based testing of web applications. IEEE Trans. Softw. Eng. 33(10), 643–658 (2007)
Shepperd M.J.: Foundations of software measurement. Prentice Hall, New Jersey (1995)
Sprenkle, S., Sampath, S., Gibson, E., Pollock, L., Souter, A.: An empirical comparison of test suite reduction techniques for user-session-based testing of web applications. In: ICSM ’05: Proceedings of the 21st IEEE International Conference on Software Maintenance, pp. 587–596, IEEE Computer Society Washington (2005)
Tip F.: A survey of program slicing techniques. J. Program. Lang. 3(3), 121–189 (1995)
Warren, P., Boldyreff, C., Munro, M.: The evolution of websites. In: IWPC ’99: Proceedings of the 7th International Workshop on Program Comprehension, p. 178. IEEE Computer Society, Washington (1999)
Wassermann, G., Yu, D., Chander, A., Dhurjati, D., Inamura, H., Su, Z.: Dynamic test input generation for web applications. In: ISSTA ’08: Proceedings of the 2008 International Symposium on Software Testing and Analysis, pp. 249–260. ACM, New York (2008)
Weiser M.: Program slicing. IEEE Trans. Softw. Eng. 10(4), 352–357 (1984)
Wu, Y., Offutt, J.: Modeling and testing web-based applications. Technical report, George Mason University (2002)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Marchetto, A., Tiella, R., Tonella, P. et al. Crawlability metrics for automated web testing. Int J Softw Tools Technol Transfer 13, 131–149 (2011). https://doi.org/10.1007/s10009-010-0177-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10009-010-0177-3