Cited By
View all- Moreira DSeara JPavia JSerrão C(2024)Intelligent Platform for Automating Vulnerability Detection in Web ApplicationsElectronics10.3390/electronics1401007914:1(79)Online publication date: 27-Dec-2024
A Quantitative Assessment of the Detection Performance of Web Vulnerability Scanners
ARES '22: Proceedings of the 17th International Conference on Availability, Reliability and Security
Article No.: 149, Pages 1 - 10
Abstract
Software developers use web application vulnerability scanners to automatically identify security weaknesses in their web applications. The scanners inspect source code or analyze the running application, and look for specific vulnerability types. While it can be expected that a scanner will not discover every vulnerability, no information is available on the expected efficacy of currently available vulnerability scanners for a given vulnerability type. We present an analysis of 24 web vulnerability scanners and determine their effectiveness on 11 vulnerability types. Our study offers insights into the trade-offs when selecting a specific type of scanner. We show that for some vulnerability types, most vulnerability scanners perform poorly.
References
[1]
Azwar Al Anhar and Yohan Suryanto. 2021. Evaluation of Web Application Vulnerability Scanner for Modern Web Application. In 2021 International Conference on Artificial Intelligence and Computer Science Technology (ICAICST). IEEE, 200–204.
[2]
Shafi Alassmi, Pavol Zavarsky, Dale Lindskog, Ron Ruhl, Ahmed Alasiri, Muteb Alzaidi, 2012. An analysis of the Effectiveness of Black-box Web Application Scanners in Detection of Stored XSSI Vulnerabilities. International Journal of Information Technology and Computer Science 4, 1(2012).
[3]
Richard Amankwah, Jinfu Chen, Patrick Kwaku Kudjo, and Dave Towey. 2020. An empirical comparison of commercial and open-source web vulnerability scanners. Software: Practice and Experience 50, 9 (2020), 1842–1857.
[4]
Nuno Antunes and Marco Vieira. 2015. On the metrics for benchmarking vulnerability detection tools. In Proceedings of the 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 505–516.
[5]
Ricardo Araújo, António Pinto, and Pedro Pinto. 2021. A Performance Assessment of Free-to-Use Vulnerability Scanners - Revisited. In Proceedings of the International Conference on ICT Systems Security and Privacy Protection (IFIP SEC). Springer, 53–65.
[6]
Jason Bau, Elie Bursztein, Divij Gupta, and John Mitchell. 2010. State of the art: Automated black-box web application vulnerability testing. In Proceedings of the IEEE Symposium on Security and Privacy (S&P). IEEE, 332–345.
[7]
Elisa Burato, Pietro Ferrara, and Fausto Spoto. 2017. Security analysis of the OWASP benchmark with Julia. Proceedings of ITASEC(2017).
[8]
Shay Chen. [n.d.]. The Web Application Vulnerability Scanner Evaluation Project (WAVSEP). https://github.com/sectooladdict/wavsep. accessed on 20-Apr-2022.
[9]
Nor Izyani Daud, Khairul Azmi Abu Bakar, and Mohd Shafeq Md Hasan. 2014. A case study on web application vulnerability scanning tools. In Proceedings of the Science and Information Conference (SAI). IEEE, 595–600.
[10]
Adam Doupé, Marco Cova, and Giovanni Vigna. 2010. Why Johnny Can’t Pentest: An Analysis of Black-Box Web Vulnerability Scanners. In Proceedings of the International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment (DIMVA). Springer, 111–131.
[11]
Joao A Duraes and Henrique S Madeira. 2006. Emulation of software faults: A field data study and a practical approach. Transactions on Software Engineering 32, 11 (2006), 849–867.
[12]
Malaka El, Emma McMahon, Sagar Samtani, Mark Patton, and Hsinchun Chen. 2017. Benchmarking vulnerability scanners: An experiment on SCADA devices and scientific instruments. In Proceedings of the International Conference on Intelligence and Security Informatics (ISI). IEEE, 83–88.
[13]
Ronen Fluss, David Faraggi, and Benjamin Reiser. 2005. Estimation of the Youden Index and its associated cutoff point. Biometrical Journal: Journal of Mathematical Methods in Biosciences (2005), 458–472.
[14]
Jose Fonseca, Marco Vieira, and Henrique Madeira. 2007. Testing and comparing web vulnerability scanning tools for SQL injection and XSS attacks. In Proceedings of the 13th Pacific Rim international symposium on dependable computing (PRDC 2007). IEEE, 365–372.
[15]
The OWASP Foundation. [n.d.]. OWASP Top 10 2007. https://owasp.org/www-pdf-archive/OWASP_Top_10_2007.pdf. accessed on 15-Jun-2022.
[16]
Juan Higuera, Javier Higuera, Juan Montalvo, Javier Villalba, and Juan Pérez. 2020. Benchmarking Approach to Compare Web Applications Static Analysis Tools Detecting OWASP Top Ten Security Vulnerabilities. Computers, Materials and Continua 64 (2020).
[17]
S.E. Idrissi, N. Berbiche, F. Guerouate, and M. Shibi. 2017. Performance evaluation of web application security scanners for prevention and protection against vulnerabilities. International Journal of Applied Engineering Research 12, 21 (2017), 11068–11076.
[18]
Rahul Johari and Pankaj Sharma. 2012. A survey on web application vulnerabilities (SQLIA, XSS) exploitation and security engine for SQL injection. In Proceedings of the International Conference on Communication Systems and Network Technologies. IEEE, 453–458.
[19]
Erwan Le Rousseau. 2013. Damn Vulnerable Web Application (DVWA). https://dvwa.co.uk/. accessed on 20-Apr-2022.
[20]
Yuma Makino and Vitaly Klyuev. 2015. Evaluation of web vulnerability scanners. In Proceedings of the 8th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications. IEEE, 399–402.
[21]
Yuliana Martirosyan. 2012. Security evaluation of web application vulnerability scanners strengths and limitations using custom web application. Master’s thesis. East Bay, California State University.
[22]
Balume Mburano and Weisheng Si. 2018. Evaluation of Web Vulnerability Scanners Based on OWASP Benchmark. In Proceedings of the 26th International Conference on Systems Engineering (ICSEng). IEEE, 1–6.
[23]
Malik Mesellem. [n.d.]. Buggy Web App (bWAPP). https://www.mmebvba.com/sites/bwapp/. accessed on 20-Apr-2022.
[24]
Rawaa Mohammed. 2016. Assessment of Web Scanner Tools. International Journal of Computer Applications 133 (2016), 1–4.
[25]
Yuanyuan Pan. 2019. Interactive Application Security Testing. In Proceedings of the International Conference on Smart Grid and Electrical Automation (ICSGEA). IEEE, 558–561.
[26]
Andrey Petukhov and Dmitry Kozlov. 2008. Detecting security vulnerabilities in web applications using dynamic analysis with penetration testing. Computing Systems Lab(2008).
[27]
Deepika Sagar, Sahil Kukreja, Jwngfu Brahma, Shobha Tyagi, and Prateek Jain. 2018. Studying open source vulnerability scanners for vulnerabilities in web applications. IIOAB JOURNAL 9, 2 (2018), 43–49.
[28]
Y Smeets. 2015. Improving the adoption of dynamic web security vulnerability scanners. Master’s thesis. Radboud University, NL.
[29]
Natasa Suteva, Dragi Zlatkovski, and Aleksandra Mileva. 2013. Evaluation and testing of several free/open source web vulnerability scanners. In Proceedings of the 10th Conference for Informatics and Information Technology (CIIT).
[30]
Larry Suto. 2010. Analyzing the accuracy and time costs of web application security scanners.
[31]
Emin İslam Tatli and Bedirhan Urgun. 2017. WIVET — Benchmarking Coverage Qualities of Web Crawlers. Comput. J. (2017), 555–572.
[32]
The OWASP Foundation. [n.d.]. OWASP Benchmark Project. https://owasp.org/www-project-benchmark/. accessed on 20-Apr-2022.
[33]
The OWASP Foundation. [n.d.]. OWASP VulnerableApp. https://owasp.org/www-project-vulnerableapp/. accessed on 20-Apr-2022.
[34]
The OWASP Foundation. 2021. OWASP Top Ten Web Application Security Risks. https://owasp.org/www-project-top-ten/. accessed on 20-Apr-2022.
[35]
Bedirhan Urgun. [n.d.]. Web Input Vector Extractor Teaser (WIVET). https://github.com/bedirhan/wivet. accessed on 20-Apr-2022.
[36]
Esteban Alejandro Armas Vega, Ana Lucila Sandoval Orozco, and Luis Javier García Villalba. 2017. Benchmarking of pentesting tools. International Journal of Computer, Electrical, Automation, Control and Information Engineering 11, 5 (2017), 602–605.
[37]
Marco Vieira, Nuno Antunes, and Henrique Madeira. 2009. Using web security scanners to detect vulnerabilities in web services. In Proceedings of the IEEE/IFIP International Conference on Dependable Systems & Networks (DSN). IEEE, 566–571.
Index Terms
- A Quantitative Assessment of the Detection Performance of Web Vulnerability Scanners
Recommendations
A Survey on XSS Attack Detection and Prevention in Web Applications
ICMLC '20: Proceedings of the 2020 12th International Conference on Machine Learning and ComputingWith the popularity of web technology, web applications become more increasingly vulnerable and are exposed to malicious attacks. Cross Site Scripting(XSS) is a typical attack in web applications. When a vulnerability is exploited, an attacker may ...
Evaluation of web vulnerability scanners
2015 IEEE 8th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS)In recent yeas a lot of web applications have been released in the world. At the same time, cyber attacks against web application vulnerabilities have also increased. In such a situation, it is necessary to make web applications more secure. However ...
Enlargement of vulnerable web applications for testing
There are two main kinds of vulnerable web applications, usual applications developed with a specific aim and applications which are vulnerable by design. On one hand, the usual applications are those that are used everywhere and on a daily basis, and ...
Comments
Information & Contributors
Information
Published In
August 2022
1371 pages
ISBN:9781450396707
DOI:10.1145/3538969
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 23 August 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ARES 2022
ARES 2022: The 17th International Conference on Availability, Reliability and Security
August 23 - 26, 2022
Vienna, Austria
Acceptance Rates
Overall Acceptance Rate 228 of 451 submissions, 51%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 315Total Downloads
- Downloads (Last 12 months)96
- Downloads (Last 6 weeks)8
Reflects downloads up to 09 Feb 2025
Other Metrics
Citations
Cited By
View all- Moreira DSeara JPavia JSerrão C(2024)Intelligent Platform for Automating Vulnerability Detection in Web ApplicationsElectronics10.3390/electronics1401007914:1(79)Online publication date: 27-Dec-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format