short-paper

Prompt-Enhanced Software Vulnerability Detection Using ChatGPT

Authors:

Hao Liu,

Hui LiAuthors Info & Claims

ICSE-Companion '24: Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings

Pages 276 - 277

https://doi.org/10.1145/3639478.3643065

Published: 23 May 2024 Publication History

Get Access

Abstract

With the increase in software vulnerabilities that cause significant economic and social losses, automatic vulnerability detection has become essential in software development and maintenance. Recently, large language models (LLMs) have received considerable attention due to their stunning intelligence, and some studies consider using ChatGPT for vulnerability detection. However, they do not fully consider the characteristics of LLMs, since their designed questions to ChatGPT are simple without a prompt design tailored for vulnerability detection. This paper launches a study on the performance of software vulnerability detection using ChatGPT with different prompt designs. Firstly, we complement previous work by applying various improvements to the basic prompt. Moreover, we incorporate structural and sequential auxiliary information to improve the prompt design. Moreover, we leverage ChatGPT's ability of memorizing multi-round dialogue to design suitable prompts for vulnerability detection. We conduct extensive experiments on two vulnerability datasets to demonstrate the effectiveness of prompt-enhanced vulnerability detection using ChatGPT.

References

[1]

Jialun Cao, Meiziniu Li, Ming Wen, and Shing-Chi Cheung. 2023. A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair. arXiv Preprint (2023). https://arxiv.org/abs/2304.08191

Google Scholar

[2]

Dominik Sobania, Martin Briesch, Carol Hanna, and Justyna Petke. 2023. An Analysis of the Automatic Bug Fixing Performance of ChatGPT. arXiv Preprint (2023). https://arxiv.org/abs/2301.08653

Google Scholar

[3]

Song Wang, Devin Chollak, Dana Movshovitz-Attias, and Lin Tan. 2016. Bugram: bug detection with n-gram language models. In ASE. 708--719.

Google Scholar

[4]

Jian Zhang, Xu Wang, Hongyu Zhang, Hailong Sun, Xudong Liu, Chunming Hu, and Yang Liu. 2023. Detecting Condition-Related Bugs with Control Flow Graph Neural Network. In ISSTA. 1370--1382.

Google Scholar

Cited By

View all

Zhang JBu HWen HLiu YFei HXi RLi LYang YZhu HMeng D(2025)When LLMs meet cybersecurity: a systematic literature reviewCybersecurity10.1186/s42400-025-00361-w8:1Online publication date: 5-Feb-2025
https://doi.org/10.1186/s42400-025-00361-w
Guo JWang MYin HSong BChi YYu FYuen C(2025)Large Language Models and Artificial Intelligence Generated Content Technologies Meet Communication NetworksIEEE Internet of Things Journal10.1109/JIOT.2024.349649112:2(1529-1553)Online publication date: 15-Jan-2025
https://doi.org/10.1109/JIOT.2024.3496491
Cotroneo DDe Luca RLiguori P(2025)DeVAIC: A tool for security assessment of AI-generated codeInformation and Software Technology10.1016/j.infsof.2024.107572177(107572)Online publication date: Jan-2025
https://doi.org/10.1016/j.infsof.2024.107572
Show More Cited By

Recommendations

Collaboration to Repository-Level Vulnerability Detection
ISSTA 2024: Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis

Large Language Model (LLM)-based methods have proven to be effective for many software engineering domains, with a potential for substantial productivity effective for software vulnerability detection. However, due to the limitation of the length of ...
Software Vulnerability Detection Using an Enhanced Generalization Strategy
Dependable Software Engineering. Theories, Tools, and Applications
Abstract
Detecting vulnerabilities in software is crucial for preventing cybersecurity attacks, and current machine learning-based methods rely on large amounts of labeled data to train detection models. On the one hand, a major assumption is that the ...
A Systematic Literature Review on Automated Software Vulnerability Detection Using Machine Learning
In recent years, numerous Machine Learning (ML) models, including Deep Learning (DL) and classic ML models, have been developed to detect software vulnerabilities. However, there is a notable lack of comprehensive and systematic surveys that summarize, ...

Comments

Information & Contributors

Information

Published In

ICSE-Companion '24: Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings

April 2024

531 pages

ISBN:9798400705021

DOI:10.1145/3639478

Co-chairs:
Ana Paiva,
Rui Abreu,
Program Co-chairs:
Abhik Roychoudhury,
Margaret Storey

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

Faculty of Engineering of University of Porto

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 May 2024

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICSE-Companion '24

Sponsor:

SIGSOFT

ICSE-Companion '24: 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings

April 14 - 20, 2024

Lisbon, Portugal

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
571
Total Downloads

Downloads (Last 12 months)571
Downloads (Last 6 weeks)81

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhang JBu HWen HLiu YFei HXi RLi LYang YZhu HMeng D(2025)When LLMs meet cybersecurity: a systematic literature reviewCybersecurity10.1186/s42400-025-00361-w8:1Online publication date: 5-Feb-2025
https://doi.org/10.1186/s42400-025-00361-w
Guo JWang MYin HSong BChi YYu FYuen C(2025)Large Language Models and Artificial Intelligence Generated Content Technologies Meet Communication NetworksIEEE Internet of Things Journal10.1109/JIOT.2024.349649112:2(1529-1553)Online publication date: 15-Jan-2025
https://doi.org/10.1109/JIOT.2024.3496491
Cotroneo DDe Luca RLiguori P(2025)DeVAIC: A tool for security assessment of AI-generated codeInformation and Software Technology10.1016/j.infsof.2024.107572177(107572)Online publication date: Jan-2025
https://doi.org/10.1016/j.infsof.2024.107572
Shao MDing YCao JLi Y(2025)GraphFVD: Property graph-based fine-grained vulnerability detectionComputers & Security10.1016/j.cose.2025.104350151(104350)Online publication date: Apr-2025
https://doi.org/10.1016/j.cose.2025.104350
Lin RFu YYi WYang JCao JDong ZXie FLi H(2024)Vulnerabilities and Security Patches Detection in OSS: A SurveyACM Computing Surveys10.1145/3694782Online publication date: 9-Sep-2024
https://doi.org/10.1145/3694782
Gohar UHunter MLutz RCohen MFilkov VRay BZhou M(2024)CoDefeater: Using LLMs To Find Defeaters in Assurance CasesProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695296(2262-2267)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695296
Li JZhang MLi NWeyns DJin ZTei K(2024)Generative AI for Self-Adaptive Systems: State of the Art and Research RoadmapACM Transactions on Autonomous and Adaptive Systems10.1145/368680319:3(1-60)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3686803
Kholoosi MBabar MCroft R(2024)A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicality2024 IEEE 6th International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications (TPS-ISA)10.1109/TPS-ISA62245.2024.00022(107-117)Online publication date: 28-Oct-2024
https://doi.org/10.1109/TPS-ISA62245.2024.00022
Mao ZLi JJin DLi MTei K(2024)Multi-Role Consensus Through LLMs Discussions for Vulnerability Detection2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C63300.2024.00173(1318-1319)Online publication date: 1-Jul-2024
https://doi.org/10.1109/QRS-C63300.2024.00173
Ma JFeng SZeng JLu JChen J(2024)Smart Contract Vulnerability Detection Based on Prompt-guided ChatGPT2024 International Conference on Networking and Network Applications (NaNA)10.1109/NaNA63151.2024.00060(321-326)Online publication date: 9-Aug-2024
https://doi.org/10.1109/NaNA63151.2024.00060
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Collaboration to Repository-Level Vulnerability Detection

Software Vulnerability Detection Using an Enhanced Generalization Strategy

A Systematic Literature Review on Automated Software Vulnerability Detection Using Machine Learning

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations