The Midas Touch: Triggering the Capability of LLMs for RM-API Misuse Detection

We provide a detailed definition for the RM-APIs. Specifically, we consider the APIs that call the malloc function to have allocation semantics; and the APIs that call the free function to have releasing semantics. We manually check each API’s source code and documentation according to the procedure proposed in Section V. To further confirm the allocation semantics of the API, we obtain the allocated object of the current API through data-flow analysis and control-flow analysis. If the allocated object needs an extra releasing operation outside the API, we assume the current API has allocation semantics. The semantics of the specific API is confirmed by the latest official documentation and source code. Besides, the semantics of a function is fixed from the outset and in most cases does not change due to iteration. In other words, when a function is identified to have allocation semantics, its functionality will not change due to the iteration of function versions, that is, it will not switch from having allocation semantics to having releasing semantics.

Through analysis of a large amount of API documentation, we observe that APIs with RM semantics (such as, allocate/release) usually have explicit or implicit descriptions within the documentation. We term the sentences that have the semantics of such APIs as RM sentences. More specifically, if an API calls malloc/free in the source code and its description includes the keywords that are semantically similar to malloc/free, we treat this sentence as RM sentences, the keywords that represent malloc semantics fall in “create”, “allocate”, “construct” and “initialize” etc. We then construct the GroundTruth of 244 sentences through manual inspection from three libraries including FFmpeg, Libdbus and Libpcap. The statistics show that 18.4% of RM sentences contain function names and 81.6% of them contain the allocated/released objects. Based on this, we classify the RM sentences into two types, as illustrated in Table I. The first type of RM sentence is extracted from the description of the function zip_source_filep. This sentence explicitly states that the function containing memory allocation functionality and the memory allocated is stored in “a zip source”, which we term as the RM object description. The second type of RM sentence is extracted from the return value description of the function event_new. This sentence implies the memory allocation semantics of the function by stating that the return value struct event, which can be referred to as RM object, represents the allocated memory and requires releasing using another function. Although extracting the RM-API name directly from this sentence is challenging, we can infer the RM object described in this sentence by understanding its semantic context.

From the preliminary study in Section II-B, directly using LLMs for RM-API identification results in many incorrect answers. We also explored adding more information to identify API’s semantics with source code or documentation(shown in Section IV-C), but neither achieved satisfactory results in the current task. Therefore, inspired by the ReAct framework [7] optimizing on Chain-of-Thought [18], we propose to decompose the complex task (RM-API constraint retrieval) into RM sentence extraction and allocation APIs identification, together with additional information retrieved from API documentation.

According to the definitions above, RM sentence describes the functionality of the RM APIs and ChatDetector uses the prompt in Figure 4 to extract them from API documentation descriptions. Note that all the content such as the api and desc representing description, highlighted in brackets are automatically crawled from official documentation. The rest context in the prompt is the template which is a one-time effort. The procedure is illustrated as follows. First, ChatDetector prompts the LLMs to provide the first sentence in the documentation describing the current API’s functionality and check if it possesses RM semantics. When multiple API descriptions are present on the same documentation page, the sentence outlining a particular API’s functionality typically resides close to that API. In cases where documentation exclusively details a single API and contains multiple sentences with RM semantics, to prevent redundancy, we exclusively select the first sentence that delineates the API’s functionality and exhibits RM semantics as the RM sentence. Secondly, owing to the potential issue of hallucination in the output of LLMs [23], it is imperative to verify whether the presently identified RM sentence originates from the documentation itself and is not fabricated. Furthermore, confirming the accuracy of this response is essential. Consequently, ChatDetector prompts LLMs to provide the reasoning process and the source-based evidence of RM sentence identification to support the answer.

To perform allocation APIs identification, we tried to directly apply the extracted RM sentences to the prompt. However, due to the inherent uncertainty introduced by LLMs, accurately validating the truthfulness and accuracy of current answers using a single prompt poses challenges. We observed the RM sentences can be classified into two categories: those containing both RM-API names and RM objects, and those containing RM objects only. For sentences containing RM objects only, RM APIs identification can be achieved by assessing the presence of RM objects and verifying the accuracy of LLMs’s response. On the other hand, for those sentences containing both RM-API names and RM objects, RM APIs identification necessitates additional validation for authenticity and accuracy, building upon the previous step. And also, the previous studies show that LLMs are effective in three types of prompting methods, which are in-context prompting: extracting answers from the given context and the answers can be extracted from the input context; open prompting: generating answers based on the information from the given context and the answers cannot be extracted from the input context; and indirect prompting: generating answers based on the indirect questions, both the prompts and the answers pose indirect relations to the final answers. Therefore, ChatDetector follows the above three types of prompting and adopts the observation to design the prompts in Figure 5 and Figure 6 to prompt LLMs for allocation APIs identification based on RM sentences, and utilizes the prompt in Figure 7 to obtain RM-objects description.

Specifically, the prompt in Figure 5 is designed to inquire about the API with resource allocation functionality based on the reasoning derived from the current RM sentence. Additionally, the prompt in Figure 6 is designed to ascertain whether the current API possesses resource allocation semantics. And the prompt in Figure 7 is intended to acquire the RM-object. To further confirm the allocation APIs, we design rules (details are shown in Figure 8) to cross-validate the three answers. The necessary condition is to ensure the extracted RM objects exist in the documentation; if the response to the prompt in Figure 5 is the current API or the response to the prompt in Figure 6 is YES, then ChatDetector collects the potential allocations APIs through the cross-validation.

However, the current method leads to many false positives, primarily due to ChatGPT being a black-box model, making it challenging to be guided. Besides, we observe typical self-inconsistencies between the LLMs’s output and the reasoning processes. For example, LLMs indicate that zip_close has memory allocation semantics. But the reasoning process for zip_close is as follows: “The sentence mentions that the zip_close function frees the archive, indicating that its purpose is to deallocate resources rather than allocate them. Therefore, it can be inferred that the function zip_close does not perform allocation.” The reasoning process confirms the identified RM sentence is correct, but LLMs has limitations in understanding this sentence. Even when the RM sentence clearly states that the current function’s operation is memory deallocation, the model still interprets the function’s semantics as the opposite. Therefore, we utilize an off-the-shelf NLP tool, Semantic Role Labeling [24], to extract the action verbs corresponding to the current function within the reasoning process, such as set, free and others. By comparing the semantic similarity between these action verbs and release, allocate or Neutral, ChatDetector can further determine the semantic tendency of the current function. Then we acquire the resource-allocation APIs.

We implement ChatDetector on six popular libraries documentation: FFmpeg [9], Libevent [10], Libexpat [11], Libpcap [12], Libzip [13] and OpenLdap [14] (details are shown in Table V in the Appendix), and the Top 10 applications integrating on them. In total, ChatDetector discovered 165 RM-API pairs containing 165 allocations APIs and 61 releasing APIs, for some of the allocation APIs may have multiple releasing APIs to release different parameters and some of them may have the same releasing API for multiple allocation APIs. We followed the ethical bug disclosure policy of software and reported all of the 115 security bugs, 64 of which were confirmed by the developers with a precision rate of 90.2%. And the disclosure procedure has no ethical issues. The details are shown in Table VI in the Appendix. Figure 13 illustrates a false positive case in CodeQL’s detection when inspecting the use of zip_source_buffer. ChatDetector detected the correct RM-API pair: the resource-allocation API zip_source_buffer and the resource-releasing API zip_source_free. CodeQL’s detection revealed that no subsequent call exists to zip_source_free to release the allocated resource after calling zip_source_buffer. However, as depicted in the figure, the parameter res is generated by the functions zip_file_replace or zip_file_add, and the documentation explicitly states, “NOTE: zip_source_free(3) should not be called on a source after it was used successfully in a zip_file_add or zip_file_replace call.” Consequently, in the condition “res == -1”, which corresponds to a call failure, zip_source_free is invoked to release resource. Due to CodeQL’s inability to handle vulnerability detection when multiple functions collaborate, it resulted in a false memory leak detection.

ChatDetector detectes 168 resource allocation APIs across the six software libraries with an accuracy of 92.81% and a recall of 92.06%. 14 false negatives generated during the identification process were primarily due to the prior misidentification of RM sentences. This could be attributed to LLMs’ difficulty in locating key sentences in less obvious locations. For example, the documentation description for the function zip_open_from_source is illustrated in Figure 21 (details are shown in Appendix). In this description, the sentences related to the zip_open_from_source function do not appear at the beginning of the paragraph or are not prominently highlighted. This leads LLMs to provide the incorrect RM sentence “The semantics of the API zip_open_from_source related to allocation or releasing is not described in the given description”. However, when we prompt LLMs to determine the function’s semantics based on the sentence “The zip_open_from_source() function opens a zip archive encapsulated by the zip_source zs using the provided flags. In case of error, the zip_error ze is filled in”, LLMs is able to correctly identify that the current function has allocation semantics. Therefore, we attribute the occurrence of these false negatives to the inherent limitations of LLMs’ capabilities.

Simultaneously, ChatDetector also introduced 13 false positives. Through the analysis of these cases, we found that the content in the documentation can also mislead LLMs to make incorrect judgments. For instance, in the documentation description of the function zip_source_begin_write_cloning (shown in Figure 20 in the Appendix), the phrase “Usually this involves creating temporary files or allocating buffers” explicitly indicates that the current function involves resource allocation operations. However, upon inspecting the source code, it was found that the function did not call any other functions involving malloc. Therefore, the inconsistency between the documentation description and the source code can also lead to false positives in ChatDetector, making these cases of misleading documentation errors.

To further evaluate the security impact of these bugs, we follow the evaluation methods proposed by IPPO [25] and NDI [26]. Three types of security bugs are found by ChatDetector: memory leak, use-after-free and double-free. 95.7% of the detected bugs would cause memory leaks and lead to memory exhaustion or Denial-of-Services(DoS) when triggered repeatedly. 4 bugs would cause double-free and they could lead to memory corruption when triggered. One bug causes use-after-free and it may lead to DoS when triggered repeatedly. We then use a fixed bug as a case study of a common security impact of the detected bugs. In HandBrake with 15.2k stars in GitHub, the function av_buffersrc_parameters_alloc() allocates a new AVBufferSrcParameters instance and should be freed by the caller with av_free() according to the documentation. However, the parameter par is double freed both before and within the error-handling structure. Therefore, it will lead to a double-free bug. If the bug is triggered, HandBrake may encounter memory corruption. This bug is detected by ChatDetector and the bug is confirmed and fixed by the software holder and the latest version is linked below¹¹1https://github.com/HandBrake/HandBrake/commit/ab467d13ef1f2a9a119c 2cd241c7c831ca207192.

To figure out the contribution of external knowledge for RM-API pairing, We design two types of prompts for RM sentence identification: Type-1: containing no documentation (directly using LLMs, denoting $Eval_{0}$ in Table IV); Type-2: containing API documentation (denoting ChatDetector in the same table). We further perform Chain-of-Thought and cross-validation with the identified RM sentences to RM-API pairing. Among the six libraries, the precision rate of $Eval_{0}$ on RM-API pairing is 32.14% and the F1 score is 30.86%, which are not nearly as effective as ChatDetector. The results fully explain the contribution of external knowledge to RM-API pairing.

To evaluate the contribution of Chain-of-Thought and cross-validation, we also design two types of prompts: Type-3: prompting LLMs with external knowledge, denoting as $Eval_{1}$ in Table IV; Type-4: incorporating Chain-of-Thought and cross-validation with Type-3 prompting, denoting as ChatDetector in the same table. Performing Chain-of-Thought and cross-validation can obtain 25.94% more RM-API pairs with an increase in precision from 77.97% to 98.21% and an increase in recall from 71.98% to 90.65%, which demonstrates the contribution of Chain-of-Thought and cross-validation for RM-API pairing.

To evaluate the impact of different qualities of documentation for RM-API pairing, we design four types of prompts representing different qualities and incorporate them with ChatDetector to perform the evaluation: 1) directly prompting LLMs (referred to as No_doc); 2) using the prompt that includes the first sentence of the description (referred to as First_line); 3) using the prompt that includes the entire description (referred to as Full_desc); 4) and using the prompt that includes the entire document (referred to as Full_doc). The comparative results are shown in Figure 15. The results show that Full_doc performs the best on the task among all the other prompts from three key dimensions: precision, recall and F1 score. Specifically, we assume the API documentation with no description has the lowest quality and the API documentation with a one-sentence description also has significantly lower quality compared with the one including complete documentation. Figure 15 shows that the presence of documentation has a certain impact on RM-API pairing, as the F1 score rapidly increases when the description appears. However, as the document quality improves from a one-sentence description to a detailed description, the change in the F1 score is not significant, and the performance is clearly not as good as Full_doc which is also applied in our work (ChatDetector). Therefore, we conclude that ChatDetector has limitations with undocumented API description, but does not show significant differences in other types of documentation quality.