Explaining the Contributing Factors for Vulnerability Detection in Machine Learning

Our research aims to identify the factors contributing to the learning of software vulnerabilities from source code repositories. Several approaches have been developed aiming to improve the detection of vulnerabilities (e.g., [38, 2, 21, 39, 40]). One example is applying pattern recognition techniques to detect malware [41]. This technique [41] consists of visualizing malware binary gray-scale images and classifying these images according to observations that show that malware from the same families appears to be very similar in layout and texture.

To build a vulnerability prediction model, the selection of features is essential. The most frequent features used in previous works are software metrics [42, 43, 44] and developers activity [5]. Basili et al.[43] used source code metrics to classify the C++ code into binary code vulnerabilities back to 1996. Nagappan et al. [44] used complexity metrics like module metrics that consist of the number of classes, functions and variables in the module M, in addition to per-function and per-class metrics. They used those metrics with some Microsoft systems to identify faulty components. Perl et al. [22] considered metrics from developer activities by analyzing if commits were related to a vulnerability or not. The methodology of this work [22] consists of combining machine learning using a support vector machine (SVM) classifier with code metrics gathered from repository metadata.

Recent work treats code as a form of text and uses natural language processing based methods for code analysis. Zhou and Sharma [24] used commit messages and bug reports from repositories to identify software flaws using NLP techniques such as word2vec to create the embeddings used as features and machine learning classifiers. Hovsepyan et al.[21] analyzed Java source code from Android applications using a bag-of-words representation and SVM for vulnerability prediction.

Pang et al. [39] further include n-grams in the feature vectors and used SVM for classification. Jackson and Bennett [45] using the Python Natural Language Toolkit
(NLTK) to develop a machine learning agent that uses NLP techniques to convert the code to a matrix and identify a specific flaw—SQL injection—in Java byte code using decision trees and random forests for classification.

Other works focus more on using deep learning techniques such as Russel et al. [6] attempts to identify vulnerabilities using C and C++ source code at the function level based on deep feature representation learning that directly interprets lexed source code and also Dam et al.[2] present an approach based on deep learning using an LSTM model, to automatically learn both semantic and syntactic features of code.

Apart from the work of Hovsepyan et al. [21] most of these approaches focus on the feature engineering part, like Russel et al. [6] that uses a convolutional neural network to build the feature vectors.

A recent survey [46] summarizes the techniques, the datasets and results obtained from vulnerability detection research that uses machine learning. According to their categories, our work falls in the text-based category, since we use a convolutional neural network (Resnet).

Our approach focuses on detecting vulnerabilities in source code using machine learning and natural language processing techniques. However, we use general NLP-based techniques (bag-of-word, word2vec, fastText, and tokenizing code) associated with different machine learning models to identify the key factors contributing to the learning of the software flaws from code.

Software architecture is the high-level abstract of a software system. Poor software architectural decisions are responsible for various software quality problems. Numerous previous research has underscored the impact of software architecture on security.

Software architecture is the most important determinant to systematically achieve quality attributes in a software system, including software security [47]. Software security is, for many systems, the most important quality attribute driving the design.

Due to the intrinsic connections between software architecture and security, prior studies have investigated how software architecture impacts the security of a system [25, 26, 27, 28]. However, little work has investigated how to leverage software architecture characteristics and metrics in machine learning processes to discover vulnerabilities. Researchers in software architecture have developed some measures to capture the complexity of software architecture entities [25, 26, 27, 28, 29, 30]. For example, Fan-In and Fan-Out of source files and classes are shown to impact the propagation of software quality issues through the inter-dependencies among software entities [29]. What remains unclear is whether and how different architecture metrics can be used as vulnerability representations for machine learning models to detect software vulnerabilities.

Previous research mostly focused on security assessment and evaluation from an architectural perspective. For example, Feng et al. found that software vulnerabilities are highly correlated with flawed architectural connections among source files [25]. Sohr and Berger found that software architecture analysis helps to concentrate on security-critical software modules and detect certain security flaws at the architectural level, such as the circumvention of APIs or incomplete enforcement of access control [48]. Brian and Issarny showed how software architecture benefits security by encapsulating security-related requirements at design-time [49]. Antonino et al.  [50] evaluated the security of existing service-oriented systems on the architectural level. Their method is based on recovering security-relevant facts about the system and interactive security analysis at the structural level. Alkussayer and Allen [51] proposed a security risk evaluation approach by leveraging the architectural model of a system, assuming that components propagate their security risks to higher-level components in the architecture model. Alkussayer and Allen [52] assessed the level of security supported by a given architecture and qualitatively compared multiple architectures with respect to their security support.

Despite the high recognition of an architecture’s impact on security, the is little focus on using architectural metrics as vulnerability signatures for machine learning models [53, 54]. Alshammari et al.  [55] is one of the few studies that investigated security metrics based on the composition, coupling, extensibility, inheritance, and design size of an object-oriented project. However, these metrics have not been compared with other vulnerability signatures, such as code features extracted using NLP. In addition, these metrics tightly tie into object-oriented concepts and may not be easy to transfer to other programming paradigms. Motivated by the work of Feng et al., our study focuses on eight architectural metrics that capture how software elements, i.e. source files, are interdependent on each other [25]. And these metrics are generally applicable to software projects of different characteristics, such as the programming language used. In addition, although they are measured at the file level in this work, it is easy to roll up and down to the component level or method level following the same rationale to detect vulnerabilities at different granularities in future studies. Most importantly, to the best of our knowledge, we are the first to compare architectural metrics with code features extracted through NLP as vulnerability representations.

The process of vulnerable code detection, as shown in Figure 1, contains a software repository, which provides the corpus for developing a vocabulary. Any project (even without vulnerable code labels) can be used for this purpose. Such a vocabulary is used to build the embedding of software tokens. The embedding is pretrained using word2Vec and fastText with the corpus. The tokens are then converted to numerical representations by running the embedding. In addition, architecture metrics can be extracted from projects with tags. Next, architectural metrics and embeddings of code tokens are the features input to a supervised classification model. We consider the vulnerability detection under two sources of vulnerability code labels: (1) the labels are from code within the same domain as the target software for vulnerability detection (Tables 9, 10 and 11); and (2) the models are trained with software code in one domain with vulnerability labels and used to classify software code in a different domain. For example, a model is learnt with the dataset from the Juliet dataset, then used to predict the vulnerability of source code in Android projects (Tables 7 and 12).

Tokenization is a common pre-processing step in natural language processing to transform the raw input text into a format that is more easily processed. The raw code contains 1) special symbols include punctuation characters (such as, . , : ; ? ) ( ] [   ’ " } {), 2) mathematical and logical operators (such as, + - / = * & ! % | < >); and 3) others (such as # \@ ˆ), in NLP special characters add no value to text-understanding and can induce noise in algorithms. In addition to other meta text that usually appears as code comments. These comments are any text that starts with two forward slashes (//) and any text between /* and */. To determine if those special characters and comments are important in the vulnerability prediction in source code, we use regular expressions to remove the code comments and special symbols.

Token embeddings are learned numerical representations for text where tokens or words that have similar meanings are approximated by the same value. In the domain of NLP, a corpus is a collection of texts. All tokens or words in the multiple corpora form a high-dimensional space. A learning model on this space calibrates the positions of each word or token according to its relations with all other tokens. Finally, each token has a numerical vector representation called an embedding.

In this work, the corpus is formed with software projects selected from the Github repository. We trained three models as follows to create the numerical vector representation of the source code tokens:

Software architecture refers to software elements, their relationships, and the properties of both [61]. As discussed in Section 2.2, prior research has revealed the significant impact of architecture design decisions on software security. In particular, the study in  [25] reported that complicated architectural connections among source files in a project contribute positively to the propagation of software vulnerability issues. Hence we are motivated to investigate whether metrics that measure the complexity of architectural connections at the file level contribute positively to the detection of software vulnerabilities using machine learning models.

We model a set of architectural connections as a graph, namely $G=\{F,D\}$, where $F$ is the set of source files in the system, and $D$ is the set of structural dependencies among the source files. The graph $G$ of a software system can be reverse-engineered using existing tools, such as Scitool Understand ²²2https://scitools.com/.

For each source file $f\in F$, we capture eight metrics to measure the file’s connections with the rest of the system $G$. We assume these metrics are feature representations to learn more vulnerabilities. These eight metrics are from three different but related aspects of software architecture:

First, we measure Fan-in and Fan-out of a file $f$, which counts the number of direct dependencies with $f$, and are commonly used for various analysis:

1. 1.

   Fan-in: The number of source files in $G$ that directly depends on $f$.

2. 2.

   Fan-out: The number of source files in $G$ that $f$ directly depends on.

Next, we measure the position of $f$ in the entire dependency hierarchy of $G$. Cai et al. proposed an algorithm to cluster source files into hierarchical dependency layers based on their structural dependencies in $G$ [62]. The key features of the layers are: 1) source files in the same layers form independent modules, and 2) source files in a lower layer structurally depend on the upper layer, but not vice-versa. This layered structure is called the Architectural Design Rule Hierarchy (ArchDRH). The rationale is that source files in a higher layer structurally impact source files in the lower layers. Therefore, the higher the layer of $f$, the more influential it is for the rest of the system.

1. 3.

   Design Rule Hierarchy Layer: the layer number of $f$ in the ArchDRH clustering.

Finally, we measure the complexity of the transitive connections to each $f$ in $G$. For any $f\in F$, we define the $Butterfly\_Space_{f}=\{f,UpperWing,LowerWing\}$, where $f$ is the center of the space. $UpperWing$ is the set of source files that directly and transitively depend on $f$. Similarly, $LowerWing$ is the set of source files that $f$ directly and transitively depends on. For any $f\in G$, we calculate five metrics based on the $Butterfly\_Space$ notions:

1. 4.

   Space Size: the total number of source files in
   $Butterfly\_Space_{f}$. This measures the total number of source files that $f$ is connected to directly and transitively. The higher this value, the more significant is $f$ connected to the rest of the system.

2. 5.

   Upper Width: the width of the $UpperWing$. This measures the maximal number of branches that depend on $f$.

3. 6.

   Upper Depth: the length of the longest path in the $UpperWing$. This measures the most far-reaching transitive dependency on $f$.

4. 7.

   Lower Width: the width of the $LowerWing$. This measures the maximal number of branches that $f$ depends on.

5. 8.

   Lower Depth: the length of the longest path in the $LowerWing$. This measures the most far-reaching transitive dependency from $f$.

In this study, we investigate whether and to what extent these metrics contribute to the learning of software vulnerabilities.

We perform a classification task to predict if a file is vulnerable or not. Our objective is to observe the effects of machine learning models. Since a machine learning model is part of the decision process of the classification task, we consider the model’s transparency to the classification decision. A random forest model has one form of transparency as the feature importance to the classification performance. A kernel-based Support Vector Machine is useful for data with irregular distribution or unknown distribution. The residual neural network (ResNet) model has been used to examine explainability methods[63]. Since the LSTM model was studied in the literature, we compare three kinds of machine learning models, decision tree-based Random Forests, kernel-based SVMs and deep neural networks as ResNet.

We prepare three datasets with vulnerabilities labelled, including the OWASP Benchmark project [64], the Juliet test suite for Java [11] and 15 Android applications from the previous Android study [65]. OWASP and Juliet have the vulnerability types available online. Android study follows the labels published in the paper [65].

Each line of code is parsed to produce tokens including variables, preserved keywords, operators, symbols and separators.

First, we analyze the token statistics and observe if any significant characters of the tokens. For each dataset OWASP, Juilet, and Android project, we separate the tokens in vulnerable files from tokens in non-vulnerable files and plot the token frequency distribution in Figure 3, Figure 4, and Figure 5 respectively.

For the OWASP project (shown in Figure 3), tokens are mostly grouped in the counts of occurrence that are less than 20. Beyond 20 occurrences, the counts of tokens are significantly smaller. In Juliet source code (shown in Figure 4), the distribution of the token frequency has more peaks than the OWASP token distribution. In all the Android source code (shown in Figure 5), the tokens are mostly grouped with occurrences less than 30.

The charts show two facts: (1) the frequency distribution of each project varies. This could be a factor in downgrading the accuracy of cross domain learning; (2) the high-frequency tokens are neutral such as “main”, “string_builder”. This indicates the feature representation is learnt mainly from tokens with less frequency. These two facts relate to the experimental results presented in section 4.4.

For each experiment, we evaluate the performance of vulnerability detection using traditional Information Retrieval metrics. In the context of this study, True positives (TP) are the correct identification of source files with vulnerabilities. True negatives (TN) are the correct identification of source files without vulnerabilities. False positives (FP) are the incorrect identification of source files with vulnerabilities. False negatives (FN) are the incorrect identification of source files without vulnerabilities. Based on these metrics, we define the following metrics to measure the performance of vulnerability detection:

• •

  The precision (P), which is the probability a file that is classified as vulnerable, is truly vulnerable.

  $$P=\frac{TP}{(TP+FP)}$$

  (3)

• •

  The recall (R) which is the probability that a vulnerable sample of code is classified as vulnerable.

  $$R=\frac{TP}{(TP+FN)}$$

  (4)

• •

  The f-measure (F1) is the harmonic average of precision and recall.

  $$F1=2*\frac{1}{\frac{1}{P}+\frac{1}{R}}$$

  (5)

• •

  The false positive rate (FPR) is the proportion of negative cases incorrectly identified as positive cases.

  $$FPR=\frac{FP}{(FP+TN)}$$

  (6)

• •

  The area under the precision-recall curve (PR AUC) summarizes the information in the precision-recall curve.

• •

  The area under the receiver operating characteristic curve (ROC AUC) shows the capability of the model to distinguish between classes.

To aggregate the above values of metrics for comparing different hypotheses, we define an aggregation formula below. We first add all the metrics defined above in Eq(7), $\forall s\in S\{P,R,F1,1-FPR,ROCAUC,PRAUC\}$. We then normalize the value using Eq(8), where $N$ is the number of comparison cases.

To answer the research questions, we first design experiments for each project and then developed experiments for cross-project validation. In the first set of experiments, each project’s source code is divided into training and testing partitions. The vulnerability detection models are trained per project and tested on the same project. Due to space limitations, we include the result tables in the appendix LABEL:sec:appendix.

We use the $z_{i}$ to compute the p-value from a paired t test for each comparison. When $\alpha\geq 0.5$ the hypothesis is accepted to be significantly different. Otherwise, the hypothesis is rejected. Table 6 shows the different p-values obtained for each hypothesis. A detailed analysis is presented in the following sections.

Observation: We observe whether tokenization with removing code comments and/or special symbols may improve or create noise for the detection. We run experiments with the tokens including the comments and symbols (Table 9 in Appendix) compared to tokens without them (Table  10). Each table shows the learning scores of 153 experiments (= (3 models x 3 embeddings) per project x 17 projects). Each experiment produces six scores that compute the value of $z$. Totally 306 data points of $z$ are used to compute p-value from the t test. The p-value is compared to the significance level $\alpha$.

Hypothesis Analysis: Hypothesis (1) is defined as follows:

1. 1.

   There is a statistically significant difference between the results obtained from using all tokens vs. using tokens without comments and symbols.

According to Table 6, the p-value obtained for hypothesis (1) is less than the significance level 0.05. This hypothesis is then rejected, which means there is no statistically significant difference between the two tokenization strategies.

Conclusion: We conclude that comments and symbols do not affect the learning of software vulnerabilities from source code in our experiments.

Observation: Feature extraction techniques convert tokens into a vector of features. In this experiment, we observe the effects of three feature extraction techniques, including (1) bag-of-words, (2) word2vec embedding and (3) fastText. We run experiments with features obtained from bag-of-words (Table 9 and Table 10). For each embedding technique we have 102 experiments (= (3 models x 2 tokenization methods) per project x 17 projects). Each experiment produces six scores that compute the value of $z$. Totally 204 data points of $z$ are used to compute p-value from the t test. The p-value is compared to the significance level $\alpha$.

Hypothesis Analysis: To compare those three vector representation techniques, we consider the following hypotheses:

1. 2.

   There is a statistically significant difference between the results obtained from using bag-of-words and
   word2vec as embeddings.

2. 3.

   There is a statistically significant difference between the results obtained from using bag-of-words and fastText as embeddings.

3. 4.

   There is a statistically significant difference between the results obtained from using word2vec and fastText as embeddings.

According to the Table 6 the p-values obtained from t test of hypothesis (2) and hypothesis (3) are accepted, but hypothesis (4) is rejected. This means there is a statistically significant difference between bag-of-words and word2vec and also between bag-of-words and fastText. However, there is no statistically significant difference between word2vec and fastText. Additionally, the results obtained from the classification show that, on average, the precision and recall of the experiments with bag-of-words are 6% more than the performance of the other embedding methods. That indicates that bag-of-words is better than the other two models in the learning process of vulnerabilities in our experiments.

Conclusion: We choose bag-of-words as the best way to generate embeddings for the remainder of the experiments.

Observation: In addition to the NLP-based method, we use architectural metrics as structural features to learn vulnerability patterns from the software repositories. We compare the features with code tokens only. We run experiments using the architecture metrics only and compared them to using the architecture metrics with the bag-of-words features. The Table 11 in Appendix shows the learning score of 51 (= 3 models x 17 projects) experiments for each feature we use. Totally 102 data points of $z$ are used to compute p-value from the t test. The p-value is compared to the significance level $\alpha$.

Hypothesis Analysis: The effects of using architecture metrics, extracted from the structures of the project, are explored via three hypotheses:

1. 5.

   There is a statistically significant difference between 1) the results obtained from using tokens, vs. 2) the results obtained from using the architectural metrics.

2. 6.

   There is a statistically significant difference between 1) the results obtained from only using tokens, vs. 2) the results obtained from using the combination of architectural metrics and tokens.

3. 7.

   There is a statistically significant difference between 1) the results obtained from only using the architectural metrics, vs. 2) the results obtained from using the combination of tokens and architectural metrics.

As shown in the Table 6, the p-values indicate hypotheses (5) and (7) are accepted, while hypothesis (6) is rejected. This means there is no significant difference when using tokens as input features with or without architectural metrics. Hypotheses (5) and (7) further indicate the tokens have a stronger influence on the learning performance than the architectural metrics.

Conclusion: We choose to use tokens without architectural metrics for the remainder of the experiments.

Observation: We aim to identify whether a certain machine learning model produces better vulnerability detection. We run experiments with each of the three models to compare them (Table 9 and Table 10 in Appendix). For each model, the table shows 102 experiments (= (2 tokenization methods x 3 embeddings) per project x 17 projects) Each experiment produces six scores that compute the value of $z$. Totally 204 data points of $z$ are used to compute p-value from the t test. The p-value is compared to the significance level $\alpha$.

Hypothesis Analysis: To compare the models, we define these three hypotheses:

1. 8.

   There is a statistically significant difference between the performance of the random forest model and the SVM.

2. 9.

   There is a statistically significant difference between the performance of the random forest model and the ResNet.

3. 10.

   There is a statistically significant difference between the performance between the SVM and the ResNet.

According to Table 6, the p-values of the three hypotheses (8), (9), and (10) are less than the significance level, $\alpha$. All three hypotheses are accepted. Overall the random forest model performs better than the SVM and ResNet in most of the experiments with a precision and recall higher by an average of 8%.

Conclusion: We decide to use the random forest as the model to learn the patterns of vulnerabilities in the cross-project validation experiments.