Real-time DDoS flood attack monitoring and detection (RT-AMD) model for cloud computing

DDoS attack types

DDoS attacks are classified based on targeted protocols such as the network/transport or application level.

• Network/Transport level DDoS attacks: These attacks occur mostly when using network and transport layer protocols such as TCP, UDP, and ICMP. These attacks are further categorized into three types:

  1. Volume attacks: The attacker aims to consume all the resources of the target servers and make them unavailable by sending many packets (bandwidth/flooding attack) such as TCP flood, ICMP flood, etc.

  2. Protocol attacks: In this type, the attack consumes all resources and intermediate connection media as a firewall by exploiting protocol vulnerabilities and bugs as TCP SYN flood, TCP SYN-ACK flood, etc.

  3. Reflection and amplification attacks: Attempts to consume the victim’s resources by sending fake request messages (such as ping requests) through spoofing the victim’s IP address to the reflectors. The reflectors send a high volume of response messages to the victim’s IP address such as Smurf attacks.

• Application-level DDoS attacks: These attacks aim to consume services’ resources or cause starvation of resources to disrupt customers through establishing requests, overloading the application servers. The most popular type of attack at this level is HTTP flooding attacks. Many studies have classified DDoS attack at the application level based on the following categories (Jaafar, Abdullah & Ismail, 2019):

1. Session flooding attack: Servers’ resources are disabled from being launched when session request rates are high. These requests usually are higher than those generated by valid users.

2. Request flooding attack: Sends sessions that contain more requests than the valid users.

3. Asymmetric attack: Wastes resources such as CPU and memory of the server by sending sessions with high-workload requests.

4. Slow request/response attack: Uses all server resources by sending incomplete requests slowly to keep the servers in the waiting state to receive data.

Intrusion detection system

An intrusion detection system (IDS) is a device or software tool that identifies unusual events by monitoring the network traffic to distinguish the normal from abnormal behaviors (Kaur, Kumar & Bhandari, 2017). IDS is classified into three main categories based on the analysis method. The choice between methods depends on several factors, such as the anomaly type, applied environment, security level required, and the cost (Kaur, Kumar & Bhandari, 2017).

The IDS methods classifications are signature-based, anomaly-based, and hybrid detection (Alzahrani & Hong, 2017). Signature-based detection, also called knowledge-based or rule-based detections, is suitable for detecting known attacks by comparing captured behavior. Anomaly-based detection, also known as behavior-based, is useful to detect unknown attacks. These techniques compare the observed behavior with normal behavior to detect abnormal events. Hybrid-based detection works by combining the detection techniques mentioned above. The performance of this detection depends on the types of techniques chosen. Table 1 shows the advantage and limitations of these detection methods (Alzahrani & Hong, 2017).

Signature-based detection

Tanrıverdi & Tekerek (2019), Bakshi & Dujodwala (2010), and Modi et al. (2013) proposed a signature-based detection method. Tanrıverdi & Tekerek (2019) presented the detection of web attacks using a blockchain-based attack detection model. The signatures listed in this study are automatically updated by blockchain technology. An additional advantage of this proposed method is that it can be used against zero-day attacks. Bakshi & Dujodwala (2010) presented a method to distinguish between the normal and abnormal traffic in VMs. It uses Snort to analyze the collected traffic to determine the attack. The virtual server then drops packets coming from the specified IP address. Modi et al. (2013) presented a method to detect known attacks and derivatives of known attacks. It also uses a Snort tool to detect the known attacks from network traffic. The detected attack is input to a signature DB to predict derivatives of the attack by using signature as a priority.

Anomaly-based detection

Hong et al. (2017) and Kemp, Calvert & Khoshgoftaar (2018) presented an anomaly-based solution to detect slow HTTP attacks, a type of DDoS attack. Hong et al. (2017) developed a software-defined networking (SDN) controller; however, Kemp, Calvert & Khoshgoftaar (2018) deployed the proposed model using machine learning techniques. It selected eight classification algorithms for predictive models: random forest, decision trees, K-nearest neighbor, multilayer perceptron, RIPPER (JRip), support vector machines, and Naïve Bayes. The authors used the Weka machine learning toolkit to build these models. ANOVA was used to compare the values of slow attack detection among the eight models. They evaluated the models by area under the receiver operating characteristic curve (AUC), receiver operating characteristic (ROC) curve graphs, true positive rate (TPR), and false positive rate (FPR).

Singh, Jeong & Park (2016), Lima Filho et al. (2019), Wang et al. (2014), and Sreeram & Vuppala (2019) presented anomaly-based solutions to detect HTTP attacks. Singh, Jeong & Park (2016) used a multilayer perceptron with a genetic algorithm (MLP-GA)-based method for detecting DDoS attacks on incoming traffic. Authors in Singh, Jeong & Park (2016) identified four features to detect application-layer attacks; first is the number of HTTP counts, referring to the count number of requests per IP address. It is assumed that any single IP address that sends more than 15–20 HTTP GET/POST requests is an attack. Second is the number of IP addresses, referring to the number of IP addresses in small windows time. It is assumed the attacks have more than 20 IPs in windows time. The third was the constant mapping function; the attacker’s ports are different from legitimate users’ as the one used by the attacker is varied and remains open. The fourth is fixed frame length; codes with fixed frame length are considered as an attack.

In comparison, Lima Filho et al. (2019) proposed an online smart detection system for DoS/DDoS attack detection. The detection approach used the random forest tree algorithm to classify various types of DoS/DDoS attacks such as flood TCP, flood UDP, flood HTTP, and slow HTTP. However, Wang et al. (2014) proposed a detection scheme for HTTP-flooding (HTTP-Soldier) based on web browsing clicks. HTTP-soldier used the large-deviation principle of webpage popularity to be able to distinguish between normal and abnormal traffic. The large-deviation probability-based detection may affect some normal users. The authors mentioned that their proposed scheme could not detect a single uniform resource locator (URL) attack. The false positive of a Multi-URL attack with the most popular webpages is 12.2%, but the false positive of Multi-URL attack with the least popular webpages is at 17.1%. This solution can achieve high performance in Multi-URL attacks with the most popular web pages. Sreeram & Vuppala (2019) used bio-inspired machine learning metrics to detect HTTP flood attacks to achieve fast and early detection. Authors in Sreeram & Vuppala (2019) adopted the Bat algorithm, which has low process complexity, as a bio-inspired approach.

Choi et al. (2014), Aborujilah & Musa (2017), and Sahi et al. (2017) presented a cloud-based flood attack detection method. Choi et al. (2014) proposed a method to integrate the detection of DDoS flood attacks and MapReduce processing in a cloud computing environment. The proposed framework consists of three parts: first is the packet and log collection module (PLCM), which analyses packet transmission and web server logs in the first part. Second is the pattern analysis module (PAM), which produces the pattern for DDoS attack detection. Finally is the detection module (DM), which detects DDoS attacks by comparing them with a normal behavior model. However, Aborujilah & Musa (2017) presented the detection based on the covariance matrix approach. The proposed detection was divided into training and testing phases. A training phase aimed to construct a normal network traffic profile. The testing phase was to detect any abnormal traffic by the deviation between the normal and any other network traffic. The normal traffic is captured from end-users browsing the Internet in their cloud, whereas the flooding attack traffic is generated using the PageRebooter tool. It was evaluated by using the confusion matrix and present results for an internal and external cloud environment, while Sahi et al. (2017) proposed a detection model for TCP flood DDoS attacks. This model selected different classifiers, the least squares support vector machine (LS-SVM), Naïve Bayes, K-nearest, and multilayer perceptron.

Lin, Ye & Xu (2019), Li et al. (2019), and Nawir et al. (2019) presented an anomaly-based detection method for detecting DDoS attacks. The proposal of Lin, Ye & Xu (2019) and Li et al. (2019) used deep learning techniques. Lin, Ye & Xu (2019) used long short-term memory (LSTM) to build the neural network model, which is a specific recurrent neural network structure (RNN). Li et al. (2019) used LSTM and gated recurrent units (GRU) recurrent neural networks. It used BGP and NSL-KDD datasets. The best accuracy achieved was using the BGP dataset in the range of 90–95%. However, the proposed of Nawir et al. (2019) used machine learning algorithms. The authors in [Reff9+] selected five machine learning algorithms to include Naïve Bayes (NB), averaged one dependence estimator (AODE), radial basis function network (RBFN), multi-layer perceptron (MLP), and J48 trees. A comparison was drawn between these algorithms based on accuracy and processing time. A UNSW-NB15 dataset was selected in this experiment.

Haider et al. (2020) proposed a deep convolutional neural network (CNN) framework for efficient DDoS attack detection in SDN. This proposed framework has been evaluated using hybrid state-of-the-art algorithms on CICIDS2017 dataset. Hwang et al. (2020) proposed an unsupervised deep learning model for early network traffic anomaly detection, namely D-PACK based on CNN. The experimental results show low false-positive rate and high accuracy. Novaes et al. (2020) proposed using short-term memory and fuzzy logic for DDoS attack detection and mitigation in SDN. The proposed system consists of three phases: Characterization, anomaly detection, and mitigation. The evaluation of this system has been conducted using CICDDoS2019 dataset with archived accuracy of 96.22%.

Hybrid-based detection

Hatef et al. (2018), Zekri et al. (2017), and Novaes et al. (2020) proposed a hybrid-based detection system. While Hatef et al. (2018) and Zekri et al. (2017) deployed cloud environment approaches. Hatef et al. (2018) is a hybrid intrusion detection approach in cloud computing (HIDCC). The applied detection was a combination of signature-based and anomaly-based detection techniques. The Snort tool is used for known attacks (signature-based detection) by employing the Apriori algorithm to generate a pattern from derived attacks. Both clustering and classification algorithms are applied for the undetectable attack through Snort. The clustering module receives and determines the input packet based on the sample vector. Then the classifier module determines the final class of the packet through algorithm C4.5 as a decision tree classifier according to the found cluster. However, Zekri et al. (2017) proposed using machine learning for anomaly detection and Snort technique for signature detection. Three algorithms were selected: decision tree, Naïve Bayes, and K-means algorithms. The decision tree achieved the best accuracy. The proposed Saleh, Talaat & Labib (2019) was applied in real time and deployed in three stages. First, the Naïve Bayes feature selection (NBFS) technique was employed to reduce the dimensionality of sample data. Second, optimized support vector machines (OSVM) were used to reject the noisy input sample as it might have caused misclassification. Finally, the attacks were detected by prioritized K-nearest neighbors (PKNN) classifier. This proposed scheme takes time in the first and second stages at feature selection and outlier rejection before attack detection.

This study will explore the use of data mining techniques (classification) in real-time detection. We will focus on the network/transport level as it is the core layer of network architecture. There are few existing studies on volume-based network/transport-level DDoS attack detection in the cloud environment. Moreover, there are very few studies that have proposed online detection with a high detection rate. This study will employ different classification algorithms (machine learning) that suit our needs to build detection models. We will then evaluate these models by comparing them against two main factors: efficiency of detection and detection rate.

Main components

The proposed RT-AMD model consists of two main components: monitoring and detection.

1. The monitoring component is responsible for monitoring the network traffic for requests coming to the webserver on the cloud and extracting the traffic from the network log. If the traffic from the log is found in the Blacklist, an alert with corresponding information is sent to the cloud admin. If it is not in the Blacklist, it will move into the next component, detection.

2. The detection component uses trained classifiers to detect if the incoming traffic behavior is normal or abnormal. When abnormal behavior occurs, it will alert the system, send information to the admin, then update the Blacklist with the new traffic information. Otherwise, it can access the cloud and benefit from its services. An overview of the proposed environment is shown in Fig. 2.

At the beginning, the admin needs to log in/register to benefit from the RT-AMD tool. Once the registration is done, RT-AMD will send a verification code for the entered email to ensure that the email address is correct. The tool will then start monitoring the network traffic to detect any malicious behavior. Once an attack occurs, the RT-AMD will detect it, then send all information assigned to this traffic to the admin email. This makes it easier for the admin to act on any malicious behavior. A flowchart of the proposed detection framework is shown in Fig. 3.

Data mining classification methods

Data mining analyzes large volumes of data to identify and predict any threats; this helps to solve problems and reduce risks. Data mining can answer questions that have typically been time-consuming to address manually by using several statistical techniques to analyze data in various ways.

Recently, high arrival rates of online data streams have imposed high resource requirements on data mining processing systems. Datastream mining (also known as stream learning) is a technique of extracting knowledge structures from an unbounded and ordered sequence of data that exists over time (stream data) (Gomes et al., 2017). Some differences between stream data mining and traditional data mining are shown in the following (Ramírez-Gallego et al., 2017):

1. Machine learning of streaming scenarios cannot retrieve all of the data of the dataset in advance. Data chunks are available in a stream, one by one, or bundle by block.

2. Data arriving over time in streams may be unlimited in their number, resulting in difficulty storing all arrival data in the memory.

3. Data from streams need to be analyzed quickly to provide real-time response and prevent data waiting.

4. Some incoming stream data may lack accurate class labels because of the label query’s high cost for each data stream.

Building knowledge from stream data mining is called incremental learning (Ramírez-Gallego et al., 2017). Incremental learning has received much attention from both academia and industry. It is a machine learning approach where knowledge is applied as new instances arrive, and what has been learned is updated according to the new instances (Xie & Lam, 2006).

There are several techniques in data mining, such as regression, clustering association, and classification. Different techniques serve different purposes. However, most data mining techniques usually applied for this area are classification techniques. Classification is a type of supervised learning that predicts the class label to which data belongs. The classifier works by obtaining a training dataset containing several attributes and class labels. The classifier then tests the dataset to evaluate the model.

The proposed framework will employ the classifiers in the detection component of the framework with two class labels. The attributes assigned to normal behavior are labelled “normal” and those to abnormal behavior “anomaly.” Some classifiers were found to have better detection results than others based on the recommendations in the related works (Kemp, Calvert & Khoshgoftaar, 2018; Lima Filho et al., 2019; Saleh, Talaat & Labib, 2019). We selected the following: Naïve Bayes, decision trees, K-nearest neighbor, and random forest. These classifiers have been selected to build the predictive models.