AI-Based Intrusion Detection Systems for In-Vehicle Networks: A Survey

The papers reviewed in this study were selected using PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [93] protocol. Figure 1 illustrates the PRISMA selection process.

Google Scholar is considered a good starting point, as it helps avoid bias for any specific publisher [172]. This study selected Google Scholar as the search engine. Although this is a comprehensive review, there will still be good papers not selected, as they are out of defined eligibility criteria. Only the papers written in the English language were considered. Due to these limitations, this work may have overlooked some important works.

Seo et al. [140] proposed a novel IDS for IVNs based on GAN. The GAN-Based Intrusion Detection System (GIDS) can learn to detect unknown attacks using only benign data. Two models—a generative model to capture the data distribution and a discriminate model to estimate the probability that a sample comes from the training data—were used in GIDS. Two discriminators were combined to detect both known and unknown attacks. Hyundai’s YF Sonata was used as a testing vehicle to generate the Hacking and Countermeasure Research Labs Car Hacking (HCRL CH) dataset [51] and launched DoS, fuzzy, RPM, and gear spoofing attacks. The first discriminator achieved 100% average accuracy, whereas the second discriminator achieved 98% average accuracy. As per the authors, GIDS is difficult to manipulate by an attacker due to the pre-trained DL method. Further, it can detect intrusions in real time. The same data pre-processing technique used in the work of Seo et al. [140] was used for the GAN-based IDS proposed by Chen et al. [25]. They replaced the GAN’s true false classifier with additional double classifiers. This model outperformed the model proposed by Seo et al. [140] for all attacks. The HCRL CH dataset was used to evaluate the GAN and convolutional adversarial autoencoder-based model [58]. This model was trained with unlabeled data to learn the normal patterns. Experimental results showed that the proposed model outperforms baseline models for both detection rate and latency. However, this work was only limited to message injection attack detection. Kavousi-Fard et al. [75] also used a GAN-based model for IVN anomaly detection. Frequency for each CAN ID was used as the feature, which limits the detection of attacks such as masquerade attacks.

Avatefipour et al. [13] proposed an Anomaly Detection Model (ADM) based on a modified One-Class Support Vector Machine (OCSVM). The authors used a modified bat algorithm as the parameter optimization algorithm. For the model evaluation, CAN bus data from an unmodified vehicle and two other public CAN bus datasets [88] were used. Isolation Forest (IF) and classical OCSVM were selected to benchmark the proposed model. The proposed model outperformed the baseline models for DoS attacks. Furthermore, the computational time requirement was sufficient to deploy the proposed solution in a real-world environment. Similarly, the OCSVM-based ADM proposed by Al-Saud et al. [3] used ID frequencies as features. They used the social spider optimization algorithm to find the best support vector regression parameters. A real vehicle dataset with a DoS attack was used to evaluate the proposed model. Despite the promising results of both models, the lack of testing against various real-life attack scenarios can be identified as a common limitation. IDS, which used an adapted streaming data IF algorithm [143], showed that CAN traffic demonstrates insignificant concept drift. Therefore, model retraining based on a sliding window did not improve the model performance.

Kalutarage et al. [71] developed a context-aware anomaly detector for monitoring cyber attacks on CAN bus using sequence modeling. N-gram distributions were used to build the sequence model. The authors have estimated maximum likelihood estimators (MLEs) for each N-gram and developed an algorithm to calculate the anomaly certainty ratio using pre-build N-gram models, a predefined threshold, and observation windows. The anomaly certainty score and predefined threshold were used to classify the message as anomaly or benign. Experimental results that utilized the HCRL CH dataset showed that the proposed model could identify RPM and gear spoofing attacks with higher accuracy. This was tested against only two types of spoofing attacks, and the computational efficiency of the algorithm was not discussed. An ensemble model based on GRU and time-based models was proposed by Rajapaksha et al. [134] to overcome the computational inefficiency of N-gram-based models and to detect a wide variety of attacks. The GRU-based model predicts the next CAN ID, whereas the time-based model monitors ID inter-arrival times. Three public datasets and 16 attacks were used to evaluate the proposed model and achieved a greater than 99% F1-score for 13 attacks. This work showed ensemble models’ effectiveness in detecting a wide variety of CAN bus attacks. Shi et al. [144] proposed a temporal convolutional network based IDS using word embedding of CAN IDs. Experimental results on the HCRL CH dataset showed good detection for fuzzy and DoS attacks. Treating CAN ID sequences as word sentences, Nam et al. [118] introduced a Generative Pretrained Transformer (GPT) model to learn the pattern of a normal CAN ID sequence. Deviations from normal patterns were identified as attacks. They combined two GPT networks in a bi-directional manner. This outperformed the single unidirectional GPT model. The proposed model was designed only to detect injection attacks. A bag-of-words approach was used in the work of Baldini [16] to detect intrusions in IVNs. They generated frequency counts of the presence of words for each sliding window and used them as a feature for ML models. Marchetti and Stabili [107] proposed an anomaly detection algorithm considering the recurring pattern of CAN IDs. They created a transition matrix with all possible transitions between consecutive CAN IDs. This is equivalent to 2-grams in the N-gram based model used in the work of Kalatarage et al. [71]. Instead of probabilities, the authors used true and false status to generate the transition matrix. In the attack detection stage, a validated transition matrix was used to check the availability of consecutive ID sequences and identified new IDs as normal or anomalies. A real dataset with replay, bad injection, and mixed injection attacks was used for model evaluation. Experimental results showed a low detection rate of around 20% to 40% for replay attack. Compared to Kalatarage et al. [71], this might produce a high false-positive rate, as this approach assigned labels for each message, whereas Kalatarage et al. [71] classified messages based on a window.

Desta et al. [32] implemented an IDS using an LSTM model. Two approaches were used for the performance evaluation. The first approach compared the predicted ID with the actual ID. This only achieved 60% accuracy. The second approach used log loss and a predefined threshold to identify anomalies and achieved reasonable accuracy. A real car dataset was used with attacks such as insertion, drop, and illegal IDs. In another work [33], Desta et al. improved the previous IDS [32] by training separate LSTM models for each ID and combining them to create a single anomaly signal. This achieved 100% detection for all attacks. Song and Kim [147] proposed a self-supervised method for in-vehicle anomaly detection using noised pseudo-normal data. The proposed model included two models: a generator and a predictor. The generator used an LSTM model similar to Desta et al. [32] to predict the next CAN ID and the predictor used the Reduced Inception-ResNet model proposed in the work of Song et al. [149] to detect anomalies. They used the noised pseudo-normal data generated by the generator model to train the ADM. The HCRL CH dataset was used for the performance comparison, and the proposed model outperformed the other algorithms such as SVM, OCSVM, and CNN. Sharmin and Mansor [142] proposed IDS based on IF to detect message injection attacks using CAN ID timing as features. The HCRL CH dataset was used to evaluate the algorithm with gear and RPM spoofing attacks. This was trained using a one-class approach. Linear time complexity and low resource requirement were the advantages of the proposed solution.

Kuwahara et al. [86] used two types of features including total counting and ID counting in a CAN sequence window to detect malicious messages. Both supervised and unsupervised methods were used as the classification approach. Principal Component Analysis and k-d tree were used to optimize the nearest neighbor discovery. Experimental results that used a real vehicle dataset with simulated attacks showed that the supervised method outperformed the unsupervised method. However, supervised methods require attack data during the training and fail to identify unknown attacks. Han et al. [54] proposed an anomaly detection and attack identification method for IVNs. They calculated statistical features for event-triggered intervals for each CAN ID. Calculated feature values were used to train ML models such as DT, RF, and XGBoost to identify attacks. Experimental results that used two real datasets with realistic attacks showed high anomaly and attack detection capability.

The sequential behavior of CAN data can be used to detect anomalous behavior. Using this property, Song et al. [149] proposed a Deep Convolutional Neural Network (DCNN)-based IDS to protect the CAN bus from cyber attacks. During the injection attack period, a sequential pattern of the ID changes due to frequent frame injection. The authors capitalized on this change to detect message injection attacks. Inception-ResNet was used as the DCNN model with 29 \(\times\) 29 \(\times\) 1 input and binary output. The HCRL CH dataset was used to evaluate the proposed solution. The DCNN model outperformed the baseline models for all attack types. Further, the proposed model required 5 ms to process one sample, which included 29 CAN messages under GPU acceleration. Desta et al. [34] also used a CNN-based IDS trained on recurrence plots. Deployment on NVIDIA’s Jetson TX2 showed that higher detection latency of 117 ms. A lightweight multi-attack quantized ML model deployed using Xilinx’s DL processing unit IP on a Zynq Ultrascale+ (XCZU3EG) FPGA was proposed in the work of Shankar [141]. This system used CNN as the detection algorithm and outperformed the baseline models. A blockchain-based federated forest Software-Defined Networking (SDN)-enabled IDS was proposed by Aliyu et al. [8]. This system created a RF model to detect attacks. Fourier transformation was used to create features from CAN IDs. The HCRL OTIDS [48] dataset was used for the performance evaluation. The usage of blockchain reduces the risk of adversary poisoning, which potentially improves the security of the AI model. This model helps vehicle owners and manufacturers keep the underlying data confidential.

Jedh et al. [67] used an LSTM model to detect malicious message injections in the CAN bus. They used message sequence graphs of CAN IDs at successive time windows to calculate Pearson and cosine similarities, which were then used as the features for the LSTM model. A real vehicle dataset with fabricated RPM and speed messages was used to evaluate the model performance. Experimental results showed that the detection capability of the algorithm depends on the selected window size. Refat et al. [136] also used a similar graph-based model as a CAN IDS. They extracted seven graph properties as features and used them to train SVM and KNN ML models. The proposed model achieved a greater than 95% F1-score for DoS, fuzzy, and spoofing attacks in the HCRL CH dataset.

One of the major drawbacks of ID-based IDSs is their limited ability to detect attacks that manipulate the message payload without changing ID sequences or frequencies. However, even for an attack that manipulates only the payload, CAN ID sequences might change due to event-triggered messages in the CAN bus [122].

Chockalingam et al. [30] tested LSTM and OCSVM to detect anomalies in CAN frames. The authors used a real dataset and created fuzzing and misplaced non-anomalous packets. Experimental results have shown that the OCSVM model produced a 7% false-positive rate using the linear kernel. The non-linear kernel took much time to optimize. The LSTM model outperformed OCSVM. However, this was tested with two attack types, and complete evaluation results for individual attack types are unavailable. Tomlinson et al. [161] used a one-class compound classifier to detect attacks in the IVN. Payload values of three separate CAN IDs were considered for the analysis. The authors used fuzzing attacks to test the classifier. However, evaluation results were not promising and produced many false positives. They proposed ensemble detection methods for CAN IDs to overcome problems that arise with one classifier. Tomlinson et al. [162] used OCSVM, the compound classifier, and the Local Outlier Factor (LOF) to identify attacks in CAN data values. Experiment results showed that OCSVM and LOF outperformed the compound classifier. However, the results were not acceptable to use in real-world situations.

Narayanan et al. [120] built an anomaly detection system called OBD_SecureAlert. This concept is based on formulating a sequence of CAN bus messages as a time series ML problem. The authors considered the vehicle movement as a sequence of states dependent on its previous state. All observations of a sliding window were used to determine the posterior probability of that sequence. The anomalous status was identified by considering the probability of such sequence and a defined threshold. Evaluation results showed that OBD_SecureAlert works well with single and multiple observations. However, this was tested against limited anomalous states, and identifying specific sensor data in CAN messages is challenging. Levi et al. [90] proposed a HMM-based hybrid anomaly detection algorithm using a new temporal detection technique. The authors used a rule-based engine to monitor different interfaces and generated events using raw data. These events from different interfaces were used to train a HMM model as a normal behavior. Experimental results showed that the proposed model achieves high AUC and F1 measures with low false-positive and false-negative rates. They proposed a hybrid deployment approach that uses a rule-based client to send data to the back-end and run the detection algorithm on the cloud. This cloud-based platform facilitates monitoring car fleets in addition to individual cars. Despite these advantages, the proposed method is based on events and relevant attributes. It is challenging to obtain the complete list of events and attributes.

Taylor et al. [157] introduced an LSTM model for CAN bus anomaly detection. The underlying concept of this approach is that the ML model can be trained to predict the next packet data value, and its deviation from the actual value can be used to identify the anomalies. Nineteen CAN IDs were selected, and they trained different LSTM models for each ID. The authors selected a real dataset and created five types of attacks considering three basic cases: new packets are added, expected packets are missing, and the payload of packets is unusual. Experimental results have shown that different IDs achieve various ROC curve values ranging from 0.17 to 1 for five attack types. For the practical use of the proposed model, the threshold needs to be selected with more experiments. Further, since they considered separate models for IDs, it cannot utilize the inter-dependencies between IDs for anomaly detection. Tanksale [154] proposed an LSTM-based IDS using CAN measurements such as longitudinal acceleration, RPM, and brake position. This model can predict the future values of selected CAN signals based on previous signal values. The deviation between the predicted and actual values was used to identify the anomalous signals. A dataset collected from 10 cars was used to train the algorithm, whereas a subset of the dataset with injected anomalous frames was used for the performance evaluation. The proposed model showed greater than 98% accuracy with a 1% to 2% false-positive rate. A similar LSTM-based model with an embedding layer was proposed in the work of Balaji and Ghaderi [15] for CAN payload values. The usage of payload values of other IDs as the context ensured the capturing of inter-ID correlation. Experimental results on gear attacks of the HCRL CH dataset showed a relatively lower detection rate. Hanselmann et al. [55] introduced CANet, an LSTM-based IDS to identify attacks in the CAN bus. They used separate LSTM models for each CAN ID and concatenated the outputs to a single latent vector. This was trained in an unsupervised manner, and the difference between initial and reconstructed signal values was used to define the normal status. The authors used a real vehicle dataset with 13 IDs and a synthetic dataset (SynCAN) with 10 IDs for the performance evaluation. Six attacks and two baseline models were selected to evaluate the model performance. Experimental results showed that the proposed model achieved a higher detection rate and low false-positive/negative rates for all attacks in both datasets. It outperformed the baseline models for almost all attack types. However, their anomaly score is only feasible with a limited number of signals and hence not suitable for real-world applications.

Novikova et al. [124] developed an autoencoder to detect anomalous payload in the CAN bus. A large dataset from nine different vehicles was used to train the algorithm. They identified 81 signals common to all vehicles and grouped them into 32 subgroups of three signals based on the signal relationships. Experimental results on modified payload values showed a higher detection rate. The SynCAN dataset was used to evaluate the reproducibility of the proposed model. Detection rates of plateau, continuous change, and playback attacks were 99%, 94%, and 95%, respectively. Subgroups require a separate autoencoder for each group, and deploying a large number of DL models under a resource-constrained IVN is a challenging task. Kukkala et al. [83] proposed a GRU-based recurrent autoencoder to detect anomalies in the CAN bus. The SynCAN dataset was selected as the evaluation dataset, and separate autoencoder models were trained for each ID. Signal-level intrusion scores between predicted and true signal values were used to identify the anomalous signal values. The authors used accuracy as the evaluation metric without considering the highly imbalanced status of the dataset. Longari et al. [103] also proposed a similar LSTM autoencoder model. A real-world dataset from an Alfa Romeo Giulia Veloce with injected anomalous frames was used for model training and evaluation. Kukkala et al. [84] improved the GRU-based recurrent autoencoder [83] by replacing the GRU layer with an LSTM layer and introducing the self-attention mechanism. Instead of identifying the threshold value as the intrusion score, the proposed model used OCSVM as the attack detector. Thiruloga et al. [158] introduced a novel anomaly detection framework using a temporal CNN. The proposed model used a DT-based classifier as the attack detector. This model achieved an improvement of 32.7% in false-negative rate compared to the best-performing baseline model. All of these models [83, 84, 103, 158] processed ID-wise data independently and trained separate models for each ID. This limits the capability of detecting signal correlations to detect anomalies such as collective anomalies. The deep contractive autoencoder-based ADM proposed by Lokman et al. [102] achieved 91% to 100% detection rates for three attacks. Gherbi et al. [45] proposed a multivariate time series representation to represent CAN payload data and used autoencoder-based DL models such as the fully connected network, CNN, LSTM, and temporal convolution network to detect intrusions in the CAN bus. The proposed models achieved a higher F1-score for all attacks in the SynCAN dataset. However, the proposed feature matrix might be inefficient for a real vehicle due to the many ECUs.

Narasimhan et al. [119] trained an autoencoder-based model and used a Gaussian mixture model to identify intrusions in the CAN bus. All other autoencoder-based models discussed earlier used the reconstructed signal for the anomaly detection, whereas this model used the latent space as the input to the Gaussian mixture model. A real dataset of Mercedes ML350 with DoS and fuzzy attacks was used for the performance evaluation. However, this dataset included only four CAN IDs; therefore, results obtained through this model might hinder the practical performance of the model. A multi-layer denoising autoencoder model was used by Wei et al. [171]. He et al. [57] proposed the Hybrid Similar Neighborhood Robust Factorization Machine Model (HSNRFM). They used data fields of similar neighbors to enhance the feature representation. The factorization machine model used the second-order interaction features to predict the final probability of anomalous outcomes. Both of these models [57, 171] used only two CAN IDs to train and evaluate the proposed models. Tanaka et al. [153] employed a density ratio estimation method using a Neural Network (NN) model. This approach is based on the change detection method to detect packet frequency changes. However, this model also used only three CAN IDs for the model evaluation. CNN-LSTM with attention mechanism based IDS was proposed by Sun et al. [151]. This model used one-dimensional convolution to extract the abstract features, whereas bi-directional LSTM was used to extract time dependence. They only considered the continuous physical values extracted from the 64-bit payload. Bit flip rate was used to identify continuous fields. The CAN Signal Extraction and Translation Dataset (HCRL-SET) provided by HCRL was used for the model evaluation with simulated payload attacks. The proposed model outperformed the baseline models. They evaluated the model detection time under attack in a real vehicle. This showed that the attacks could be detected within 5.7 ms in a real vehicle. However, the proposed model has a few limitations, including selecting a subset of signal values and ignoring payload correlation between different IDs.

Kang and Kang [73] proposed a DNN-based IDS for IVNs. CAN payload was selected to generate the features, whereas mode and value information were used as the dimensionality reduction technique. Initial weights for the DNN model were obtained using a separate Deep Belief Network. Finally, the authors used a template-matching technique to compare the training sample and a new CAN packet to identify the attack scenarios. The authors used a simulation dataset with packet injection attacks for the evaluation. Experimental results showed that the DNN model outperformed baseline models. Zhou et al. [185] proposed a DNN and a triplet loss network for CAN bus anomaly detection. The proposed model used the distance between anchor samples and the positive and negative examples to identify anomalies. Experimental results showed the real-time detection capability of the algorithm. However, both Kang and Kang [73] and Zhou et al. [185] relied on mode and value information of CAN data, and identifying these information is quite challenging without having the DBC file. Zhang et al. [181] proposed DNN-based IDS for the CAN bus. They have used Gradient Descent with Momentum and Gradient Descent with Momentum and Adaptive Gain to improve efficiency and accuracy. Performance evaluation was done using a real dataset collected from a car. Experimental results revealed that the proposed model could detect replay attacks with a high detection rate. Further, it was noticed that the Gradient Descent with Momentum and Adaptive Gain algorithm achieved faster convergence compared to the ADM algorithm. The authors had access to the sensor values and used those as separate features. However, these values cannot be distinguished without having the DBC file or knowledge about the CAN payload.

Fenzl et al. [40] introduced a continuous field classification algorithm to identify the payload value alignments. Then, a DL-based approach was used to identify the anomalous fields. Datasets from Renault Zoe electric car and manipulated signals were used to evaluate the model performance. Interestingly, their continuous field classification approach showed slightly better detection capability than the field classification obtained by the DBC file. However, these attacks are not realistic, as they were created during the post-processing. This approach does not reflect the inter-dependencies among variables. Martinelli et al. [109] used four k-Nearest Neighbor (KNN) classifiers to identify four types of attacks that target the CAN bus. These algorithms include two types of fuzzy-roughKNNs the discernibility classifier and a fuzzy unordered rule induction algorithm. Fenzl et al. [41] used DTs modeled through genetic programming to detect intrusions in the CAN bus. Features and feature boundaries selection were based on the CAN DBC files. Three datasets, including the HCRL CH dataset, were used to evaluate the performance of the algorithm. Experimental results showed that the proposed algorithm achieved similar detection capability as ANN algorithms with much-improved detection time.

Wang et al. [168] proposed an LSTM model with optimized parameters. They implemented an LSTM model to evaluate the vulnerability of an ADM with a black-box attack. A dataset of the velocity of a vehicle was collected from the CAN bus for evaluation purposes. The threshold to identify anomalies can be defined considering the maximum MSE. Evaluation results for the ADM under attack scenarios are not available, as the authors only focused on the evaluation results of the victim model for attacks. Similarly, Li et al. [92] used an LSTM model to test an adversarial attack defending system. To this end, they used the LSTM model proposed by Khan et al. [78]. This model achieved a 98% accuracy for the used simple attack scenario.

Berger et al. [20] tested NN, LSTM, SVM, and OCSVM algorithms for IVN attack detection. Experimental results that used the HCRL CH dataset showed that NN outperformed other models. A mobile edge-assisted LSTM-based anomaly detection approach was proposed by Zhu et al. [187] to overcome the computational limitations of IVNs. A real-time performance of 0.61 ms was observed in the proposed model with around 90% accuracy. Gao et al. [43] introduced a new in-vehicle IDS based on DL and SOEKS (set of experience knowledge structures). Experimental results that used a real vehicle dataset showed that usage of SOEKS and information entropy improved attack detection. Barletta et al. [17] proposed an unsupervised Kohonen SOM (self-organizing map)-based anomaly detector for the CAN bus. They integrated Kohonen SOM with a k-means clustering algorithm using a distance-based approach. This model was tested with DoS, fuzzy, gear, and RPM spoofing attacks. They compared this with the traditional approach where the k-means algorithm processed a neuron’s codebook vectors. Experimental results have shown that the proposed technique outperforms the traditional approach for all attack datasets. Leslie [89] proposed an ensemble hierarchical agglomerative clustering-based model to detect malicious traffic in heavy-duty ground vehicles. The author used a dataset related to the SAE J1939 protocol, which is based on the CAN bus. This was evaluated using spoofed engine speed messages and showed a higher detection rate.

Lin et al. [96] proposed a deep denoising autoencoder-based model to detect injection attacks on IVNs. They used an evolutionary-based optimization algorithm to overcome premature convergence and find the optimum network structure. Experimental results that used the HCRL OTIDS and two real datasets showed that the proposed model outperformed selected baseline models. Nakamura et al. [117] proposed a hybrid model of a LightGBM-based supervised model and an autoencoder-based unsupervised model. Time differences of consecutive CAN IDs, CAN ID, and payload values were used as the features. Experimental results that used the HCRL Survival Analysis (HCRL SA) dataset showed that the hybrid model outperformed the pre-trained LightGBM model. However, a comparison between the pre-trained and autoencoder models is not available to make a fair comparison of the hybrid model performance. Qin et al. [133] proposed an LSTM-based anomaly detection algorithm to detect the abnormal behavior of the CAN bus. Experimental results have shown that the proposed model can detect anomalous data with greater than 90% accuracy. Further, the authors tested this with two more vehicles, and the performance was not good enough to generalize the model to other vehicles. An LSTM model with an improved feature processing technique was used in the work of Khan et al. [76] for IVN malicious activity detection. The HCRL CH dataset based experimental evaluation outperformed the baseline models for both detection rate and latency. The LSTM autoencoder-based model proposed by Ashraf et al. [12] also used the HCRL CH dataset. Packet count and bandwidth of the outbound traffic of a fixed window were used as the features. These features are only suitable for detecting injection attacks. Zhou et al. [186] proposed an autoencoder model with dedicated models for each CAN ID. An improved IF method with data mass was used to detect tempering attacks in the work of Duan et al. [36]. This was evaluated using a simulation environment and outperformed the OCSVM and LOF algorithms.

Tian et al. [159] proposed an IDS based on the Gradient Boosting Decision Tree for the CAN bus. Nine features were used for the classification, including the payload of CAN message and entropy-based feature. They changed the payload values of a real dataset to create abnormal messages. Experimental results showed that the true-positive rate was 97.67% and the false-positive rate was 1.2%. However, this was tested with a very basic attack scenario of CAN payload values changing, and real-world attack detection will be much more complex. Wasicek et al. [169] implemented a CAID (context-aware IDS) framework using ANN to identify manipulations in IVNs. CAID is equipped with three modules: the monitor module reads and aggregates information, the detectors module identifies anomalies, and the reporter module connects with the user. Features used for ANN model include vehicle speed, engine RPM, fuel rate, and calculated load. This model was evaluated using a real vehicle for chip tuning and power boxing manipulations. Experimental results have shown that it could accurately recognize the manipulated attacks. However, this experiment was done in a constrained environment, whereas the real-world environment might be quite different. The ANN-based lightweight model proposed by Basavaraj and Tayeb [19]. This model marginally outperformed the baseline models. Alshammari et al. [9] proposed KNN and SVM algorithms to cluster and classify DoS and fuzzy attacks in the CAN bus. As per the experimental results, KNN outperformed the SVM algorithm for both attacks of the HCRL CH dataset. However, the DoS detection rate was comparatively low compared to the fuzzy attack.

Zhang et al. [184] proposed an IDS for the CAN bus considering the balance between the efficiency of the rule-based approach and the high detection rate of the DNN-based approach. The first stage, which is the rule-based approach, enables efficient anomaly detection. CAN frames, which pass the rule-based detection model, send to the DNN-based detection model to further identify undetected anomalies. Evaluation against five types of attacks using three real datasets showed high detection rates and low false-positive rates for all datasets. However, evaluation results with regard to five attack types are not included in this work. Similarly, Zhang and Ma [183] introduced a hybrid approach for in-vehicle intrusion detection. Datasets related to four real vehicles were used for the performance evaluation. This approach was only applicable to periodic messages. Weber et al. [170] proposed a hybrid IDS that is capable of identifying both point and contextual anomalies. The authors used eight classes of sensor data defined by Müter et al. [116]. They used LODA (a lightweight online detector of anomalies) [131] as the classification algorithm. Synthetic CAN data with an altered sequence was used to evaluate the proposed model. Despite the promising results, this was tested with limited simplified anomaly scenarios. Rule-based and RF-based hybrid IDS was proposed by Kang et al. [72]. Time interval, data field differences, and ID lag values were used as the features. The RF model showed a poor detection capability than the rule-based approach.

Kalkan and Sahingoz [70] used six different ML models—RF, bagging, ADA boosting, NB, Logistic Regression, and NN—to compare their attack detection capability of a large CAN dataset. The authors could achieve a promising detection rate using simple ML algorithms with default parameters. However, they did not discuss the dataset creation or features used to train the algorithms. Similarly, Alfardus and Rawat [6] also used ML algorithms such as KNN, RF, SVM, and Multilayer Perceptron (MLP) to detect CAN bus attacks. Moulahi et al. [114] used RF, DT, SVM, and MLP to compare the detection capability. Features related to time, ID, DLC, and payload values were used. Performance evaluation using the HCRL OTIDS dataset showed very low detection capability for fuzzy attacks. Amato et al. [10] used NN and MLP-based models to detect attacks on the HCRL CH dataset. Dong et al. [35] did a comparative study on supervised versus semi-supervised ML for IVNs anomaly detection. Minawi et al. [113] used Random Tree, RF, Stochastic Gradient Descent with hinge loss, and NB to detect gear and RPM spoofing, DoS, and fuzzy attacks in the HCRL CH dataset. CAN ID and payload values were used as the features. Except for the fuzzy attack, all attacks were detected with a 100% F1-score. Anjum et al. [11] also used the HCRL CH dataset to evaluate the XGBoost-based CAN IDS. Park and Choi [129] used multi-labeled hierarchical classification as the intrusion detection model. Experimental results that used the HCRL SA dataset showed that the proposed model outperformed the selected baseline models. The same dataset was used in the NN-based IDS proposed by Francia and El-Sheikh [42]. The main objective of the proposed approach was to identify vehicle models and anomalies. All of these works [6, 10, 11, 35, 70, 113, 114, 129] can be considered as basic ML and DL model comparisons for CAN attacks. None of these models has the capability to detect unknown attacks. The XGBoost classifier outperformed the VGG16 model for gear and RPM spoofing attacks in the work of Lin et al. [94]. Aksu and Aydin [1] proposed a meta-heuristic algorithm called the modified genetic algorithm for the CAN feature selection. This can be considered as a dimensionality reduction approach. They used ML models such as SVM and DT to evaluate the effectiveness of the feature selection.

Suda et al. [150] proposed LSTM-based IDS, which utilized the time series features of the CAN frame. These features include frame interval (derived from the time), ID, and payload values. Data was collected from a real vehicle and used modified ID, data field, and flooding as attacks to evaluate the system. Khan et al. [77] proposed an LSTM-based attack detection model for IVNs. They used two attack-free CAN bus datasets—HCRL CH and the AEGIS repository [68]—to create replay and amplitude-shift attacks. Experiment results for replay and amplitude-shift attacks showed that the LSTM model achieved the best accuracy for both datasets. Even though LSTM recorded the best results comparatively, these figures are not promising, as accuracy and precision values were around 80% to 90% and recall values were around 30% to 40%. Further, the DBC file and processed data with features are hard to find. Xiao et al. [177] introduced a novel RNN-based IDS by optimizing LSTM and GRU architectures and using a simplified attention model to make the model lightweight. The RF algorithm was used as the classification algorithm using the features generated by the RNN model. In contrast, CAN ID, DLC, and payload fields were used as the input features for the RNN model. They validated their approach using the HCRL OTIDS dataset and compared the performance with eight variants of the proposed model. However, the RF algorithm learns only to detect attacks in the training dataset and may fail to detect new attacks. Ma et al. [106] proposed a GRU-based lightweight IDS for CAN bus intrusion detection. They also used a low-complexity feature extraction algorithm to extract features from CAN frames. The proposed model showed near real-time performance and a higher detection rate than the baseline models. However, the usage of the supervised learning approach limits novel attack detection. An attention-based technique was used in the work of NasirEldin et al. [121]. An attention layer was used to capture the most important part of the data, whereas a self-attention layer was used to identify the relationship between each data element. They used positional encoding to capture the positional information. Performance evaluations that used HCRL CH data showed that the proposed model marginally outperformed baseline models, including an LSTM model.

Hossain et al. [62] proposed an IDS for the CAN bus based on LSTM. The authors used both binary and multi-class classification to evaluate the IDS with vanilla LSTM and stacked LSTM models. Experimental results that used the HCRL SA dataset showed that the proposed vanilla LSTM model outperformed the compared survival analysis method. Since both CAN ID and payload have been considered in the model, it can detect both point and contextual anomalies. They used the same model in another work [61]. Hossain et al. [60] used a CNN model instead of the LSTM model proposed in their other work [62]. They collected datasets from three cars and injected anomalous frames to create attacks. The proposed model achieved a high attack detection rate for all attacks. Due to the supervised learning approach used, both of these models [60, 62] cannot detect unknown attacks. The CAN bus attack detection framework introduced by Tariq et al. [155] utilized both rule-based and DL (LSTM) models. DoS, fuzzing, and replay attack were used to evaluate the proposed model. The ensemble model achieved better accuracy than the individual rule-based or LSTM model for all attacks. Detection time analysis showed that the average detection time delay was 0.02 seconds. This was evaluated against three simple attacks that changed the ID frequency significantly. They also introduced CANTransfer, a transfer learning based IDS for CAN bus [156] using the same data, features, and attacks. The authors trained a convolutional LSTM model (ConvLSTM) as a binary classification problem. One-shot transfer learning was used to retrain the model to detect new attacks. DoS attack was used during the training phase, and fuzzing and replay attacks were used with one-shot transfer learning. They could achieve 26.60% performance gain compared to the best baseline model. A deep transfer learning based P-LeNet method used in the work of Mehedi et al. [110] outperformed the baseline models. Transfer learning will help reduce the need for collecting a large amount of data to detect each new type of attack. LSTM-based simple IDS proposed by Kishore et al. [80] outperformed the traditional ML models such as RF and XGBoost.

Rehman et al. [137] proposed CANintelliIDS, a novel approach to detect intrusions in the CAN bus based on CNN and attention-based GRU models. Unlike other approaches that predicted binary classes, this model predicted the attack type. The authors evaluated this algorithm with a single attack data sequence and mixed attack data sequence separately. Binary output was compared with recent state-of-the-art baseline models (e.g., [149, 156]). It outperformed all models with a maximum 5.32 F1-score gain. This work proved that DL-based ensemble models could be successfully used to detect different attacks on vehicle networks. However, the computational efficiency of the proposed approach has not been discussed. Lo et al. [100] used a hybrid model of CNN and LSTM networks for in-vehicle attack detection. CNN was used to extract spatial features, whereas LSTM was used to extract temporal features from CAN data frames. Experimental results that used the HCRL CH dataset showed an approximately 100% detection rate. They used the same model in the work of Aldhyani and Alkahtani [4].

Ale et al. [5] used a Deep Bayesian Learning (DBL) model to detect and analyze car hacking behaviors. Experimental results that used the HCRL CH dataset showed a slightly lower accuracy than a deterministic DL model. However, the DBL model is capable of providing more information about its prediction, which can help further analysis of abnormal behaviors. Islam et al. [65] developed a hybrid quantum-classical NN to detect an amplitude shift cyber attack on the CAN bus. The usage of the DBC file for feature creation reduces the generalization capability of the proposed model. DNN and incremental learning based IDS was introduced by Lin et al. [95] to address the driving environment and behavior changes. Predicted class labels of the DNN model were used as the labels for online model updates. This approach has a risk of reducing the model performance when the predictions of the original model are incorrect. Rumez et al. [138] employed a similar approach like Kalutarage et al. [71] to develop a hybrid anomaly detection framework for diagnostics communication. In addition to the sequence-based model that uses the n-gram distribution for CAN IDs, the authors used the byte-based model to utilize the CAN messages payload for attack detection. Real and synthetic datasets with three attack types were used for the model evaluation. Their detection framework is only limited to automotive diagnostic communication.

In ID-based detection, Kalutarage et al. [71] used IDs in hexadecimal format without using any data pre-processing. Limited or no data pre-processing helps reduce the detection latency of the IDS. However, this limits the wide variety of attack detection capabilities, as some attacks do not significantly change the raw data properties. Marchetti and Stabili [107] also used the hexadecimal IDs and created a transition matrix to learn possible ID transitions. This approach is computationally more efficient than calculating 2-grams in the work of Kalutarage et al. [71]. SVM-based models [3, 13] used ID frequency as the feature. The selection of this feature as the only feature makes the model lightweight. However, it limits the attack detection capability of infrequent IDs. Both Avatefipour et al. [13] and Marchetti and Stabili [107] assigned labels for each message, whereas Kalutarage et al. [71] assigned labels for message windows. The window-based approach helps reduce the false positives even though it requires a small additional time to process all frames in the window. In addition, certain types of attacks might not create point anomalies. Instead, they might create contextual or collective anomalies. The window-based approach is highly beneficial in identifying these types of contextual and collective anomalies. Han et al. [54] and Kuwahara et al. [86] used observation windows to extract features. Kuwahara et al. [86] used a fixed time window and selected total-counting feature and ID-counting feature. The total counting feature counts the number of messages in a window. In contrast, the ID-counting feature is a vector, each of whose elements is the number of messages associated with each ID. Han et al. [54] considered a window between consecutive CAN IDs and defined it as an event-triggered interval. Mean, variance, first quartile, third quartile, interquartile range, skewness, and kurtosis were calculated for each ID as the features. Sharmin and Mansor [142] calculate the time between consecutive CAN IDs as the feature. All of these feature values [54, 86, 142] change as a result of injection attacks. However, the amount of feature value change depends on the attack injection rate. In addition, these features are insufficient to detect more sophisticated attacks such as masquerade attacks. IDSs proposed by both Jedh et al. [67] and Refat et al. [136] used graph-based techniques to extract features. Refat et al. [136] converted a window of CAN IDs into a graph and extracted graph properties such as the number of nodes, number of edges, radius, diameter, density, reciprocity, average clustering coefficient, and assortative coefficient to use as features for ML models. Similarly, Jedh et al. [67] calculated cosine similarity and Pearson correlation between successive time windows. However, graph-based feature selection might be computationally expensive when a vehicle has a large number of ECUs.

An LSTM-based IDS [32] converted the hexadecimal IDs into integer values from 0 to the number of CAN IDs and then numbers were one hot encoded to consider each CAN ID as a class. Output was the softmax probability for each class. Similarly, for the same task, Rajapaksha et al. [134] converted the hexadecimal IDs into integer values. Instead of one-hot encoding, they created the word vectors for each CAN ID. This helps learn the semantic relationship of CAN IDs better than the one-hot encoding approach. However, the size of the word vectors needs to be selected carefully to keep the model lightweight and efficient. In the work of Seo et al. [140], each digit of raw CAN ID was converted to binary and then to one-hot encode vector (concatenation of three binary numbers of 16 digits), which was finally used as an image for the algorithm. The input size selected was 64, and it assigned one label for an image. If the image included at least one attack packet, it was considered an attack image. Song et al. [148] converted a 29-bit ID to binary and considered 29 consecutive IDs into one frame (making it 29 \(\times\) 29 two-dimensional grid data frame). The same logic used by Seo et al. [140] was used to define the class label. In the work of Berger et al. [32], 20 consecutive CAN IDs were selected and converted into one-hot encode vectors. This resulted in a 20 \(\times\) 42 (42 IDs) input frame to LSTM. Output was softmax probabilities relevant to 42 IDs. Both of these approaches converted CAN ID sequences to two-dimensional grids. This data structure makes it possible to use image processing algorithms such as CNN on CAN data.

Autoencoder-based models used by some authors [83, 84, 103] split the datasets into groups based on the CAN IDs, and each group was processed independently. Even though this reduces the model complexity of each model, dependencies among CAN IDs cannot be exploited to detect some attacks. Novikova et al. [124] grouped the data considering the dependencies among payload signals. This overcomes the aforementioned issue. However, they used a manual approach for the grouping. The SynCAN dataset used to evaluate autoencoder-based models [55, 83, 84, 124] includes pre-processed signal values. This can be done by converting an 8-byte hexadecimal payload into eight integer (or decimal) values and then scaling the integer values to the 0-1 range. This helps avoid slow and unstable training due to the large range of signal values. Instead of treating 8 bytes as eight features, Tomlinson et al. [162] combined these bytes into several fields if they represent a single reading from a sensor. They used an algorithm to concatenate these fields for each CAN ID. Similarly, Fenzl et al. [40] also used a field classification approach to align 8 bytes into several fields. This [40, 162] payload value concatenation helps dimensionality reduction. Concatenation algorithms need to be accurate and efficient to avoid incorrect field classifications.

The majority of CAN frame-based IDSs [20, 106, 155] used the timestamp (as a time interval for consecutive IDs), decimal ID, and payload fields as features. In contrast, limited works [117, 133] used the binary ID and payload instead of decimal conversion. This increases the dimensionality of the features. In the work of Tian et al. [159], in addition to the payload values, the authors created entropy-based features using ID and time. The creation of additional features such as entropy-based features helps detect a wide variety of injection attacks. Zero padding was used by the authors [117, 133, 155, 156] to replace the missing values of the CAN payload. This helps obtain a uniform data field to train AI algorithms. For the DNN-based IDS proposed by Zhang et al. [184], they used the CAN ID, number of occurrences in the past second, relative distance between IDs, and change in system entropy as the features. Zhang and Ma [183] extracted additional features using the CAN payload field. The new features set includes the CAN ID, Hamming distance between the data fields of two normal consecutive CAN ID, entropy of data filed, and bytes of importance (most important two bytes). Usage of the Hamming distance of payload data helps detect attacks on infrequent IDs. However, since all features are calculated for ID groups, inter-correlation among IDs cannot be detected for any feature. This limits the wide range of attack detection capability of the proposed solution. Khan et al. [76] used data pre-processing to enhance the scalability and performance efficiency of the proposed IDS. This included feature conversion, feature reduction, and feature normalization. Principal Component Analysis was used to feature reduction. Experimental results showed that the feature pre-processing led to 19.31% accuracy improvement compared to raw data.

The performance of an AI-based algorithm highly depends on the data it uses. The usage of low-quality data in AI algorithms leads to bad outputs. The poor quality of publicly available datasets can be identified as a limitation for IDS research in this area. These datasets in particular suffered from the simulation of the attack under realistic conditions. Section 5.5 discussed the benefits and drawbacks of the existing datasets. It is difficult to evaluate, compare, and improve the IDSs without having a proper dataset. There are three reasons for this limitation. First, it is costly to produce vehicle networks data with real attacks except for simple message injection attacks. Second, there is a risk involved with creating realistic attack data for running vehicles on public roads and, third, issues with the disclosure of sensitive information [166]. Often, researchers used datasets created by themselves with synthetic attacks that were not reflected in real-world situations. Considering the publicly available CAN datasets, the ROAD dataset [166] will be the best dataset to use to evaluate and compare in-vehicle IDSs, as it consists of multiple real attack types along with benign datasets under various driving conditions. Usage of multiple datasets is another feasible solution.

ECUs in a modern vehicle generate about 2,000 CAN frames per second to the CAN bus [140]. Therefore, even 1% of false-negative rate miss 200 attack frames per second. Missing a detection of a particular attack may lead to serious safety problems. Detecting a normal message as an attack also brings unwanted countermeasures that cause inconvenience for the driver. Even though 99% of detection accuracy is a good achievement in other application domains, this might not be enough in this domain. Various attacks and different characteristics make it hard to improve the detection rate. The majority of proposed solutions were not able to detect low frequent (low-volume) attacks, as these attacks have little effect on CAN bus data behavior. DL-based (particularly unsupervised methods) ensemble models and hybrid models are possible future research directions to improve the accuracy and detection capability of low frequent attacks. Moreover, transformer-based models are also a possible direction, as these models have successfully been used in other domains for time series forecasting [174]. Combining CAN data frame features with physical characteristics features to improve the performance will be an interesting direction to study in the future.

Message transmission in IVNs happens in real time. IVN IDSs should detect and take appropriate countermeasures in real or near real time. However, the majority of reviewed DL-based literature was not able to detect attacks in real or near real time. DL-based IDSs can utilize the large number of computational resources in the cloud to improve the detection time. However, since vehicles are moving objects, connection stability is a key factor to consider for cloud deployments. Edge computing will be another option despite the computationally constrained environment. This is an area to explore in the future with different experiments under real-world conditions.

IDSs in the literature evaluated their proposed models using collected real data, public real data, or synthetic data. Since the evaluation of collected real and synthetic data was done under different adversarial settings, it is challenging to compare their security uniformly. Performance comparisons for public real or synthetic datasets are possible, as they share the same benign and attack data. However, the majority of reviewed works did not use common evaluation metrics to compare their security. For example, a few works used accuracy and precision, whereas others used F1-score, precision, and recall as the evaluation metrics. Comparing the performance using only one common metric such as precision or recall is challenging. The Matthews correlation coefficient [26] is considered a more reliable metric for binary classification problems such as anomaly detection. Some works only presented visual evaluations such as bar or line charts. This also makes the model comparison much more difficult. Therefore, it is vital to use a few metrics such as the Matthews correlation coefficient, F1-score, precision, recall, false-positive rate, and false-negative rate to make a fair comparison. Accuracy as a metric is inappropriate in this case, as all discussed attack datasets in Section 5.5 are highly imbalanced. Detection latency is another critical factor for an in-vehicle IDS. However, only limited works evaluated their models for detection latency and discussed the used experimental platform. Hence, including these metrics in the evaluation criteria helps identify more effective methods and improve attack detection in IVNs.

Unsupervised learning (OCSVM, autoencoders) is well suited for the CAN bus, as CAN bus dataflow is predictable and constant [161]. Another reason is that collecting attack data is more expensive in vehicle networks than benign data collection. In unsupervised learning, only benign data is used to model normal behavior, and a threshold is determined to detect anomalies. However, one major limitation for this approach is the need for a large dataset that sufficiently represents the normal profile. To this end, streaming learning can be considered as a future research direction. The model needs to be deployed in a vehicle, and the parameters and threshold can be updated for a sufficiently large time to cover various normal driving conditions.

Usually, AI algorithms require a large dataset for model training. However, as discussed earlier, the availability of realistic attack and benign datasets is a major limitation in this domain. Learning from a few examples is a key challenge for IVN attack detection. Algorithms such as transfer learning [163], one-shot learning [167], and zero-shot learning [176] were used in other domains such as image recognition and Natural Language Processing applications to address this challenge. Adapting them to vehicle network data could be future research directions to utilize small datasets to detect new attack types.

The majority of reviewed literature was not focused on deployment requirements and countermeasures. ECUs in vehicle networks have limited memory storage, computing power, and bandwidth. IDS development and deployment are bounded by these resources. IDSs can be deployed as host-based IDSs or network-based IDSs. Host-based IDSs are not a viable solution for vehicles, as they require a change in ECUs that are not cost effective. Therefore, deploying a network-based IDS as an additional node in the CAN bus would be the most appropriate solution. Deploying the IDS in clouds can be considered as another feasible solution.

Even though AI-based models can identify anomalies in vehicle networks with a high detection rate, these models themselves are vulnerable to cyber attacks such as white-box, black-box, and tempering attacks. None of the discussed literature focuses on protecting the proposed models from cyber attacks except the models proposed by Aliyu et al. [8] and Li et al. [92]. Aliyu et al. [8] claimed that blockchain technology could be used to improve the security of IDSs. In contrast, Li et al. [92] proposed a defending scheme against adversarial attacks on LSTM-based IDSs. Developing a secure IDS for IVNs under an adversarial setting is a challenging future research direction. Solutions used in other domains can also be adapted to vehicle networks.