Securing The Smart Grid A Comprehensive

Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2017.DOI
Securing the Smart Grid: A

Comprehensive Compilation of Intrusion
Detection and Prevention Systems
PANAGIOTIS I. RADOGLOU-GRAMMATIKIS, PANAGIOTIS G. SARIGIANNIDIS, (Member, IEEE)
1
University of Western Macedonia, Department of Informatics & Telecommunications Engineering, Greece
Corresponding author: Panagiotis G. Sarigiannidis (e-mail: psarigiannidis@uowm.gr).
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement
No. 787011 (SPEAR).
ABSTRACT The Smart Grid (SG) paradigm is the next technological leap of the conventional electrical
grid, contributing to the protection of the physical environment and providing multiple advantages such as
increased reliability, better service quality, as well as efficient utilisation of the existing infrastructure and
the renewable energy resources. However, despite the fact that it brings beneficial environmental, economic
and social changes, the existence of such a system possesses important security and privacy challenges,
since it includes a combination of heterogeneous, co-existing smart and legacy technologies. Based on
the rapid evolution of the Cyber-Physical Systems (CPS), both academia and industry have developed
appropriate measures for enhancing the security surface of the SG paradigm by, for example, integrating
efficient, lightweight encryption and authorisation mechanisms. Nevertheless, these mechanisms may not
prevent various security threats, such as Denial of Service (DoS) attacks that target on the availability
of the underlying systems. An efficient countermeasure against several cyberattacks is the Intrusion
Detection and Prevention System (IDPS). In this paper, we examine the contribution of the Intrusion
Detection and Prevention Systems (IDPS) in the SG paradigm, providing an analysis of 37 cases.
More detailed, these systems can be considered as a secondary defence mechanism, which enhances the
cryptographic processes, by timely detecting or/and preventing potential security violations. For instance,
if a cyberattack bypasses the essential encryption and authorisation mechanisms, then the IDPS systems
can act as a secondary protection service, informing the system operator for the presence of the specific
attack or enabling appropriate preventive countermeasures. The cases we study focused on the Advanced
Metering Infrastructure (AMI), Supervisory Control and Data Acquisition (SCADA) systems, substations
and synchrophasors. Based on our comparative analysis, the limitations and the shortcomings of the
current IDPS systems are identified, while appropriate recommendations are provided for future research
efforts.
INDEX TERMS Advanced metering infrastructure, Cyberattacks, Intrusion detection system, Intrusion
prevention system, SCADA, Security, Smart grid, Substation, Synchrophasor.
I. INTRODUCTION distribution lines. On the other hand, as illustrated in Fig. 1

[1], SG provides the required infrastructure and the com-
he Smart Grid (SG) constitutes a technological evolu-
T tion of the traditional electrical grid, by introducing
Information and Communications Technology (ICT) ser-
munication channels that allow the real-time bidirectional
interaction between the consumers and the utility compa-
nies. This communication can provide multiple benefits such
vices. The functionality of a typical electrical grid is mainly as processes that enable auto metering and maintenance,
based on the energy generation, transmission and distribu- self-healing, efficient energy management, reliability and
tion processes. More concretely, it includes power plants, security [2]–[6].
step-up transmission substations, step-down transmission
substations, distribution substations and transmission and However, despite the fact that SG introduces multiple
This work appears in IEEE Access: The Multidisciplinary Open Access Journal
VOLUME 4, 2016 1
P. Radoglou-Grammatikis and P. Sarigiannidis, "Securing the Smart Grid: AComprehensive Compilation of IntrusionDetection and
Prevention Systems", IEEE Access, 2019, pp. 1-30.
Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS
advantages, it also introduces crucial security challenges, characterize these systems and provide research directions
since it combines heterogeneous communications networks for future work.
[7] such as Internet of Things (IoT) [8]–[11] devices, indus- In particular, the rest of this paper is organised as follows:
trial devices [12], wireless components and Wireless Sensor Section II discusses the related surveys in the literature
Networks (WSNs) [13] characterized by various security and provides the motivation and contributions of our study.
threats [14], [15]. In addition, the integration of smart Sections III and IV introduce an overview of SG and
devices, such as smart meters, that communicate with each IDPS systems respectively. Section V presents and explains
other without human intervention induces more security the requirements that should characterise these systems.
concerns. Furthermore, the necessary existence of legacy Section VI provides an analysis of 37 IDPS cases, by
technologies, such as conventional Supervisory Control and investigating their main characteristics. Section VII inter-
Data Acquisition (SCADA) systems, increase the potential prets, evaluates and compares the results exported from the
risks, since these systems may not integrate modernised previous analysis. Finally, Section VIII provides trends and
security solutions. The security breaches in SG mainly research directions concerning the security of SG, focusing
target on the availability, integrity and confidentiality of on IDPS systems, while section IX presents the concluding
individual entities [14], [15]. In more detail, the different remarks of this study.
kinds of Denial of Service (DoS) attacks aim to disrupt
the network services and cause significant damages such
as a power outage [16]–[18]. A characteristic example was
the cyberattack against a Ukrainian substation resulting
in the power outage for more than 225,000 people [19].
On the other hand, the false data injection attacks [20]–
[23] can modify the data of smart meters in order to
succeed in more economical pricing. Finally, various types
of Man in the Middle (MiTM) can violate the privacy of
the systems [24], [25]. Furthermore, a remarkable and more
dangerous category of cyberattacks, which threatens the SG
architecture, is the Advanced Persistent Threat (APT). This
term specifies a set of organised and long duration attacks
by security specialists against a particular target, such as
ICT Communication flows
politicians and industries. Examples of these attacks are Electrical flows
Stuxnet [26], Duqu [27], Flame [27], and Gauss [27].
An Intrusion Detection System (IDS) and even its evolu- FIGURE 1: An abstract architecture model of the SG [1].
tion, the Intrusion Prevention System (IPS), can operate as
a second line of defence in a communication network, by
enhancing the operation of the encryption and authorisation
Generation System Attacks, Transmission System Attacks,
mechanisms. For instance, if a cyberattack bypasses the Protocol and Subsytem
Threats
Distribution System, SCADA Attacks, Substation Attacks, AMI
Attacks, Protocol Attacks
encryption and authorisation mechanisms, the IDS or IPS
can timely inform the security administrator or perform
State Estimation Anomalies, Power System Control Centre
appropriate preventive countermeasures. The term Intrusion Anomalies Anomalies, Substation Anomalies, SCADA Anomalies, Smart
Detection and Prevention System (IDPS) will be used Metering Anomalies, Synchrophasor Anomalies
from now on in this paper for referring to both previous

terms. In general, the rapid progress of computer networks Physical Threats
Natural Disasters, Environmental Threats, Human-Caused Physical
Threats
necessitated the development of appropriate mechanisms
that have the ability to automate the process of detecting
or/and preventing possible security violations. The presence FIGURE 2: SG Cyberattacks.
of these systems in SG is required, since the security policy
violations in this ecosystem may cause dangerous situations
and disastrous accidents. A significant advantage of the II. MOTIVATION AND CONTRIBUTION
specific systems is that they possess the ability to recognise Although SG can provide multiple benefits, like better
zero-day attacks by using artificial intelligence mechanisms. energy management and improved reliability, its indepen-
Therefore, in this paper, we provide an analysis of 37 dent and interconnected nature generates at the same time
cases of IDPS systems devoted to SG, by evaluating and critical cybersecurity vulnerabilities that in turn can lead
comparing the cyberattacks that they are able to detect, to a wide range of consequences such as power outage,
their methodology, the detection performance and finally brownout, energy theft, energy consumer privacy breach.
the consumption of computing resources. Based on this In particular, most of the communication protocols adopted
analysis, we specify the limitations and shortcomings that by SG are characterised by severe security gaps, since do
2 VOLUME 4, 2016
not comprise authentication and access control mechanisms, components, but also SCADA systems, substations and syn-
thus enabling possible adversaries to launch various cyber- chrophasors. Furthermore, contrary to previous works, we
physical attacks. Fig. 2 depicts a pictorial view of such analyse thoroughly each case, by investigating its architec-
attacks against SG. A characteristic example of cyberattacks ture, the detection technique, the kinds of cyberattacks that
against a critical infrastructure was the Stuxnet worm [26], are detected, the resources consumption, performance, the
which exploited four zero-days vulnerabilities. Furthermore, utilised datasets and the software packages. In conclusion,
the diversity and complexity of communications that take the desired purpose of this paper is to constitute a stopping
place in SG, as well as the huge volume of data generated by point for the interested parties that intend to work with
the various subsystems, hinder the adoption of conventional the IDPS systems for SG. The contribution of our work
security measures. Therefore, it is clear that the presence is summarised in the following sentences:
of IDPS systems is vital for the entire operation of SG • Identifying the requirements for effective IDPS
and mainly for ensuring the essential security requirements: systems devoted to protecting the SG components:
Confidentiality, Integrity and Availability (CIA). Since SG consists of several and heterogeneous tech-
Several studies have examined the security issues in the nologies, components and communication interfaces,
SG paradigm, by analysing security challenges, threats and the conventional IDPS systems (coming from computer
corresponding countermeasures. Some of these are listed in networks) cannot meet the security requirements of
[8], [14], [15], [28]–[38]. Since that the nature and means SG. In this paper, we identify these requirements
of cyberthreats evolve rapidly, the creation of corresponding that subsequently are utilised to evaluate the various
surveys and review papers is quite crucial, as they present relevant IDPS found in the literature.
state of the art and identify possible challenges, security • Providing a comprehensive and comparative anal-
gaps and research directions. Other works follow a more ysis of IDPS systems devoted to protecting SG: In
precise approach, by examining the security issues regarding particular, we investigate thoroughly 37 IDPSs capable
particular protocols that are commonly utilised in the SG of detecting cyberattacks against either the entire SG
communications. Concretely, in [39], [40], the authors ex- ecosystem, AMI, SCADA, substations and synchropha-
amined the security issues of IPv6 over Low-Power Wireless sors.
Personal Area Networks (6LoWPAN) and IEC 61850 [41], • Identifying existing weaknesses of the current IDPS
[42] standards respectively. Similarly, in [43] the authors systems for SG: Based on our analysis and taking into
investigate various encryption and authentication protocols account the requirements of IDPS systems for the SG
for SG. Nevertheless, only a few studies have examined paradigm, we identify the weaknesses of the existing
the contribution of the IDPS systems for the contemporary IDPSs found in the literature.
electrical grid. Specifically, in [44], the authors provided an • Identifying the appropriate IDPS for the entire SG
extensive study and comparison of multiple IDPSs devoted ecosystem: Accordingly, based on our analysis and
to the Cyber-Physical Systems (CPSs), such as SG. Sim- after identifying the weaknesses of the existing IDPS,
ilarly, in [45], [46] the authors investigated various IDPS we specify the appropriate IDPS for SG, as well as its
instances concerning the protection of IoT; SG is considered type and attributes.
as the largest use case of IoT [47]. On the contrary to the • Determining the current research trends and pro-
previous studies, the papers [48], [49] follow a more specific viding directions for future work in this field:
approach and examine the IDPS systems devoted to the Finally, we present the ongoing trends in this field,
protection of the Advanced Metering Infrastructure (AMI). by identifying possible directions and technologies for
Finally, the work [50] evaluates three open-source Security future research work.
Information and Event Management (SIEM) systems for
SG. In particular, the platforms studied are a) the AlienVault
OSSIM [51], b) the Cyberoam iView [52] and c) the Prelude III. SMART GRID PARADIGM
SIEM [53]. According to the authors’ evaluation criteria, Many organisations such as the Electric Power Research
AlienVault OSSIM and Prelude SIEM present the best Institute (ERPI), the Department of Energy (DoE) and the
performance. European Commission Task Force for Smart Grid have been
Based on the previous description, only two studies [48], involved in the definition of the SG paradigm. The term of
[49] focus exclusively on the examination of the IDPS SG is defined as the connection of the current electrical grid
systems for SG; however they are limited only to protect- with ICT services, by ensuring the corresponding sustain-
ing the AMI domain. In the light of the aforementioned ability and allowing the remote control of all processes from
results, this work is motivated by the importance of the generation to distribution, the bidirectional communication
security issues in SG, providing a comprehensive survey of between consumers and utilities, the distributed production,
the IDPS systems which discusses critical topics such as storage and smart measurement of electricity. In this section,
the detection methodology, limitations, shortcomings and we provide an overview of the SG paradigm by analysing
the ongoing security requirements. Moreover, this survey its components and the corresponding communications.
examines not only IDPSs that monitor and control the AMI
VOLUME 4, 2016 3
WiMAX, PLC, Cellular, Satellite Zigbee, Z-Wave,

DSL, Fiber Optic Wi-Fi, PLC
Smart Meter
AMI HAN
Headend WiMAX, Wi-Fi, PLC,
Cellular, Satellite
Zigbee, Z-Wave,
Wi-Fi, PLC
Generation
Data Collector
Smart Meter
Distribution
Substation IAN
Transmission
WAN Substation
NAN Zigbee, Z-Wave,
Wi-Fi, PLC
Smart Meter
BAN
Layer 3 Layer 2 Layer 1
WAN Mediums NAN Mediums HAN / IAN / BAN Mediums
WiMAX PLC Cellular WiMAX Wi-Fi PLC Zigbee Z-Wave Wi-Fi
Fiber Cellular Satellite

Satellite DSL PLC
Optic
Industrial communication concerning electricity transfer

from the generation process to the consumption
Information communication between HAN (Smart Meter)
and NAN (Data Collector)
Information communication between IAN (Smart Meter)
Information communication between IAN (Smart Meter)
Information communication between NAN (Data Collector)
and WAN (AMI Headend)
Denotes the decisions that are taken by AMI Headend
concerning the generation, transmission and generation
processes.
FIGURE 3: The SG architecture in terms of communication.
A. SMART GRID COMPONENTS appliances. Data collectors are responsible for storing the
The SG paradigm combines various kinds of systems, information provided by multiple smart meters that belong
technologies and infrastructures such as microgrids, AMI, in a specific geographic area. Finally, the AMI headend is a
substations, synchrophasor systems, SCADA systems and central server of the utility company which receives, stores
electric vehicles [14], [54]. From these technologies, AMI and manages the information of the data collectors. Based
and SCADA systems are the most critical and vulnerable to on the information aggregated on the AMI headend, the
cyberattacks and for this reason, most of the IDPS systems utility company is able to take the right decisions concerning
analysed below focus on these technologies. Furthermore, the processes of the electricity generation, transmission and
substations and synchrophasor systems are also an attracted distribution. It is noteworthy that these components belong
target for cyberattackers, since they are crucial for the nor- to different geographic areas that can be characterised by
mal functionality of SG. In addition, a remarkable attribute different attributes and constraints. Hence, each of these
of SG is its ability to form microgrids whose operation is areas utilises appropriate communication technologies that
based on renewable energy resources. Nevertheless, such are determined according to the corresponding attributes.
microgrids infrastructures characterised by special features SCADA systems are part of the industrial environment
may exhibit different kinds of vulnerability. Subsequently, and their primary operation is to monitor and control the
we provide a brief overview of these technologies. More automated function of other components. In particular, a
information about the components of SG is provided in [54]. SCADA system consists of a) measuring instruments, b)
The AMI provides all operations that are necessitated logic controllers such as a programmable logic controller
for the bidirectional data exchange between the end users or a Remote Terminal Unit (RTU), c) a Master Termi-
and utility companies. In particular, AMI consists of three nal Unit (MTU) d) a communication network and e) an
kinds of components: a) smart meters, b) data collectors HMI. Measuring instruments refer to sensors that monitor
and c) AMI headend. Smart meters undertake to monitor the physical measurements such as the temperature, pressure
power consumption and other measurements of the electrical and voltage. Logic controllers are mainly responsible for
4 VOLUME 4, 2016
collecting data from the measuring instruments, detecting network, which includes electronic and smart devices of a
abnormal behaviours and activating or deactivating technical home. The second type, i.e., the BAN, represents a network,
components. The logic controllers interact with MTU which which comprises devices and technologies required for the
is a central host through which the system operator can send functionality of an organisation. Lastly, the IAN identifies
commands to logic controllers and receive data. The inter- a network, which incorporates all the functional elements
action between MTU and the logic controllers is realised via required for industry. As illustrated in Fig. 3, the devices
the communication network. This communication network of these networks usually utilise ZigBee and Z-wave [14],
is based on industrial protocols, such as Modbus [55]–[57] [54]. In rare cases, they also can use IEEE 802.11 (Wi-Fi)
and Distributed Network Protocol 3 (DNP3) [58]. Finally, or Power Line Communications (PLC).
HMI is a software package with graphics capabilities in- On the other hand, the second layer refers to the Neigh-
stalled on MTU and facilitates the interaction between MTU bour Area Network (NAN) which identifies a small ge-
and logic controllers. ographic area of multiple HANs, BANs and IANs. This
Substations play a significant role in the electrical grid op- network comprises data collector devices that communicate
eration. They participate in the transmission and distribution with smart meters of the previous networks and aggregate
operations of the electrical grid. Specifically, they receive the information coming from them. In this kind of net-
the generated power, configure the distribution function and work, the respective devices usually employ IEEE 802.16
control the power increase [54]. They can include various (WiMAX - Worldwide Interoperability for Microwave Ac-
devices and software components such as Intelligent Elec- cess), IEEE 802.11 (WiFi - Wireless Fidelity) standards
tronic Devices (IEDs), RTUs, HMI and Global Positioning [14], [54]. Alternatively, they can also use PLC, satellite,
System (GPS). cellular, or Digital Subscriber Line (DSL) communications.
A synchrophasor system constitutes an emerging tech- The third layer is characterised by the Wide Area Net-
nology which is necessary for the operations of the mod- works (WANs) that are responsible for connecting multiple
ern electrical grid. Mainly, it consists of Phasor Measure- NANs with many other entities such as the AMI headend,
ment Units (PMUs), Phasor Data Concentrators (PDCs), microgrids and transmission networks. This layer aggregates
a communication network and a Graphical User Interface various information from multiple entities in order to opti-
(GUI) software. A PMU is a device which executes vari- mise the generation, transmission and distribution processes.
ous measurements from current/voltage waveforms, such as The elements of the particular network can communicate
frequency, phase angle, active power and reactive power. A with each other with various communication types such as
PDC undertakes to aggregate the information of PMUs and IEEE 802.16, PLC, DSL, satellite, cellular and fibre-optic
transform them into a single flow. The communication be- communications [14], [54].
tween PMUs and PDCs is usually carried out through IEEE Finally, it should be noted that Fig. 3 presents a general
C37.118.2 and IEC 61850 [41], [42] standards. Finally, the architectural schema, from which one or more network areas
GUI application is responsible for visualising appropriately can be excluded in some cases. For example, the presence of
the various data from PDCs. NAN can be excluded in some cases where the data collector
A special characteristic of SG is its ability to form isolated is not needed. Nevertheless, the exclusion of NAN does not
microgrids that can operate either with the support of the exclude the distribution process.
main electrical grid or independently. Microgrids usually
employ renewable energy resources such as solar energy, IV. OVERVIEW OF IDPS SYSTEMS
wind energy and hydroelectric energy. At this point, it The rapid evolution of the computing systems and the global
should be noted that based on the existing literature we utilisation of Internet generate new security threats as well
could not find any IDPS system which focuses on protecting as the need for appropriate security measures such as the
microgrids. This state is a crucial research challenge in IDPS systems. According to the RFC document 2828, the
this field, since microgrids are characterised by different intrusion detection process aims at auditing and analysing
operation features compared to the main electrical grid that security events in order to identify timely potential malicious
may exhibit various kinds of vulnerabilities. activities. In 1980, the term of IDS was introduced, which
can be considered as a hardware and/or software system
B. SMART GRID COMMUNICATIONS automating the process of monitoring, auditing, analysing
Fig. 3 illustrates a generic architecture of SG divided in and identifying possible threats. Specifically, in 1980, James
terms of communication features. In the first layer, there Anderson [59] inferred that the log files of a computing
are three types of network areas: a) Home Area Networks system can be a very efficient source for monitoring its
(HANs), b) Business Area Networks (BANs) and c) Indus- state and how the individual users interact with it. Based on
try Area Networks (IANs), characterised by the presence Anderson’s technical report, researchers started to develop
of the consumer. In particular, the main characteristic of the first IDSs that suitably analysed log files for facilitating
these network areas is the presence of smart meters that the security administrators’ work. A remarkable case is
monitor the energy consumption of electronic appliances Dorothy Denning’s paper [60], in which she proposed a
and transmit them to the next layer. HAN refers to a theoretical IDS model that is based on an abstract pattern of
VOLUME 4, 2016 5
features. Based on her work, if a computing system does not B. INTRUSION DETECTION TECHNIQUES
meet the features defined, then it will have probably been The Analysis Engine utilises specific techniques to detect
affected by a kind of threat. The next subsections provide an possible threats and anomalies. Mainly, three types of intru-
overview of the IDPS systems, emphasising the architecture sion detection techniques are defined: a) Signature-based, b)
and the detection techniques. Anomaly-based and c) Specification-based. The functional-
ity of the first type (Signature-based) is based on matching
the actions that take place in a computing system with a pre-
determined set of intrusion patterns called signatures. If the
characteristics of an action match with one of the signatures,
Agent 1 then a corresponding alert is extracted. It is noteworthy that
Signatures Anomalies Specifications
this technique requires the knowledge of all vulnerabilities
of the system tested. The use of this technique yields great
reliability with a low rate of false positives, but its weak
Agent 2 Analysis Engine point lies in the inability to detect unknown attacks that are
. not specified by any signature. As a result, IDPSs utilizing
. this method must refresh regularly the set of signatures in
.
order to include new kinds of attacks. On the other side,
Agent N the functionality of the second technique (Anomaly-based)
is based on the determination of the abnormal behaviours as
intrusions. Usually, this method employs statistical analysis
FIGURE 4: IDS/IPS Architecture. processes or machine learning techniques such as Bayesian
networks, neural networks [61], [62] and Markov models to
detect malicious activities. The use of this technique is more
inaccurate in comparison with the previous one. However,
A. ARCHITECTURE OF IDPS SYSTEMS it has the advantage of recognising unknown cyberattacks.
Finally, the third technique (Specification-based) utilises a
As illustrated in Fig. 4 an IDS usually consists of three main
set of predetermined rules that define the normal behaviour
modules: a) one or more Agents, b) the Analysis Engine and
of the system tested. These rules are called specifications. If
c) the Response Module. The Agents aim at auditing and
the characteristics of an action differ with one of the speci-
collecting useful information that is preprocessed and trans-
fications, then a corresponding alert is exported. Therefore,
mitted to the Analysis Engine. Usually, this information is
this method can detect unknown attacks, since it can detect
obtained from the log files and network traffic. The number
the possible anomalies. In comparison with the signature-
of Agents is defined depending on the network topology. In
based approach, this technique is based on the assumption
this context, based on the Agent location, an IDS can be
that if all specifications are applied, the security policy of the
classified into three categories: a) Host-based IDS (HIDS),
system cannot be compromised. Conversely, the signature-
b) Network-based IDS (NIDS) and c) Distributed IDS. The
based technique does not make any such assumption. At this
first type, called HIDS monitors and records only data
point, it should be noted that the term ’hybrid’ is adopted
related to a single computing system, such as the processes
from now on for characterising an IDPS that use two or
of the operating system and system calls. NIDS focuses on
more of the above techniques.
the total network traffic, which is exchanged between the
entities of a network, by analysing attributes and patterns of
the communication protocols. Finally, the Distributed IDS TP + TN
combines the two aforementioned cases by aggregating in- ACC = (1)
formation regarding the total network traffic (case of NIDS)
TP + TN + FP + FN
as well as utilising appropriate agents, each of which can
TP
P recision = (2)
monitor a single computing system, as in the case of HIDS. TP + FP
Next, the Analysis Engine aims at analysing the collected TP
information and detecting cyberattack patterns or possible TPR = (3)
TP + FN
abnormal behaviours, utilising specific attack signatures or TN
statistical and artificial intelligence techniques. Finally, the T NR = (4)
Response Module informs the system administrator through TN + FP
alerts and warnings regarding the outcome of the Analysis FP
FPR = = 1 − T NR (5)
Engine. In some cases, the Response Module may be able FP + TN
to execute specific actions to mitigate automatically the FN
intrusions. In such a case, the system is called IPS. FNR = = 1 − TPR (6)
FN + TP
6 VOLUME 4, 2016
V. REQUIREMENTS OF IDPS SYSTEMS IN THE SMART normal behavior samples and 2% malicious behavior
GRID samples, then the training accuracy of the classification
The IDPS systems devoted to protecting SG present different model can easily reach 98%, predicting each case as
requirements compared to the IDPS of the conventional normal behavior. Conversely, if the training set consists
computer networks. Therefore this section is focused on of 60% normal behaviors samples and 40% malicious
analysing these requirements and the evaluation metrics behaviors samples, then the training accuracy may be
we adopt for evaluating and comparing the IDPS cases reduced to 60%. Therefore, in some cases, ACC can
studied in the next section. According to the previous IDPS mislead a security operator, by giving the false sense
overview, the primary purpose of an IDPS system is to of achieving high classification accuracy. Precision is
identify timely indications of possible intrusions attempts. It calculated by dividing TP with the sum of TP and
would be desirable that the results of an intrusion detection FP. Particularly, Precision expresses what proportion
process can originate from the value of a binary variable. of samples that are classified as malicious behavior, in-
However, the cyberattacks are characterised by more com- deed present a malicious behavior. Consequently, Preci-
plicated operations and the information generated by IDPSs sion provides information concerning the performance
is more complex. Consequently, we identify the following of the classification with respect to FP; nevertheless we
requirements for evaluating the performance of the IDPS consider that an intrusion detection classification in an
cases in the next section. industrial environment, such as SG should pay more
attention to FN. Accordingly, TPR is calculated by
• Detecting a wide range of intrusions: Identifying dividing TP with the sum of TP and FN. Specifically,
malicious activities that originate from external unau- this metric measures what proportion of intrusions that
thorized users or malicious insiders. It should be high- truly present a malicious behavior was categorized by
lighted that the modern IDPSs must include appropriate the classification model as an intrusion. In contrast to
mechanisms to deal with zero-day attacks. Precision, TPR provides information with respect to
• Timely intrusion detection: The term ’timely’ does FN. TNR is the fraction between TN and the sum of TN
not necessarily refer to real-time detection, as this state and FP, indicating the proportion of normal behaviors
introduces significant operational and response issues. that are predicted as normal. Actually, TNR is the
However, it is required to detect an intrusion within a opposite of TPR. In some cases, TNR is also called as
reasonable time. Thus, the detection latency should be Specificity or Selectivity. FPR or differently Fall-Out
calculated during the development and testing process is calculated by dividing FP with the sum of FP and
of a modern IDPS. TN. Actually, FPR is the opposite of TNR, identifying
• High detection performance: A number of basic the proportion of normal behaviors that are detected as
terms are explained before defining the adopted IDPS intrusions. Finally, FNR is the fraction of FN with the
performance metrics in this work. As True Positive sum of FN and TP. Respectively with the previous case,
(TP) is considered as the number of the correct clas- FNR is the opposite of TPR, indicating the proportion
sifications that detected the cyberattacks as abnormal of intrusions that are detected as normal behaviors.
behavior. On the other hand, as True Negative (TN) is Also, it is worth mentioning that many researchers
identified as the number of correct classifications that utilize Receiver Operating Characteristic (ROC) curves
recognized non-malicious activities as normal behavior. to evaluate the performance of a classifier. This curve
Accordingly, as False Positive (FP) is considered as the constitutes a graphical plot between FPR in the x-
number of incorrect classifications that identified non- axis and TPR in the y-axis. Normally, in order to
malicious activities as abnormal behavior. Finally, as define the performance of ROC curve in a numerical
False Negative (FN) is deemed as the number of incor- value, the Area Under the Curve (AUC) is calculated.
rect classifications that recognized cyberattacks as nor- This value refers to the probability of a classifier to
mal behavior. On the basis of these terms, many metrics rank a randomly selected positive event higher than a
can be calculated to evaluate the classification perfor- randomly selected negative event.
mance. Some of them that are defined by the Equations • Attentive performance of computing resources:
(1)-(6) are: Accuracy (ACC), Precision, True Positive Some entities in SG, such as the smart meters, are char-
Rate (TPR), False Positive Rate (FPR), True Negative acterized by constrained computing resources. There-
Rate (TNR) and the False Negative Rate (FNR). It fore, they may not support the computationally ex-
should be noted that TPR is also called ’detection rate’, pensive operations of the conventional IDPSs. Conse-
’recall’, ’sensitivity’ or ’probability of detection’. More quently, the memory, the computational power and the
detailed, ACC represents the ratio between the correct energy consumption should be taken into consideration
predictions and the total number of samples. ACC is during the development and testing process of an IDPS.
considered as an efficient metric when there is an equal • Scalability: SG consists of several technologies and
number of samples between the predefined classes. components that define the corresponding different
For instance, if a training set is composed of 98% communication interfaces. Therefore, an efficient IDPS
VOLUME 4, 2016 7
for SG should be scalable, having the capability to A. IDPS SYSTEMS FOR THE ENTIRE SG ECOSYSTEM
monitor and interpret these communications, by decod- As described before, SG consists of multiple and hetero-
ing and analysing the corresponding communication geneous communications that may present various secu-
protocols of SG, thus identifying possible cyberattack rity gaps and vulnerabilities, thereby making it possible
patterns. Moreover, it should be capable of aggregating to launch disastrous cyberattacks. Moreover, SG includes
and analysing logs from the various SG components. components characterised by constrained resources that hin-
• Resilient against Cyberattacks: An IDPS for SG der the adoption of conventional cybersecurity mechanisms.
should be resilient against cyberattacks, possessing the Thus, it is clear that the presence of efficient and lightweight
capability to prevent various cyberattacks, protect itself IDPS systems is necessary for the protection of SG. Sub-
and activate appropriate self-healing mechanisms in sequently, we investigate per paragraph appropriate IDPS
case of emergency. For instance, if a cyberattack cannot systems capable of protecting the entire SG ecosystem.
be hindered, an appropriate mechanism should replace In [63], the authors proposed an IDS for the entire
the violated component, thus ensuring the normal op- SG ecosystem, whose functionality is mainly based on
eration. three entities: a) an Ontology Knowledge Base (OKB),
• Friendly visual-based user interface: The informa- b) a Support Vector Machine (SVM) [64] model and c)
tion generated by IDPS (alerts and warnings) should a fuzzy risk analyzer. The system architecture consists
be presented appropriately to the SG operator or the of a number of HIDSs and NIDSs that are allocated to
security administrator. different elements of SG. In more detail, each NIDS or
HIDS includes four function modules: a) the trust manager,
b) the autonomic manager, c) the knowledge manager and d)
VI. IDPS SYSTEMS IN THE SMART GRID the fuzzy risk manager. The detection of the possible threats
It is clear that the IDPS systems devoted to protecting SG is accomplished by applying an SVM [64] model whose
differ substantially from the IDPSs focused on conventional training process lasted for 30 hours by using a dataset, which
computer networks. In particular, the multiple intercon- includes 3600 records of attacks. The specific dataset is a
nected and at the same time, independent interactions among part of OKB and includes a) records from the KDD 1999
the aforementioned SG components require a distributed dataset [65] and b) simulated experiments from the authors.
IDPS which will be able to monitor and control the network It includes multiple types of attacks, such as DoS attacks,
traffic and syslogs of all subsystems and connections. More- packet splitting attacks, command insertion attacks, payload
over, such an IDPS has to take into account the hybrid nature mutation attacks, brute force attacks, duplicate insertion
of SG which includes both industrial and ICT components. and shellcode mutation attacks. Next, in order to reduce
Specifically, it has to adapt its functionality depending on the FP alarms, the authors utilized a fuzzy logic technique
the legacy nature and constrained computing capabilities of to determine a risk value for each element of the SG
the industrial and IoT devices, such as RTUs and smart environment. These values vary from 0 to 1. Finally, OKB
meters. Finally, it has to handle and address timely a wide is employed to identify the targets of attacks. An ontology
variety of cyberattacks and possible anomalies due to the can be characterized as a dictionary which determines the
heterogeneous character of SG components. information about an application domain and the relations
In this section, we study 37 different cases of IDPSs for between them. By using the Protege software [66], the
SG. Table 1 summarises these cases cumulatively, while particular IDS is connected to the CoreSec ontology in
Table 2 compares them by presenting their most significant order to determine the most appropriate option of OKB.
characteristics. The comparison of the IDPSs examined is Concerning the evaluation of the proposed system, the
based on the target system they monitor as well as their authors argue that AUC approaches 0.99451.
In this article [67], Y. Zhang et al. suggested a distributed
detection technique and performance. The target system
IDS for the entire SG ecosystem, which is called SGDIDS
can be a) the entire SG ecosystem, b) AMI, c) SCADA
and is based on the functionality of an Artificial Immune
system, d) substation and e) synchrophasor. In particular,
System (AIS). The particular system consists of individual
subsection VI-A discusses the IDPS systems concerning
IDS modules that cooperate in a hierarchical manner. More
the entire SG ecosystem. Subsection VI-B presents those
concretely, each HAN, NAN and WAN includes a distinct
IDPSs focusing on AMI. Subsections VI-C and VI-D are
IDS which is responsible for monitoring and controlling the
devoted to the IDPSs monitoring the SCADA systems and
corresponding communications. The HAN IDS is composed
substations respectively. Finally, subsection VI-E focuses
of three units: a) data collector unit, b) AIS classification
on IDPSs regarding synchrophasors. Since each IDPS is
model and c) detection results recording unit. On the
devoted to protecting a specific category of target systems,
other hand, the NAN IDS receives the results of HAN
we can examine and compare their architecture, detection
IDSs and also utilizes the AIS algorithms. Accordingly,
technique, the kinds of cyberattacks they can detect and
the WAN IDS obtains the alerts or warnings of the NAN
finally their performance.
IDSs and utilizes the same classification algorithms. If a
lower layer IDS (e.g., HAN IDS) cannot classify some
8 VOLUME 4, 2016
network activities, then the next higher layer IDS (e.g., be implemented as an individual hardware card. Regarding
NAN IDS) will undertake to categorize these activities. Each the evaluation of the evolving machine learning algorithms,
IDS employs the CLONALG and AIRS2Parallel detection the authors utilized the KDD CUP 1999 dataset and an
algorithms. However, each type of the previous IDSs was improved version of this, called NSL-KDD [65], [68],
trained with different samples of the NSL-KDD dataset [65], [69] that include multiple types of attacks, such as, DoS,
[68], [69], since different areas networks are commonly Remote to Local (R2L) attacks, User to Root (U2R) attacks
exposed to different attacks. The training processes were and probing attacks. Also, they utilized multiple evaluation
carried out with the utilization of the WEKA [70], [71] measures such as: a) ACC, b) the size of the classifier
software package. Finally, the authors argue that ACC of in Kilobyte (KB), c) the processing time of the classifier,
the CLONALG and AIRS2Parallel algorithms reach 99.7% d) the consumption rate of the Random Access Memory
and 98.7% respectively. (RAM), e) FPR and f) FNR. The MOA software provides
In this work [72], the authors proposed new locally 16 evolving machine learning algorithms, from which seven
optimum tests and apply them in SG intrusion and fault were evaluated. These algorithms are a) Accuracy Updated
detection problems. Considering that the dynamic time Ensemble b) Active Classifier, c) Leveraging Bagging, d)
behavior of an examined system can be approached as a Limited Attribute Classifier, e) Bagging using ADWIN, f)
discrete-time linear state-space model, a failure or intrusion Bagging using Adaptive-Size Hoeffding Tree and g) Single
can be recognized by observing a change in specific system Classifier Drift. Active Classifier and Single Classifier Drift
parameters. In particular, one way to detect such changes are proposed for the IDS which controls network activities
is the utilization of hypothesis testing. For this reason, the of smart meters. Correspondingly, the authors consider that
authors develop two locally optimum tests: the Locally the Leveraging Bagging algorithm is suitable for the IDS
Optimum Unknown Direction (LOUD) and the Locally which is responsible for the data collector. Finally, the
Optimum Estimated Direction (LOED) tests. Both of them Active Classifier algorithm is suggested for the IDS of the
are appropriate for detecting small changes in the examined AMI headends.
system. However, if the change is large, the Generalized In [77] R. Vijayanand et al. presented an anomaly-based
Likelihood Ratio (GLR) test can be applied in this case. IDS which controls the AMI communications. In detail, the
Consequently, in this paper, the combination of the above proposed system is integrated into the data collector and
methods was proposed, i.e., the LOUD-GLR and the LOED- utilizes a Multi-SVM classifier [64]. A Multi-SVM [64]
GLR tests. The combined test employ LOUD or LOED, if classifier consists of multiple SVM [64] classifiers that can
the change in the system is quite small and then switches detect various types of attacks. More specifically, the authors
to GLR, if the change looks large. Finally, concerning the employed the ADFA-LD dataset [78], [79] and applied the
evaluation of the proposed method, the best TPR approaches mutual information technique to select the most important
95%. features from the particular dataset. The mutual information
technique is a filter feature selection method which is based
B. IDPS SYSTEMS FOR AMI on the entropy concept and distinguishes those features that
AMI constitutes the main novelty of SG which enables achieve the best classification ACC. The features that were
a bidirectional communication between the utility com- selected from ADFA-LD Dataset are a) Source bytes, b)
panies and energy consumers. Nevertheless, although this Destination time to leave (ttl), c) Source mean, d) Desti-
communication benefits both directions, it is based on nation mean and e) Ct_state_ttl. The possible attacks that
ICT services and components that may be characterised can be detected utilizing the aforementioned features are a)
by severe vulnerabilities. A characteristic example is the exploits, b) DoS attacks, c) fuzzers, d) backdoors, e) worms
false data injection attacks against smart meters. Hence, and f) generic attacks. Considering the training process of
the corresponding intrusion detection mechanisms should be the proposed model, for each of these attacks, an SVM [64]
adapted appropriately in order to control AMI components. classifier was developed by using a different kernel function.
The following paragraphs analyse IDPS systems suitable for In particular, the polynomial function was employed for DoS
the AMI protection. and backdoor attacks; the Gaussian function was utilized for
In this article [73], the authors presented a novel intrusion normal behaviours and generic attacks and the mlp function
detection architecture for AMI and evaluated a plethora of was used for worms, fuzzers and exploits. Concerning the
evolving machine learning algorithms by using the Massive evaluation of the proposed system, ACC exceeds 90%. TPR
Online Analysis (MOA) software [74]–[76]. In particular, and TNR are calculated at 89.2% and 93.4% respectively.
the proposed architecture consists of three different IDSs, Finally, it is worth mentioning that the training and testing
which can be installed in smart meters, data collectors processes were conducted by using the Matlab software
and AMI headends respectively. Each IDS includes four package.
components: a) the data acceptor module, b) the pre- Y. Li et al. [80] introduced an intrusion detection method
processing unit, c) the stream mining module and d) the for AMI, whose operation is mainly based on the Online
decision-making unit. It is worth mentioning that IDSs can Sequence Extreme Learning Machine (OS-ELM) [81]. OS-
either be incorporated into the AMI components or can ELM is a special feedforward neural network model which
VOLUME 4, 2016 9
utilizes the online sequence learning for its training pro- number of dropped packets. Finally, to evaluate their IDS
cess. More specifically, their methodology consists of three they used the Waikato Environment for Knowledge Analysis
phases: a) data preprocessing phase, b) initialization phase (WEKA) [70], [71] software. The authors claim that their
and c) online sequence learning phase. In the first phase, system recorded 100%T P R, 99%ACC, 66% Precision and
the training data is preprocessed by using the Gain Ratio AUC approaches 1.
Evaluation feature selection method. The second phase I. Ullah and H. Mahmoud in [87] presented an intru-
initializes randomly the parameters for the training process sion detection framework for AMI, which also applies the
of the neural network. Finally, the third method constitutes anomaly detection technique. The architecture of the pro-
the training process. The dataset that was employed for the posed system is composed of individual IDS modules that
training process can be found on the website [82]. However, are placed in different locations in HANs, NAN and WAN
it is highlighted that the specific dataset does not include correspondingly. If an IDS module detects a possible threat,
network records that identify cyberattacks nor abnormal then a related notification will be sent to the system admin-
behavior patterns. Regarding the evaluation process, mul- istrator of AMI. Also, there is a central IDS module which
tiple experiments were conducted in order to determine the aggregates and examines further the alarms generated by the
appropriate parameters for the proposed model. Moreover, various IDS modules. The authors utilized the ISCX2012
the authors evaluated their model with other classification dataset [88], [89] and the WEKA [70], [71] software in order
algorithms. They claim that their solution overtakes the other to evaluate a plethora of machine learning classification
algorithms and ACC approaches 97.239%. Accordingly, algorithms. The particular dataset includes various network
FPR and FNR are calculated at 5.897 and 3.614 respectively. attacks that are classified into four categories: DoS, LAN to
This article [83] describes an anomaly-based intrusion LAN (L2L), Secure Shell (SSH) and Botnet. They evaluated
detection method which focuses on the false data injection 20 algorithms of which the most efficient are: J48 [90], JRip,
attacks. In particular, the proposed method is based on a spa- BayesNet, SVM [64] and MLP. The most efficient algorithm
tiotemporal evaluation, which controls the correlations be- was J48 [90] which achieved 99.70% Precision and 99.60%
tween the state estimations of AMI. As state estimations are TPR.
considered various actions such as, energy supply/demand In this work [91], the authors suggested a flow-based
and electricity pricing. In more detail, the specific method distributed IDS for AMI, based on the clustering technique.
can mainly be divided into two phases. The first method The proposed system is composed of multiple IDS units that
creates a set of state estimations which is characterized by are installed on the data collectors and the AMI headend.
spatial correlations and temporal consistencies. The second Initially, the IDS units of the data collectors monitor and
method applies a voting system which classifies each state analyze the network traffic, which is exchanged between the
estimation into three categories: a) good, b) abnormal and data collectors and smart meters. Subsequently, they detect
c) unknown. Concerning the evaluation of the proposed the potential abnormal flows and send a summary report
method, two false data injection attacks were simulated. of them to the IDS unit of the AMI headend. The latter
The target of the first attack was to maximize the energy undertakes to investigate further the specific anomalies. The
transmission costs, while the second attack intended to cause detection process is based on the Mini-Batch K-Means
a power outage. The authors declare that for the first attack, algorithm and a sliding window technique. For the training
their method does not generate any FP. On the other hand, procedure of the Mini-Batch K-Means clustering algorithm,
the second attack results 0.43% FPR. the authors created their own dataset which consists of the
N. Boumkheld et al. [84] developed an IDS which ex- Transmission Control Protocol/Internet Protocol (TCP/IP)
clusively focuses on blackhole attacks. The specific kind network flows features. Also, it is worth mentioning that
of attacks constitutes a DoS attack which aims to drop all they utilized the Principal Component Analysis (PCA) tech-
network packets by advertising malicious nodes or malicious nique in order to reduce the dimensionality of the dataset.
paths. More concretely, their system controls the communi- Finally, the number of clusters (k) was specified at 4, as the
cations of an AMI NAN. To simulate the specific kind of specific value achieved the best silhouette score and FPR. In
attack, they utilized the Network Simulator 2 (NS2) [85] order to evaluate the performance of their model, the authors
simulator and examined the AMI network as an ad-hoc simulated 3 attack scenarios: a) TCP SYN Flooding DoS
network by using the Ad-Hoc On-Demand Distance Vector attacks, b) stealth port scanning attacks and c) a combination
(AODV) protocol [86]. In more detail, their simulation of the previous ones.
includes 100 smart meters nodes, 1 data collector and 2 V. Gulisano et al. [92] introduced a two-tier IDS which
malicious nodes. The IDS can be considered as a different controls the activities that take place on AMI. More con-
node that communicates only with the data collector node. cretely, their framework monitors and attempts to detect
In order to detect the possible blackhole attacks, the authors timely possible attack patterns by analyzing the network
applied the Naive Bayes Classifier which is based on the traffic features between the communications of the data
Bayes theorem. The features that were used as input in the collectors and smart meters. In order to detect timely the
Naive Bayes Classifier are a) the number of route request potential threats, the authors adopted the data streaming
packets, b) the number of route reply packets and c) the technique [93], in which the analysis of the communication
10 VOLUME 4, 2016
traffic is carried out by using acyclic directed graphs. In meter’s activities. Spying Domain consists of random stor-
more detail, their system consists of two modules called age areas that include the hash code of Secret Information.
Device Modeler and Pattern Matcher respectively. The first Through Event Log, when a cyberattacker attempt to access
module undertakes to monitor the communication traffic the storage units, an alarm is activated. Concerning the
and detect attack behaviors utilizing a Bayesian Network. evaluation procedure, the authors developed a tool which
Specifically, it monitors the number of requests from the configures appropriately the physical memory, the spying
data collectors, the hour and the ID of smart meters. On the domain and the possible storage areas that are affected by
other hand, the second module receives the corresponding the cyberattack. Evaluation figures indicate the values of
alerts and implements a secondary analysis with the support TPR according to the different parameters.
of a cybersecurity specialist. In order to evaluate their R. Mitchell and R. Chen [98] presented a specification-
system, they simulated energy exfiltration attacks, by intro- based IDS which includes individual IDSs for the AMI
ducing incorrect consumption measurements. They report headend, data collectors and smart meters. For each of the
that TPR approaches 91%. aforementioned devices, a particular set of behavior rules
In [94], the authors developed an IDS for AMI, in which have been identified and transformed into a state machine.
the communications are based on the ANSI C12.22 [95] Specifically, the IDS controlling the AMI headend has the
protocol. More specifically, the proposed system utilizes a ability to monitor the activities of the other AMI headends
specification-based model which consists of four modules and data collectors. Accordingly, the data collector IDS is
that were developed by using the Python programming able to control the behavior of the other data collectors
language. The first module is called dissector and its work and smart meters. Finally, the third kind of IDSs can only
is to capture the network traffic. The second module called monitor the other smart meters. The threat model applied
parser analyzes the network traffic by using specific patterns. by the authors, includes two kinds of attacks: reckless and
The third module applies determined specifications that random attacks. The authors argue that their methodology
define the normal behaviour of a device. Finally, the last accomplishes 100% TPR, while FPR does not exceed 0.2%
module monitors the operational state of the devices that and 6% for reckless and random attacks respectively. Also,
can be characterized by three types: a) ’in-use’, b) ’off- ROC curves are presented.
line’ and c) ’to configure’. The security specifications were In this paper [99], P.Jokar and V.Leung presented a
determined by combining a specific threat model and a specification-based IPS for the SG applications that employ
system model based on [96]. In more detail, these speci- ZigBee-based HANs. In particular, the proposed system
fications are classified into three categories: network-based, mainly focuses on the network traffic features at the Physical
device-based and application-based. In order to evaluate the (PHY) and Medium Access Control (MAC) layers. It con-
IDS, the authors utilized virtual machines as devices and sists of agents that monitor the network behavior of various
the Table TstBench software [94] to emulate the ANSI sensor nodes, while at the same time, it can be used for
C12.22 protocol. In the experimental section, they state prevention actions. Also, a central-IPS undertakes to extract
that the proposed IDS scored 100% and 99.57% TPR and and analyze particular features of the network traffic, thus
TNR respectively. However, it is noteworthy, that only two detecting possible attacks. If a potential cyberattack or an
types of attacks (meter reading attacks and service switch abnormal behavior is detected, then a specific prevention
attacks) were examined as abnormal behaviors. Finally, response will be selected by using the Q-learning method
concerning the evaluation of the computational performance, which is a reinforcement learning technique. It should be
they utilized 0.3% of the Central Processing Unit (CPU) of noted that the overall network traffic is controlled by the
the virtual machines and 10 MB of memory. central-IPS which constantly communicates with multiple
In [97], X.Liu et al. present a specification-based IDS agents. The set of the specification rules is based on 6
which has been specially designed for the smart meter’s characteristics: a) Datagram of IEEE 802.15.4 [100] and
communications. Particularly, first, they introduce a mod- Smart Energy Profile 2.0 (SEP 2.0) [101] protocols, b)
elling process which describes the information exchange traffic rate, c) Received Signal Strength (RSS), d) sequence
among the components of a smart meter based on a colored number, e) Packet Error Rate (PER) and f) node availability.
Petri net. Based on this process, they introduce a threat Regarding the evaluation of the proposed system, the authors
model which includes two classes of attacks: a) attacks on carried out a theoretical analysis of six attacks against
data and b) attacks on commands. Finally, they propose an IEEE.802.15.4, thereby demonstrating that the proposed
IDS for detecting false data injection attacks accomplished IPS can successfully address these attacks. Specifically, the
via the access of the smart meter’s physical memory. The attacks examined are: a) radio jamming attacks, b) replay
architecture of the proposed IDS consists of three elements: attacks, c) stenography attacks, d) back-off manipulation
a) Secret Information, b) Event Log and c) Spying Domain. attacks, e) DoS against data transmission during the Con-
Secret Information is a confidential data structure which is tention Free Period (CFP) and f) DoS against Guaranteed
accessible only for the legitimate procedures and also it Time Slot (GTS) requests. Subsequently, the authors con-
is utilized to encrypt the Event Log. Event Log is used ducted two experiments in order to demonstrate that their
for storing all the events that are relevant to the smart system dynamically selects the appropriate prevention activ-
VOLUME 4, 2016 11
ity. The corresponding ROC curves are presented. Finally, Nevertheless, it is worth mentioning that the authors do
the authors discussed five techniques that can bypass IDPSs. not provide numerical results regarding the effectiveness of
These techniques are: a) obfuscation, b) fragmentation, c) these rules.
protocol violation, d) generating network traffic that targets In [107], H. Li et al. focus on the DNP3 [58] protocol
on IDPS and e) DoS attacks on IDPS. They argue that providing appropriate signature rules utilizing the Snort IDS
only fragmentation techniques cannot be identified by their [104]–[106]. DNP3 is an industrial protocol, which was
proposed system. standardized by IEC TC-57 and was deployed by IEEE
In [102], the authors developed a specification-based IDS Electric Power Engineering Association (PES). According
for AMI, which combines temporal and spatial detection to the authors, the deployment process of DNP3 focused
techniques, by using Matlab. In more detailed terms, the on the reliability of communications, ignoring the informa-
proposed system focuses on blackhole and time delay at- tion security aspects. In particular, DNP3 is characterized
tacks. The blackhole attack was described previously. On by significant security deficiencies such as the lack of
the other hand, the time delay attacks aim at introducing encryption, authentication and authorization mechanisms.
additional delay time when the packets are transmitted. Therefore, it is vulnerable to a plethora of cyberattacks
In particular, their methodology monitors the number of such as reconnaissance attacks, DoS, protocol anomalies
the transmitted packets and the transmission delay time and mixed attacks. In this work, the authors developed an
between these packets by using specific numerical intervals intrusion detection template which subsequently was utilized
that were calculated by using the mean value and the for generating signature rules for the DNP3 protocol. The
standard deviation of the normal distribution. Concerning signature rules generated can detect the aforementioned
the evaluation of the proposed model, the authors compared cyberattacks. Moreover, the authors denote that the specific
their algorithm only with the spatial-based, the temporal- template can be used for developing signature rules for other
based detection technique and with the development of an industrial protocols, such as Modbus [55]–[57] and Profinet.
SVM [64] model. They report that the SVM [64] model Finally, it is noteworthy that the authors do not provide any
achieves the best TPR, but their model achieves the best evaluation process.
FPR and the second best TPR. Specifically, TPR and FPR In [108] E. Hodo et al. present an anomaly-based IDS
approach 90% and 6% respectively. for a SCADA simulated environment which utilizes the
IEC 60870-5-104 [109] (IEC-104) protocol. In 1995, the
C. IDPS SYSTEMS FOR SCADA SYSTEMS International Electromechanical Commission (IEC) was re-
The safe operation of SCADA systems is crucial for the leased IEC-60870-5-101 which includes essential telecontrol
entire functionality of critical infrastructures, such as SG. messages between a logic controller and a controlling server.
These systems enable operators to monitor, control and After six years later, IEC released IEC-104 which combines
automate the actions that take place in an industrial en- the application messages of IEC-101 with TCP/IP. However,
vironment. However, their communications are based on IEC-104 is characterized by several security issues, since
insecure protocols, such as Modbus [55]–[57] and DNP3 its functionality is based on TCP/IP which itself presents
[58] that do not integrate authentication and access control various vulnerabilities. Moreover, the application data are
mechanisms, thus enabling MiTM attacks. Hence, the IDPS exchanged without any authentication mechanism, i.e., as
systems that are responsible for protecting SG, should plaintext. The authors create their own dataset which in-
necessarily take into account the security weaknesses of cludes passive Address Resolution Protocol (ARP) poison-
SCADA communications. Below we analyse per paragraph ing attacks, DoS attacks and replay attacks that replace
appropriate IDPS systems devoted to protecting SCADA legitimate packets with malicious ones. Based on this dataset
systems. and utilizing WEKA [70], [71], they evaluated multiple
In [103], T.H. Morris et al. focus their attention on the machine learning algorithms, such as Naive Bayes IBk,
Modbus [55]–[57] protocol, providing a set of signature J48 [90], Random Forest [110], OneR, RandomTree and
rules. Modbus is a master-slave, industrial protocol, which DecisionTable. J48 [90] and DecisionTable scored the best
was released by Gould Modicon (now Schneider Electric) ACC.
in 1979 for the communication between MTU (master) and In [111] N. Goldenberg and A. Wool present an anomaly-
logic controllers (slave). MTU sends a specific query to the based IDS which is devoted to the Modbus/TCP [55]–
logic controller and subsequently the second transmits its [57] communications. More detailed, the functionality of
response to MTU. More specifically, the authors introduce the specific IDS is based on a Moore Deterministic Finite
50 signature rules that concern the Modbus/TCP as well as Automaton (DFA) which in turn is based on the high
the Modbus protocol over a serial communication interface. periodicity of the Modbus [55]–[57] network traffic. In
The Snort [104]–[106] IDS was utilized for testing these particular, the proposed DFA monitors the queries and
rules; however, the paper describes these rules in a generic responses between MTU and each logic controller, thereby
format, in order to be applied by various IDS systems. Each identifying the normal and abnormal states. More detailed,
rule is defined in a specific text field and is accompanied the DFA consists of: a) a set of states, b) an alphabet which
with specific details that concern the protocol specifications. is a set of input symbols, c) a transition function and d) the
12 VOLUME 4, 2016
first state. A state denotes how normal the Modbus [55]–[57] (SAPs). After this process, their system is capable of clas-
network traffic is and can take four values: a) Normal, b) sifying new requests as existing SAP or unexpected SAP.
Retransmission, c) Miss and d) Unknown. From the afore- Finally, the authors developed a visualisation method which
mentioned values, only the Unknown state is considered as visualises the flow graphs of the represented SAPs. Con-
a malicious behaviour. On the contrary, the Retransmission cerning the software packages utilised by the authors, they
and Miss values denote a benign behaviour with some are Conpot [123], Python 2.7 and MongoDB [125], [126].
anomalies. The input symbols and the transition function Based on the evaluation results the proposed system can
determine the states for each communication. The input detect reconnaissance and DoS attacks with TPR 90% and
symbols are divided into two classes: a) known symbols 95.12% respectively. FPR of both aforementioned attacks is
and b) unknown symbols. The first category includes those calculated at 0%.
symbols that were observed during the learning phase and In [127], Y. Yang et al. provide a specification-based
result in a known state (Normal, Retransmission, Miss), IDS for the IEC-104 [109] protocol. The core of their
while the second category implies those symbols that result system is named Detection State Machine (DSM) and its
in the Unknown state. To evaluate their methodology, the functionality is based on the Finite State Machines (FSM)
authors generated two real datasets using Wireshark [112]– methodology. More detailed, the operation of IEC-104 [109]
[114], Pcapy [115] and Impacket [116]. Based on the is determined through the correlations of FSM. In contrast
experimental results, the authors argue that their model did to the traditional FSM-based systems, their implementation
not present any false alarm. applies a set of alarms that are capable of distinguishing
In [117], S.D. Anton et al. provide a comparison of four the protocol malfunctions. To deploy and demonstrate their
machine learning algorithms concerning the detection of methodology, the authors employ the Internet Traffic and
anomalies in a Modbus/TCP dataset. More specifically, the Content Analysis (ITACA) software [128]. Concerning, the
authors utilised the dataset of Lemay and Fernandez [118] evaluation results, the authors argue that the True Positive
which was divided into three sub-datasets, namely DS1, Rate (TPR) and False Positive Rate (FPR) of their IDS are
DS2 and DS3. DS1 consists of 3319 packets and contains calculated at 100% and 0% respectively.
the network traffic between MTU and 6 RTUs, including In [129], Y. Yang et al. provide signature and specification
75 malicious cases. Similarly, with the same architecture of rules for the IEC-104 [109] protocol, by using the Snort
one MTU and 6 RTUs, DS2 contains 11166 packets from IDS [104]–[106]. After studying the security issues of the
which 10 cases are malicious. Finally, DS3 includes 365906 specific protocol, the authors deployed attack signatures and
packets with 2016 malicious cases and was generated by specification rules for the following attacks: a) unautho-
the combination of eight datasets. From these sub-datasets, rized read commands, b) unauthorized reset commands, c)
specific features were extracted and used for the training unauthorized remote control and adjustment commands, d)
of the machine learning algorithms. It is noteworthy that spontaneous packets storm, e) unauthorized interrogation
the extracted features concern only the TCP/IP stack. The commands, f) buffer overflows, g) unauthorized broadcast
algorithms evaluated are: a) SVM [64], Random Forest requests and h) IEC-104 port communication. Concerning
[110], K-Nearest Neighbour (KNN) [119] and k-means the evaluation process, 364 packets were examined from
[120]. ACC of SVM [64] with DS1, DS2 and DS3 is equal which 41 packets were malicious. Based on the experimental
to 100%, 100% and 99.99% respectively. Accordingly, ACC results, all malicious packets were detected with zero FPs.
of Random Forest [110] with DS1, DS2 and DS3 is 100%, In [130], Z.Feng et al. focus their attention on the security
99.99% and 99.99%. ACC of KNN with DS1, DS2 and DS3 of the Profinet [131], [132] protocol by deploying effective
is 99.7%, 99.9% and 99.9%. Finally, ACC of k-means [120] signature and specification rules utilizing Snort [104]–[106].
with DS1, DS2 and DS3 is 98.1%, 55.62% and 63.36%. Profinet is an industrial standard which was standardized by
In [121], P.H. Wang et al. implement an anomaly-based IEC 61158 and IEC 61784 and was developed by Profibus
IDS utilising a clustering technique as well as data captured & Profinet International. According to the authors, Profinet
by a honeypot system. A honeypot [122] is a specific suffers from severe security issues, since it does not integrate
device or software which intentionally possesses specific encryption, authentication and authorization mechanisms,
vulnerabilities in order to attract the cyberattackers. More thus making possible the accomplishment of MiTM attacks.
detailed, the proposed IDS focuses on detecting intrusions In this paper, the authors enhance the potential of Snort
against the Modbus [55]–[57] protocol, by gathering and [104]–[106] by decoding the Profinet attributes as well as
using the information provided by a Conpot [123] honey- deploying appropriate signatures for detecting MiTM, DoS
pot. Conpot [123] is a software package which represents and reconnaissance attacks. Moreover, the authors deployed
a Siemens programmable logic controller simulating the specification rules for identifying possible anomalies. To
Modbus protocol. During their experiments, the authors evaluate their work, the authors utilize the traffic package of
considered that each request to Conpot was a cyberattack. [133] and also they create a DoS attack scenario based on
Subsequently, they combined a similarity evaluation method [134]. According to the evaluation process, the proposed sig-
of the requests with an agglomerative hierarchical clustering nature and specification rules can detect intrusions against
[124] to extract representative Sequential Attack Patterns Profinet.
VOLUME 4, 2016 13
In [135], S. C. LI et al. implement an anomaly-based IDS applies content and state inspection rules in order to de-
for the Modbus protocol, adopting classification data mining tect particular attack patterns. The content inspection rules
models. In particular, they developed a J48 decision tree as examine particular conditions for each application layer
well as three neural networks, utilising WEKA. To train the packet, while the state inspection rules check the existence
above models, they create a dataset by constructing a real of specific flags that should characterize the protected de-
testbed consisting of a programmable logic controller, MTU, vices. Lastly, the state manager updates the states of the
a cyberattacker unit and a cyberdefender unit. This dataset protected devices. In order to evaluate their framework, the
includes a) reconnaissance attacks, b) response injection authors applied their stateful analysis plugin in a scenario
attacks, c) command injection attacks and d) DoS attacks. which utilizes the Manufacturing Message Specifications
To create their dataset, the authors utilised Wireshark [112]– (MMS) [141] protocol based on the directions of IEC 61850
[114] as well as a PHP script to convert the Packet Descrip- [41], [42] standard. They described two attack examples that
tion Markup Language (PDML) format of Wireshark [112]– are detected successfully, but they do not provide numerical
[114] to Comma-Separated Values (CSV) format. Since their results.
dataset includes very few malicious records, the authors This work [142] analyzes a specification-based IDS which
utilised the zeroR [136] classifier. Specifically, 92.5% of is deployed in a substation in South Korea. More specifi-
the dataset includes normal records. Hence, based on zeroR cally, their IDS is based on the analysis of Generic Object
[136], ACC of the data mining models generated by the Oriented Substation Events (GOOSE) [143] and MMS [141]
authors has to overcome 92.5%. The training process em- protocols, examining general network traffic characteristics,
ployed 39 features, but they are not specified by the paper. such as the number of bits per second (bps), the number
Based on the evaluation results, ACC of j48 is calculated at of packets per second (pps) and the number of connections
99.8361%. Accordingly, ACC of the first, second and third per second (cps). For the mentioned characteristics, specific
neural network is calculated at 97.4185%, 97.4603% and intrusion detection algorithms were created utilizing statisti-
97.3876%. cal analysis techniques. Details about the architecture of the
IDS are not provided. Regarding the evaluation procedure,
D. IDPS SYSTEMS FOR SUBSTATIONS a real dataset was utilized consisting of multiple network
A substation is a critical location of the electrical grid, attacks, such as: port scanning attacks, DoS attacks, GOOSE
where the electrical energy can be transformed, split and attacks, MMS attacks, Simple Network Management Proto-
combined. Usually, the operations of contemporary substa- col (SNMP) attacks, Network Time Protocol (NTP) attacks
tions are automated and controlled by a Substation Auto- and ARP attacks. The authors argue that their model scored
mated System (SAS) which incorporates many industrial 100% Precision, 0% FPR, 1.1% FNR and 98.9% TPR.
and ICT components such as IEDs, RTUs and computers. In [144], Y. Yang et al. provide a specification based IDPS
The communication among these components is based on devoted to protecting substations utilising the IEC 61850
the IEC 61850 [41], [42] standard which determines the [41], [42] protocol and particularly the communications
following goals: 1) interoperability, 2) long term stability based on MMS, GOOSE and Sampled Measure Value
and 3) simplified configuration. However, it should be noted (SMV). More concretely, the proposed IDPS consists of five
that IEC 61850 does not identify any cybersecurity feature modules: a) configuration module, b) network traffic capture
for the safe and normal functionality of SAS. Consequently, module, c) process core module, d) rule module and e) result
possible cyberattacks can exploit the security gaps of the module. The first one is responsible for examining the at-
protocols defined by this standard, thus making it possible tributes of a specific substation, thus determining them with
to generate disastrous consequences. Although IEC 62351 specific values and limits. The second undertakes to capture
[137] defines primary security measures, such authentica- and isolate the network traffic of MMS, GOOSE and SMV.
tion mechanisms to secure the protocols defined by IEC The process core module adopts the ITACA software in
61850, many vendors and manufacturers do not adopt these order to analyse in detail the attributes of the aforementioned
solutions. Therefore, in any case, IDPS is considered as protocols. The rule module applies the specification rules to
a necessary tool for the protection of SAS. Each of the the preprocessed IEC 61850 network traffic. Finally, the last
following paragraphs describes an IDPS instance, devoted module informs the security administrator regarding poten-
to protecting substations. tial violations. Concerning the specification rules, they can
B. Kang et al. in [138] introduced an IDS framework for be classified into four categories: a) access-control detection,
substations, which employs signatures and focuses on the b) protocol whitelisting detection, c) model-based detection
active power limitation attacks. In particular, they developed and d) multi-parameter detection. The first one specifies the
a stateful analysis plugin which can be incorporated into legitimate MAC and IP addresses as well as TCP ports,
the Suricata IDPS [105], [139], [140]. The specific plugin thereby forming a whitelist. The rules of the second category
includes three functions: a) the application layer protocol detect as malicious those packets that are not related to IEC
decoder, b) the rule match engine and c) the state manager. 61850. The next category is devoted to identifying each
The first function decodes the application layer packets and specification rule relevant to the attributes of the previous
extracts their corresponding attributes. The second function protocols. The last category includes some rules related
14 VOLUME 4, 2016
to the physical characteristics of a substation. It is worth traffic information such as pps and bps. The second process
mentioning, that all rules provided by the authors are not is employed only before the training of the classification
identified accurately. Regarding the evaluation process, data model. It is responsible for removing the outlier values of
from a real substation in China was utilised. According the training set, since such values may denote an anomalous
to the authors, the proposed IDS is capable of detecting situation. For this process, the Expectation Maximization
a plethora of cyberattacks, such as DoS, MiTM and packet (EM) [147] and Local Outlier Factor (LoF) [148] were
injection attacks. However, it should be noted that numerical utilised through the WEKA software. It should be noted
results are not provided. that in an industrial environment, an anomaly may occur
In [145], M. Kabir-Querrec et al. introduce a even if each component operates normally. Finally, the last
specification-based IDPS which focuses on IEC 61850 processes focus on training and testing the one-class SVM
[41], [42] communications of a substation. In particular, classification model respectively. The training process was
the architecture of their IDPS is based on the data object implemented by using data from a real substation. Regarding
model defined by IEC 61850, by introducing a new intrusion the evaluation process, FPR ranges between 1% and 6%.
detection function. This data object model consists of many U. Premaratne et al. in [149] introduce a hybrid signature-
Logical Nodes (LNs) that satisfy specific functions. All LNs based IDPS for a substation utilising the IEC 61850 protocol
required for a function form a new logical entity called Log- [41], [42]. The proposed IDPS combines signature and
ical Device (LD). A physical device, such as IED can consist specification rules regarding DoS attacks, traffic analysis
of many LDs. LNs can exchange data among themselves attacks, and password cracking attempts. In particular, the
using a concept named Piece of Information for COM- authors simulated these cyberattacks, thereby extracting the
munication (PICOM). Although IEC 61850 incorporates corresponding signature and specification rules that in turn
a function for security processes named Generic Security were incorporated into Snort [104]–[106]. To simulate these
Application (GSAL), the author deployed a new one which attacks, they employed the ping command, THC Hydra
is devoted to detecting possible anomalies, by determining [150] and Seringe [151]. Nevertheless, although the authors
the normal specifications of the standard. To define a new argue that their IDPS is devoted to monitoring IEC 61850
function inside IEC 61850, the following steps have to be packets, it is not able to identify cyberattacks against IEC
accomplished: a) a formal description of the function is 61850 protocols, such as GOOSE and MMS. Moreover,
needed, b) the function has to be decomposed into LNs the authors do not provide numerical results, regarding the
and c) the interaction with the other functions has to be efficiency of their system.
determined. Hence, the authors created an LN called CYSN J. Hong et al. in [152] provide a specification-based
which is responsible for sniffing the GOOSE messages IDPS which is also devoted to protecting IEC 61850 [41],
and transmitting them to two dedicated LNs that in turn [42] substations, by analysing multicast GOOSE and SMV
are devoted to checking the specifications, thus generating messages. After providing a brief description concerning
the respective alert in case of a security violation. More the format of GOOSE and SMV protocols, the authors
detailed, the first one called CYComChkSingle undertakes describe in detail two specification rules that are used to
to verify the structure and parameters of each message. Ac- detect possible GOOSE and SMV cyberattacks respectively.
cordingly, the second one named CYComChkMany verifies In particular, concerning the GOOSE cyberattacks, their
the consistency of the messages based on a specific time IDPS can detect relevant replay attacks, DoS attacks, attacks
slot. However, it is worth mentioning that the authors do generating malicious GOOSE data, malicious activities that
not provide detailed information concerning the content and change GOOSE control data and finally, actions that modify
format of these specifications. In addition, the paper does the time information. Accordingly, concerning the SMV
not include any evaluation procedure. attacks, the proposed IDPS can detect relevant DoS attacks
H. Yoo and T. Shon in [146] provide an anomaly-based and malicious actions that modify or generate SMV data.
IDPS for the substations utilising the IEC 61850 stan- Regarding the architecture of the proposed IDPS, it consists
dard. In particular, the proposed IDPS focus on MMS and of four modules: a) packet filtering module, b) packet parser
GOOSE protocols, by adopting a one-class SVM classifi- module, c) specification-based IDS module and d) HMI
cation model, thus identifying patterns that correspond only module. More detailed, the first module is responsible for
to the normal and legitimate network traffic. More detailed, capturing only GOOSE and SMV packets. Accordingly, the
their IDPS consists of four processes: a) data capturing second one undertakes to extract from the GOOSE and SMV
and preprocessing, b) outlier processing, c) one-class SVM packets the corresponding attributes. The specification-based
training and d) anomaly detection. The first process is IDS module applies the specification rules and the last mod-
devoted to capturing and preprocessing MSS and GOOSE ule informs the system operator about possible cyberattacks
packets, thus providing three sets of data. The first set and anomalies. The authors tested the effectiveness of their
comprises the attributes of each MMS and GOOSE packet. implementation under real conditions, by constructing a
These attributes are described in detail in the paper. The CPS testbed, which in turn enables the execution of the
second set includes the network flows formed by MMS and various cyberattacks. Based on the authors, FPR can reach
GOOSE communications and finally, the third one includes 1.61 × 10−4 .
VOLUME 4, 2016 15
In [153] Yi. Yang et al. have developed a specification- simulator, four relays, four PMUs, a PDC, an energy man-
based IDPS capable of identifying cyberattacks against IEC agement system, which runs the OpenPDC [156], [157] soft-
61850 [41], [42] substations. Regarding the architecture ware and a personal computer which executes Snort [104]–
of the suggested IDPS, it is composed of the follow- [106]. The input data are captured by the mentioned entities
ing modules: a) configuration module, b) network traffic and are compared with common paths. A common path
capturing module, c) IDPS process core, d) rule module is a sequence of system states that may be a specification
and e) result module. The first module determines the of normal behavior or a signature of a cyberattack. Based
configuration files that are used to specify the specification on these characteristics, the particular IDS can classify an
rules. The second module undertakes to sniff IEC 61850 activity as: a) system disturbance, b) normal operation and
packets. The following module analyses the IEC 61850 c) cyberattack. The training process of the common-path
packets, by extracting their attributes. The fourth module mining algorithm includes the creation of a dataset which
is responsible for matching the IEC 61850 packets with a comprises 25 scenarios of 10000 simulation instances. These
predefined set of specification rules. Finally, the last module scenarios are classified into three categories, namely a)
informs the system operator or the security administrator singe-line-to-ground faults, b) normal operations and c)
about the possible intrusions. Concerning the specification cyberattacks. According to the evaluation results, ACC is
rules adopted by this IDPS, they can be classified into four calculated at 90.4%.
categories: a) access control detection rules, b) protocol- R.Khan et al. [158] introduced a hybrid IDS which is
based detection rules, c) anomaly behaviour detection rules mainly based on specification-based and signature-based
and d) multi-parameter detection rules. The first kind of techniques for synchrophasor systems that utilize the IEEE
rules is responsible for allowing only the network traffic C37.118 protocol [154]. In more detail, the general archi-
coming from legitimate MAC and IP addresses. Accord- tecture of the proposed system consists of separate HIDSs
ingly, the rules of the second category undertake to allow and NIDSs called agents and sensors respectively. The
only the network traffic specified by the protocols that agents monitor the operation of PMUs or PDCs, while the
are defined by the IEC 6185 standard. The next rules sensors govern the overall network traffic. Also, there is
identify normal behaviours related to the attributes of the a management server, which aggregates and correlates all
protocols incorporated into IEC 61850. Finally, the last information coming from the individual agents or sensors. In
category identifies some specifications concerning specific addition, a database server is responsible for recording any
attributes of the physical environment. It should be noted detection alert or warning. The agents and sensors comprise
that the authors do not provide numerical results regarding six components: a) PCAP filters, b) IEEE C37.118 decoder,
the performance of their implementation. c) analyzer/detector, d) state manager, e) events manager
and f) console. The PCAP filters are developed by using
E. IDPS SYSTEMS FOR SYNCHROPHASORS the C/C++ programming language and are responsible for
The modern electrical grids usually are equipped with capturing the IEEE C37.118 packets. The IEEE C37.118
synchrophasor systems capable of providing real-time in- decoder analyzes the previous sniffing packets and extracts
formation concerning electricity measurements, such as cur- the appropriate information. The analyzer/detector utilizes
rent, voltage and frequency. These systems complement the a set of rules in order to detect abnormal behaviors. This
traditional SCADA systems, by offering additional wide set is composed of four categories rules: a) signature-based
monitoring of the entire electrical grid. Thus the system rules, b) range-based rules, c) threshold-based rules and
operator can identify possible functional problems more d) stateful behavior-based rules. According to the authors,
quickly, make better decisions and prevent devastating sit- the specific set of rules is able to detect a plethora of
uations. Although their role is passive, a successful cyber- cyberattacks, such as, ARP poisoning attacks, replay at-
attack against such systems can lead to revealing signifi- tacks, port scanning attacks, DoS attacks, GPS spoofing
cant information related to the operation of the electrical attacks, command injection attacks and physical attacks.
grid. In particular, synchrophasors usually employ the IEEE Subsequently, the analyzer/detector communicates with the
C37.118 protocol [154], which does not integrate any au- state manager, which stores possible alerts or warnings in
thentication mechanisms, thus making it possible to launch the database server. Next, the event manager communicates
MiTM cyberattacks. Therefore, it is clear that the detection with the management server, whose operation was discussed
and prevention of cyberattacks against synchrophasors are previously. Finally, the console is a command line or a
crucial. Each of the following paragraphs analyses an IDPS GUI environment with which the user can configure the
devoted to protecting such systems. operations of the previous components, e.g., the detection
S.Pan et al. [155] proposed a hybrid IDS for the syn- rules. For the evaluation process, they employ the NRL Core
chrophasor systems, which combines anomaly-based and software [159], [160]. However, it is worth mentioning that
signature-based techniques. In particular, their work is based numerical results are not provided.
on the common-path mining approach and Snort [104]– Y. Yang et al. in [161] suggest a specification-based IDPS
[106]. They examined an architecture of three bus two line capable of protecting synchrophasor systems utilising the
transmission system, which consists of a real-time digital IEEE C37.118 protocol. More specifically, their IDPS con-
16 VOLUME 4, 2016
sists of three kinds of rules including: a) access control rules,

b) protocol-based rules and c) behaviour-based rules. The
access control rules define a whitelist with the legitimate
source and destination MAC and IP addresses as well as
the corresponding ports at the transport layer based on the TABLE 1: A summarized presentation of the most impor-
Open Systems Interconnection (OSI) model. Accordingly, tant features found in compiling IDPSs for the smart grid
the protocol-based rules adopt also a whitelist which in paradigm.
turn defines the application layer protocols allowed for the Important Features Values
interaction among the synchrophasor components. In this Number of IDPSs that focused on 3
case, this list will enable only the IEEE C37.118 traffic. the entire SG ecosystem
Number of IDPSs that focused on 13
Finally, the last category identifies behaviour rules based AMI
on the attributes of the IEEE C37.118 packets, by utilising Number of IDPSs that focused on 10
a deep packet inspection process. All rules are described SCADA
Number of IDPSs that focused on 8
sufficiently in the paper. Concerning the evaluation process, Substations
the authors tested their IDPS in a real testbed, by executing Number of IDPSs that focused on 3
reconnaissance, MiTM and DoS cyberattacks. According Synchrophasors
Number of IDPSs that used 3
to the experimental results, FPR of the proposed IDPS is signature-based technique
calculated at 0%. Number of IDPSs that used 17
anomaly-based technique
Number of IDPSs that used 12
specification-based technique
Hybrid IDPSs 5
VII. DISCUSSION Visual-based IDPSs 1
SG consists of a complicated and heterogeneous set of IDPSs that take account resource 2
consumption
technologies, including AMI, SCADA systems, substations, IDPSs that calculate detection la- 0
synchrophasors electric vehicles, etc. These technologies op- tency
timize the existing processes of the traditional electrical grid, IDPSs comprising self-hiling capa- 0
but also generate multiple hazards, such as cyberattacks that bilities
1. KDD CUP 1999 [65]
can cause disastrous consequences, such as a power outage. 2. NSL-KDD [65], [68], [69]
In particular, most of the cyberattacks usually target SCADA Utilized Public Datasets
3. ADFA-LD [78], [79]
systems because they utilize insecure, legacy communica- 4. CER Smart Metering Project
[82]
tion interfaces and protocols. Characteristics examples are 5. ISCX2012 [88], [89]
the Stuxnet worm [26] and the Russian cyberattack against 1. Suricata [105], [139], [140]
a Ukrainian substation, resulting in the power outage for 2. MOA [74]–[76]
more than 225,000 people [19]. Moreover, in 2009 Chinese 3. Protege [66]
4. Matlab
and Russian cyberattackers attempted to penetrate the US 5. NS2 [85]
electrical grid, by carrying out reconnaissance cyberattacks 6. WEKA [70], [71]
[169]. Furthermore, in 2014, a campaign of cyberattacks, 7. Table TstBench [94]
8. VirtualBox [162]
named Dragonfly [170] was implemented against electrical 9. Python
energy infrastructures of many countries, including the US, 10. Wireshark [112]–[114]
Germany, France, Italy, Spain, Poland and Turkey. The 11. Snort [104]–[106]
12. OpenPDC [156], [157]
Repository of Industrial Security Incidents (RISI) [171] 13. NRL core [159], [160]
comprises 242 reported SCADA cybersecurity incidents Utilized Software Packages 14. OpenPMU [163]
dating from 1982 to 2014. It is clear that the IDPS systems 15. C/C++
16. ITACA [128]
are an efficient and necessary measure for the protection of 17. Pcapy [115]
SG, by timely detecting or even preventing the cybersecurity 18. Impacket [116]
issues. In this work, we present a comprehensive compila- 19. Conpot [123]
20. MongoDB [125], [126]
tion of 37 IDPS systems, designed for the protection of SG, 21. THC Hydra [150]
including IDPSs that protect the entire SG ecosystem, AMI, 22. Seringe [151]
SCADA systems, substations and synchropahsors. 23. Colasoft Packet Builder
[164]
Table 1 summarizes the results of our analysis by high- 24. Nmap [165]
lighting the most important features found. In particular, 25. Metasploit [166], [167]
26. hping [168]
3 IDPSs focus on the entire SG ecosystem, 13 on AMI,
10 on SCADA systems, 8 on substations and 3 on syn-
chrophasors. The majority of IDPSs employ the anomaly
detection technique or particular specifications that define
the normal behaviour. Concretely, 17 IDPSs employ the
VOLUME 4, 2016 17
anomaly detection technique, 12 models are characterized storing and interpreting processes. Finally, the projection
as specification-based, 3 IDPS employ attacks signatures level includes predictive and prescriptive algorithms that in-
and 5 cases combine the aforementioned detection methods. tend to interpret relevant events. McGuinness and Foy [173]
Each of these techniques is characterized by advantages and introduced an additional layer, named Resolution aiming to
disadvantages. The signature-based IDPS usually achieves identify the appropriate practices that optimize a specific
high performance; however, it is characterized by the inabil- situation. Therefore, based on the previous definitions, we
ity to detect unknown threats. Also, generating cyberattack consider that an appropriate IDPS for SG should apply a
signatures is a very time-consuming process. On the other hybrid methodology, including signature and specification
hand, the anomaly-based technique is able to detect zero- rules as well as anomaly detection processes. Moreover,
day attacks but presents high FPR. Finally, the specification- it should be capable of monitoring and interpreting a set
based IDPS combines the advantages of the previous ones; of various SG communication protocols from the physical
however, in an environment, such as SG which includes layer to the application layer on the basis of the OSI model,
multiple alterations and modifications, these specification thereby having the capability to detect cyberattack patterns
rules must be redefined continuously. Therefore, the solution in a cross-layer approach. Furthermore, it should analyse
of developing hybrid IDPSs sounds more promising, since logs from the various components, systems and software
the combination of the detection techniques can meet the applications, thus being capable of detecting attacks at the
aforementioned issues. application level. Finally, it should include appropriate self-
In addition, it is noteworthy that none of the examined healing mechanisms that will enable the normal operation
IDPSs include information about the detection latency, while of the entire system, in case of a disastrous cyberattack.
only two cases [73], [94] comprise information about the
consumption of the computing resources. However, the de- VIII. RESEARCH TRENDS AND DIRECTIONS
tection latency is a significant evaluation measure, especially It is clear that IDPS systems are critical for any security
in critical systems such as SG, since various cyberattacks system that is deployed in SG. Their role lies in further
can cause disastrous consequences. Also, the consumption detecting whether an attacker has compromised grid systems
of the computing resources must be taken into account, and gained access to power grid networks. They should be
given the establishment of the IoT era, which is charac- capable of identifying threats and attacks in the whole SG,
terized by constrained resource capabilities. Moreover, all by having global visibility, while being able to access both
IDPS cases studied are not quite scalable, since they cannot power and information systems such as MTU, RTU, PLC,
monitor and interpret data from multiple sources such as the PMU, smart meters and data concentrators. Moreover, they
various communication protocols utilised in SG as well as should be scalable, by combining various intrusion detection
the logs of the various components like electricity measure- techniques and monitoring different types of communication
ments of HMI and smart meters. Furthermore, none of the and data such as network traffic, software and system logs as
IDPSs examined does not include self-healing capabilities, well as raw data like electricity measurements. Thus, they
providing appropriate mechanisms in case of emergency. As should be capable of identifying the type of cyberattacks
mentioned in Section V, in critical infrastructures, such as and activating the appropriate preventive mechanisms re-
SG, recovery mechanisms, should be activated immediately spectively, such as for example the interruption of a network
in emergency situations, in order to replace the violated flow if it is considered as a DoS attack. Furthermore, IDPSs
components, thus restoring the normal operation of the for SG should be resilient against those cyberattacks that
system. Finally, it is worth mentioning that although SG aim at bypassing it, by using techniques like for example
encompasses many complex domains and a huge number obfuscation, packet fragmentation, code packing and en-
of heterogeneous components (e.g., smart devices), only one cryption, code mutation, and DoS attacks [99]. Finally, they
IDPS includes visual-based mechanisms for facilitating the should provide appropriate self-healing mechanisms that
detection process. will be activated during emergency situations, by isolating
Undoubtedly, the IDPS cases examined before provide critical parts of SG or enabling collaborative and redundant
an additional layer for the protection of SG as well as mechanisms that in turn will provide sufficient solutions,
a valuable effort in this research field. However, none until the normal operation is restored. In this section, we
of them satisfy all requirements defined by Section V. aim at determining the research trends in this field, also
In general, we consider that the security mechanisms in providing specific directions for future work.
this domain have to take into account both the physical Based on the analysis of Section VII, we have seen
and cyber features of the various components, by adopting that the existing IDPS are generally unable to interpret the
situational awareness processes in a cross-layer approach. application layer data for the SG communications, either
Based on Endsley [172], situational awareness consists of for a single packet, or at a session layer, where the state
three layers. The first layer is the perception of informa- of a connection should be monitored for inconsistencies
tion, which identifies the elements of an environment and [174]. As a result, most commercial IDPSs do not employ
their behaviour. The second layer is the comprehension of specifications rules, determining the normal attributes of
information received from the previous layer, comprising SCADA and ICS protocols (e.g., Modbus, IEC 61850 [41],
18 VOLUME 4, 2016
[42], IEC-104 [109]). Furthermore, traditional approaches the situational awareness. Hence, we think that a possible
cannot be adopted to discriminate between cyberattacks research field in this domain is the development of a SIEM
and accidental faults [175]. The Software Defined Network tool which will solve the aforementioned limitations, by
(SDN) technology can offer significant solutions regarding applying appropriate IDPS agents. More specifically, this
the previous limitations. The SDN technology provides tool should be able to decode, analyse and correlate various
global visibility and virtualization capabilities, thus making security events paying attention to the attributes of industrial
possible the generation of specification rules. More specif- protocols, such as IEC 61850 [41], [42], DNP3 [58] and
ically, SDN enables the slice of the physical communi- Modbus [55]–[57]. The distributed agents should be able to
cation network into several virtualized networks devices monitor and control each device of SG, by implementing a
and deliver traffic belonging to each critical grid control deep packet inspection process in analysing each attribute of
application. The virtualized network slices a) inherently the corresponding protocols from the physical to the appli-
enhance security with traffic isolation, b) enable more fine- cation layer and based on specific threshold values should
grained status monitoring and c) simplify the labor-intensive have the ability to identify possible anomalous behaviours.
protocol vulnerability assessment, i.e., limited to one partic- Finally, based on the analysis of Section VII, we have
ular application per virtual network slice [176]. Therefore, seen that, the IDPS systems should prevent cyberattacks
by taking full advantage of the SDN technology, we consider timely, by applying effective countermeasures, such as self-
that the research efforts should focus on developing SDN- healing mechanisms. In contrast to the traditional electrical
based IDPS systems that will also be capable of monitoring grid, SG has the ability to incorporate self-healing mech-
microgrids. However, based on the existing literature at this anisms in order to protect itself from natural disasters or
time, we could not find any IDPS devoted to protecting cyberattacks. In this field, self-healing entails the division
microgrids. of the main utility grid into individual microgrids, that can
In the light of the aforementioned remarks, the inter- collaborate with each other in the case of emergency. Based
connected and interdependent nature of SG creates new on recent studies [34], [176], [179], [180], the collaboration
challenges for the SG security, such as coordinated attacks, among individual, independent microgrids, called islands,
APTs, DoS attacks and botnets. In particular, coordinated can enhance the functionality of the entire utility grid, by
attacks and APTs represent a more dangerous category increasing its resilience and reliability. In particular, based
because they are sophisticated human-driven attacks against on the type of emergency, the self-healing mechanism is
specific targets. They are usually perpetrated over long responsible for interconnecting or isolating the correspond-
periods by groups of experts that leverage open source ing microgrids. For instance, in the case of a cyberattack,
intelligence, social engineering techniques and zero-day the self-healing should be able to isolate the compromised
vulnerabilities. The contemporary solutions for the energy systems. However, it should be highlighted that this counter-
sector protection are the SIEM systems. In particular, SIEM measure reduces the microgrid’s observability (i.e., the capa-
systems deploy multiple agents in a hierarchical man- bility to estimate the state of each system), thereby affecting
ner to aggregate and normalise information from different the situational awareness and other processes. Consequently,
resources, such as security-related events from end-user by using the visualisation capabilities of SDN, we consider
devices, servers, network devices and operating systems that it is possible to generate efficient self-healing measures
[177], [178]. Typically, these systems are composed of without reducing the observability of the whole grid, thus
six components/processes which are the source device, the providing a powerful mechanism for critical states.
log collection, the parsing/normalisation of the logs, the
rule engine, the log storage and the event monitoring and IX. CONCLUSIONS
retrieval. Moreover, they can integrate specialised security SG includes several asynchronous interconnections among
mechanisms, such as firewalls, antiviruses, and IDPSs in heterogeneous ICT and industrial components that on the
order to analyse logs and issue alert notifications or perform one hand optimise the existing processes of the traditional
another response when a threat is detected. However, the electrical grid, but also generate multiple hazards. In partic-
current SIEM systems present three significant limitations ular, the combination of legacy and smart devices as well
regarding the energy sector. Firstly, their functionality fo- as the huge volume of data generated by them hinder the
cuses only on the ICT environment without having the utilisation of conventional security measures. Moreover, the
ability to control other infrastructures, such as the industrial security gaps of SCADA and SAS protocols like Modbus
systems. Secondly, even if they can operate in the industrial [55]–[57], DNP3 [58] and IEC 61850 [41], [42] enable
sector, usually they utilise corresponding correlation rules cyberattackers to launch various attacks, thus endangering
for a few industrial protocols. Finally, the electrical grid is confidentiality, integrity and availability of the entire SG.
composed of multiple technological entities that generate Hence, an efficient IDPS system capable of protecting SG
a huge volume of data that cannot be efficiently handled communications is considered as a necessary component of
by the current SIEMs. The adaptation and integration of the contemporary electrical grid.
appropriate host and network IDPS systems inside in a In this work, we present a comprehensive compilation of
SIEM will be able to enhance significantly the level of several IDPS systems devoted to protecting SG. In partic-
VOLUME 4, 2016 19
ular, first, we identify the attributes of SG, by analysing [10] E. SpanÃš, L. Niccolini, S. D. Pascoli, and G. Iannacconeluca, “Last-
its main components, the types of networks and the cor- meter smart grid embedded in an internet-of-things platform,” IEEE
Transactions on Smart Grid, vol. 6, no. 1, pp. 468–476, Jan 2015.
responding communication technologies. Next, we provide [11] Y. E. Song, Y. Liu, S. Fang, and S. Zhang, “Research on applications
a comprehensive analysis of various IDPS systems, found of the internet of things in the smart grid,” in 2015 7th International
in the literature based on specific evaluation requirements Conference on Intelligent Human-Machine Systems and Cybernetics,
vol. 2, Aug 2015, pp. 178–181.
that need to be met. More detailed, we analyse and evaluate [12] Y. Lopes, N. C. Fernandes, and K. Obraczka, “Smart grid communi-
37 IDPS systems by studying their architecture, intrusion cation: Requirements and scada protocols analysis,” in 2018 Simposio
detection methodology as well as their programming char- Brasileiro de Sistemas Eletricos (SBSE), May 2018, pp. 1–6.
[13] E. Fadel, V. Gungor, L. Nassef, N. Akkari, M. A. Malik, S. Almasri,
acteristics. Finally, based on this analysis, we specify the and I. F. Akyildiz, “A survey on wireless sensor networks for
appropriate IDPS for SG and determine research directions smart grid,” Computer Communications, vol. 71, pp. 22 – 33,
for future work. 2015. [Online]. Available: http://www.sciencedirect.com/science/article/
pii/S0140366415003400
In our future work, we intend to address the aforemen- [14] Z. E. Mrabet, N. Kaabouch, H. E. Ghazi, and H. E. Ghazi, “Cyber-
tioned deficiencies by developing a SIEM system exclu- security in smart grid: Survey and challenges,” Computers & Electrical
sively for the SG paradigm. The proposed SIEM will be Engineering, vol. 67, pp. 469 – 482, 2018. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0045790617313423
based on the SDN technology and will integrate big data [15] F. Wang, Z. Lei, X. Yin, Z. Li, Z. Cao, and Y. Wang, “Information security
analytics and specification-based techniques. More specifi- in the smart grid: Survey and challenges,” in Geo-Spatial Knowledge and
cally, it will be able to aggregate, normalize and correlate Intelligence, H. Yuan, J. Geng, C. Liu, F. Bian, and T. Surapunt, Eds.
Singapore: Springer Singapore, 2018, pp. 55–66.
various security events as well as decode and analyze [16] K. Wang, M. Du, S. Maharjan, and Y. Sun, “Strategic honeypot game
multiple industrial and ICT protocols, thus defining the model for distributed denial of service attacks in the smart grid,” IEEE
corresponding specification and correlation rules. Transactions on Smart Grid, vol. 8, no. 5, pp. 2474–2482, Sept 2017.
[17] R. C. Diovu and J. T. Agee, “Quantitative analysis of firewall security
under ddos attacks in smart grid ami networks,” in 2017 IEEE 3rd Inter-
X. ACKNOWLEDGEMENT national Conference on Electro-Technology for National Development
(NIGERCON), Nov 2017, pp. 696–701.
This project has received funding from the European [18] ——, “A cloud-based openflow firewall for mitigation against ddos
Union’s Horizon 2020 research and innovation programme attacks in smart grid ami networks,” in 2017 IEEE PES PowerAfrica,
under grant agreement No. 787011 (SPEAR). June 2017, pp. 28–33.
[19] A. Hansen, J. Staggs, and S. Shenoi, “Security analysis of an
advanced metering infrastructure,” International Journal of Critical
REFERENCES Infrastructure Protection, vol. 18, pp. 3 – 19, 2017. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S1874548217300495
[1] D. Von Dollen et al., “Report to nist on the smart grid interoperability [20] Z. Guan, N. Sun, Y. Xu, and T. Yang, “A comprehensive survey of false
standards roadmap,” Electric Power Research Institute (EPRI) and Na- data injection in smart grid,” International Journal of Wireless and Mobile
tional Institute of Standards and Technology, 2009. Computing, vol. 8, no. 1, pp. 27–33, 2015.
[2] M. L. Tuballa and M. L. Abundo, “A review of the development [21] J. Zhao, G. Zhang, M. L. Scala, Z. Y. Dong, C. Chen, and J. Wang, “Short-
of smart grid technologies,” Renewable and Sustainable Energy term state forecasting-aided method for detection of smart grid general
Reviews, vol. 59, pp. 710 – 725, 2016. [Online]. Available: http: false data injection attacks,” IEEE Transactions on Smart Grid, vol. 8,
//www.sciencedirect.com/science/article/pii/S1364032116000393 no. 4, pp. 1580–1590, July 2017.
[3] Y. Kabalci, “A survey on smart metering and smart grid communication,” [22] W. L. Chin, C. H. Lee, and T. Jiang, “Blind false data attacks against ac
Renewable and Sustainable Energy Reviews, vol. 57, pp. 302 – 318, state estimation based on geometric approach in smart grid communica-
2016. [Online]. Available: http://www.sciencedirect.com/science/article/ tions,” IEEE Transactions on Smart Grid, pp. 1–1, 2017.
pii/S1364032115014975 [23] Y. He, G. J. Mendis, and J. Wei, “Real-time detection of false data injec-
[4] N. S. Nafi, K. Ahmed, M. A. Gregory, and M. Datta, “A survey tion attacks in smart grid: A deep learning-based intelligent mechanism,”
of smart grid architectures, applications, benefits and standardization,” IEEE Transactions on Smart Grid, vol. 8, no. 5, pp. 2505–2516, Sept
Journal of Network and Computer Applications, vol. 76, pp. 23 – 36, 2017.
2016. [Online]. Available: http://www.sciencedirect.com/science/article/ [24] J. Zhao, J. Wang, and L. Yin, “Detection and control against replay
pii/S1084804516302314 attacks in smart grid,” in 2016 12th International Conference on Com-
[5] A. A. Cecilia and K. Sudarsanan, “A survey on smart grid,” in 2016 In- putational Intelligence and Security (CIS), Dec 2016, pp. 624–627.
ternational Conference on Emerging Trends in Engineering, Technology [25] T. T. Tran, O. S. Shin, and J. H. Lee, “Detection of replay attacks in smart
and Science (ICETETS), Feb 2016, pp. 1–7. grid systems,” in 2013 International Conference on Computing, Manage-
[6] J. N. Bharothu, M. Sridhar, and R. S. Rao, “A literature survey report ment and Telecommunications (ComManTel), Jan 2013, pp. 298–302.
on smart grid technologies,” in 2014 International Conference on Smart [26] C. Baylon, Lessons from Stuxnet and the Realm of Cyber and Nuclear
Electric Grid (ISEG), Sept 2014, pp. 1–8. Security: Implications for Ethics in Cyber Warfare. Cham: Springer
[7] F. Khan, A. u. Rehman, M. Arif, M. Aftab, and B. K. Jadoon, “A International Publishing, 2017, pp. 213–229. [Online]. Available:
survey of communication technologies for smart grid connectivity,” in https://doi.org/10.1007/978-3-319-45300-2_12
2016 International Conference on Computing, Electronic and Electrical [27] B. BencsÃath, ˛ G. PÃl’k, L. ButtyÃan, ˛ and M. FÃl’legyhÃazi, ˛
Engineering (ICE Cube), April 2016, pp. 256–261. “The cousins of stuxnet: Duqu, flame, and gauss,” Future Internet,
[8] C. Bekara, “Security issues and challenges for the iot- vol. 4, no. 4, pp. 971–1003, 2012. [Online]. Available: http:
based smart grid,” Procedia Computer Science, vol. 34, //www.mdpi.com/1999-5903/4/4/971
pp. 532 – 537, 2014, the 9th International Conference [28] A. O. Otuoze, M. W. Mustafa, and R. M. Larik, “Smart grids security
on Future Networks and Communications (FNC’14)/The 11th challenges: Classification by sources of threats,” Journal of Electrical
International Conference on Mobile Systems and Pervasive Systems and Information Technology, 2018. [Online]. Available:
Computing (MobiSPC’14)/Affiliated Workshops. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S2314717218300163
http://www.sciencedirect.com/science/article/pii/S1877050914009193 [29] T. Bhatt, C. Kotwal, and N. Chaubey, “Survey on smart grid: Threats,
[9] Z. M. Fadlullah, A.-S. K. Pathan, and K. Singh, “Smart grid internet vulnerabilities and security protocol,” International Journal of Electron-
of things,” Mobile Networks and Applications, Oct 2017. [Online]. ics, Electrical and Computational System (IJEECS), vol. 6, no. 9, pp.
Available: https://doi.org/10.1007/s11036-017-0954-2 340–348, 2017.
20 VOLUME 4, 2016
[30] A. Anzalchi and A. Sarwat, “A survey on security assessment of metering [51] “Ossim: Open source siem,” {https://www.alienvault.com/products/
infrastructure in smart grid systems,” in SoutheastCon 2015. IEEE, April ossim}.
2015, pp. 1–4. [52] “Cyberoam iview : The intelligent logging & reporting solution.” {https:
[31] B. B. Gupta and T. Akhtar, “A survey on smart power grid: frameworks, //www.cyberoam.com/cyberoamiview.html}.
tools, security issues, and solutions,” Annals of Telecommunications, [53] “Prelude siem: Prelude universal open-source siem project.” {https://
vol. 72, no. 9, pp. 517–549, Oct 2017. [Online]. Available: https: www.prelude-siem.org/}.
//doi.org/10.1007/s12243-017-0605-4 [54] V. C. Gungor, D. Sahin, T. Kocak, S. Ergut, C. Buccella, C. Cecati, and
[32] N. Komninos, E. Philippou, and A. Pitsillides, “Survey in smart grid G. P. Hancke, “A survey on smart grid potential applications and com-
and smart home security: Issues, challenges and countermeasures,” IEEE munication requirements,” IEEE Transactions on Industrial Informatics,
Communications Surveys Tutorials, vol. 16, no. 4, pp. 1933–1954, vol. 9, no. 1, pp. 28–42, Feb 2013.
Fourthquarter 2014. [55] I. Modbus, “Modbus application protocol specification v1. 1a,” http:
[33] M. M. Pour, A. Anzalchi, and A. Sarwat, “A review on cyber security //www.modbus.org/docs/Modbus_Application_Protocol_V1_1a.pdf,
issues and mitigation methods in smart grid systems,” in SoutheastCon 2004.
2017, March 2017, pp. 1–4. [56] ——, “Modbus messaging on tcp,” http://www.modbus.org/docs/
[34] S. Tan, D. De, W. Song, J. Yang, and S. K. Das, “Survey of security Modbus_Messaging_Implementation_Guide_V1_0b.pdf, 2004.
advances in smart grid: A data driven approach,” IEEE Communications [57] P. Huitsing, R. Chandia, M. Papa, and S. Shenoi, “Attack taxonomies for
Surveys Tutorials, vol. 19, no. 1, pp. 397–422, Firstquarter 2017. the modbus protocols,” International Journal of Critical Infrastructure
[35] S. Goel and Y. Hong, Security Challenges in Smart Grid Implementation. Protection, vol. 1, pp. 37 – 44, 2008. [Online]. Available: http:
London: Springer London, 2015, pp. 1–39. [Online]. Available: //www.sciencedirect.com/science/article/pii/S187454820800005X
https://doi.org/10.1007/978-1-4471-6663-4_1 [58] S. East, J. Butts, M. Papa, and S. Shenoi, “A taxonomy of attacks on the
[36] A. Sanjab, W. Saad, I. Güvenç, A. I. Sarwat, and S. Biswas, “Smart grid dnp3 protocol,” in Critical Infrastructure Protection III, C. Palmer and
security: Threats, challenges, and solutions,” CoRR, vol. abs/1606.06992, S. Shenoi, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009,
2016. [Online]. Available: http://arxiv.org/abs/1606.06992 pp. 67–81.
[37] D. B. Rawat and C. Bajracharya, “Cyber security for smart grid systems: [59] J. P. Anderson, “Computer security threat monitoring and surveillance,”
Status, challenges and perspectives,” in SoutheastCon 2015, April 2015, Technical Report, James P. Anderson Company, pp. 1–56, 1980.
pp. 1–6. [60] D. E. Denning, “An intrusion-detection model,” IEEE Transactions on
[38] R. Leszczyna, “Standards on cyber security assessment of smart grid,” Software Engineering, vol. SE-13, no. 2, pp. 222–232, Feb 1987.
International Journal of Critical Infrastructure Protection, vol. 22, pp. 70 [61] K. Gurney, An introduction to neural networks. CRC press, 2014.
– 89, 2018. [Online]. Available: http://www.sciencedirect.com/science/ [62] J. Schmidhuber, “Deep learning in neural networks: An overview,”
article/pii/S1874548216301421 Neural Networks, vol. 61, pp. 85 – 117, 2015. [Online]. Available:
[39] A. E. Ibhaze, M. U. Akpabio, and S. N. John, “A review on smart http://www.sciencedirect.com/science/article/pii/S0893608014002135
grid network security issues over 6lowpan,” in Proceedings of the [63] A. Patel, H. Alhussian, J. M. Pedersen, B. Bounabat, J. C.
Second International Conference on Internet of Things, Data and Cloud JÃžnior, and S. Katsikas, “A nifty collaborative intrusion detection
Computing, ser. ICC ’17. New York, NY, USA: ACM, 2017, pp. 180:1– and prevention architecture for smart grid ecosystems,” Computers
180:5. [Online]. Available: http://doi.acm.org/10.1145/3018896.3056797 & Security, vol. 64, pp. 92 – 109, 2017. [Online]. Available:
[40] A. Elgargouri, R. Virrankoski, and M. Elmusrati, “Iec 61850 based http://www.sciencedirect.com/science/article/pii/S0167404816300748
smart grid security,” in 2015 IEEE International Conference on Industrial [64] M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, and B. Scholkopf, “Sup-
Technology (ICIT), March 2015, pp. 2461–2465. port vector machines,” IEEE Intelligent Systems and their Applications,
[41] P. Matouš, “Description of iec 61850 communication,” Faculty of vol. 13, no. 4, pp. 18–28, July 1998.
Information Technology BUT, Tech. Rep., 2018. [Online]. Available: [65] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis
http://www.fit.vutbr.cz/research/view_pub.php.en?id=11832 of the kdd cup 99 data set,” in 2009 IEEE Symposium on Computational
[42] Mackiewicz, “Overview of iec 61850 and benefits,” in 2005/2006 Intelligence for Security and Defense Applications, July 2009, pp. 1–6.
IEEE/PES Transmission and Distribution Conference and Exhibition, [66] R. R. de Azevedo, F. Freitas, S. C. de Almeida, M. J. S. C. Almeida,
May 2006, pp. 376–383. E. C. de Barros C. Filho, and W. C. Veras, “Coresec: An ontology of
[43] M. Sharma and A. Agarwal, “Survey on authentication and encryption security aplied to the business process of management,” in Proceedings
techniquesfor smart grid communication,” in 2016 7th India International of the 2008 Euro American Conference on Telematics and Information
Conference on Power Electronics (IICPE), Nov 2016, pp. 1–5. Systems, ser. EATIS ’08. New York, NY, USA: ACM, 2008, pp. 13:1–
[44] R. Mitchell and I.-R. Chen, “A survey of intrusion detection techniques 13:7. [Online]. Available: http://doi.acm.org/10.1145/1621087.1621100
for cyber-physical systems,” ACM Comput. Surv., vol. 46, no. 4, pp. [67] Y. Zhang, L. Wang, W. Sun, R. C. Green, and M. Alam, “Artificial
55:1–55:29, Mar. 2014. [Online]. Available: http://doi.acm.org/10.1145/ immune system based intrusion detection in a distributed hierarchical
2542049 network architecture of smart grid,” in 2011 IEEE Power and Energy
[45] B. B. ZarpelÃčo, R. S. Miani, C. T. Kawakani, and S. C. de Alvarenga, Society General Meeting, July 2011, pp. 1–8.
“A survey of intrusion detection in internet of things,” Journal [68] L. Dhanabal and S. Shantharajah, “A study on nsl-kdd dataset for intru-
of Network and Computer Applications, vol. 84, pp. 25 – 37, sion detection system based on classification algorithms,” International
2017. [Online]. Available: http://www.sciencedirect.com/science/article/ Journal of Advanced Research in Computer and Communication Engi-
pii/S1084804517300802 neering, vol. 4, no. 6, pp. 446–452, 2015.
[46] A. A. Gendreau and M. Moorman, “Survey of intrusion detection systems [69] S. Revathi and A. Malathi, “A detailed analysis on nsl-kdd dataset
towards an end to end secure internet of things,” in 2016 IEEE 4th Inter- using various machine learning techniques for intrusion detection,” In-
national Conference on Future Internet of Things and Cloud (FiCloud), ternational Journal of Engineering Research and Technology. ESRSA
Aug 2016, pp. 84–90. Publications, vol. 2, no. 12, pp. 1848–1853, 2013.
[47] S. Tan, D. De, W. Song, J. Yang, and S. K. Das, “Survey of security [70] I. H. Witten, E. Frank, M. A. Hall, and C. J. Pal, Data Mining: Practical
advances in smart grid: A data driven approach,” IEEE Communications machine learning tools and techniques. Morgan Kaufmann, 2016.
Surveys Tutorials, vol. 19, no. 1, pp. 397–422, Firstquarter 2017. [71] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H.
[48] W. Tong, L. Lu, Z. Li, J. Lin, and X. Jin, “A survey on intrusion Witten, “The weka data mining software: An update,” SIGKDD Explor.
detection system for advanced metering infrastructure,” in 2016 Sixth Newsl., vol. 11, no. 1, pp. 10–18, Nov. 2009. [Online]. Available:
International Conference on Instrumentation Measurement, Computer, http://doi.acm.org/10.1145/1656274.1656278
Communication and Control (IMCCC), July 2016, pp. 33–37. [72] Q. He and R. S. Blum, “Smart grid monitoring for intrusion and fault
[49] J. Jow, Y. Xiao, and W. Han, “A survey of intrusion detection systems in detection with new locally optimum testing procedures,” in 2011 IEEE
smart grid,” International Journal of Sensor Networks, vol. 23, no. 3, pp. International Conference on Acoustics, Speech and Signal Processing
170–186, 2017. (ICASSP), May 2011, pp. 3852–3855.
[50] R. Leszczyna and M. R. WrÃşbel, “Evaluation of open source siem for [73] M. A. Faisal, Z. Aung, J. R. Williams, and A. Sanchez, “Data-stream-
situation awareness platform in the smart grid environment,” in 2015 based intrusion detection system for advanced metering infrastructure in
IEEE World Conference on Factory Communication Systems (WFCS), smart grid: A feasibility study,” IEEE Systems Journal, vol. 9, no. 1, pp.
May 2015, pp. 1–4. 31–44, March 2015.
VOLUME 4, 2016 21
[74] A. Bifet, G. Holmes, R. Kirkby, and B. Pfahringer, “Moa: Massive online International Symposium on Dependable Computing, Dec 2011, pp. 184–
analysis,” Journal of Machine Learning Research, vol. 11, no. May, pp. 193.
1601–1604, 2010. [95] A. F. Snyder and M. T. G. Stuber, “The ansi c12 protocol suite -
[75] “Massive online analysis,” {https://moa.cms.waikato.ac.nz/}. updated and now with network capabilities,” in 2007 Power Systems
[76] G. H. B. P. Albert Bifet, Ricard Gavalda, Machine Learning for Data Conference: Advanced Metering, Protection, Control, Communication,
Streams with Practical Examples in MOA. MIT Press, 2018, "https: and Distributed Resources, March 2007, pp. 117–122.
//moa.cms.waikato.ac.nz/book/". [96] C. Ko, P. Brutch, J. Rowe, G. Tsafnat, and K. Levitt, “System health
[77] R. Vijayanand, D. Devaraj, and B. Kannapiran, “Support vector ma- and intrusion monitoring using a hierarchy of constraints,” in Recent
chine based intrusion detection system with reduced input features for Advances in Intrusion Detection, W. Lee, L. Mé, and A. Wespi, Eds.
advanced metering infrastructure of smart grid,” in 2017 4th Interna- Berlin, Heidelberg: Springer Berlin Heidelberg, 2001, pp. 190–203.
tional Conference on Advanced Computing and Communication Systems [97] X. Liu, P. Zhu, Y. Zhang, and K. Chen, “A collaborative intrusion detec-
(ICACCS), Jan 2017, pp. 1–7. tion mechanism against false data injection attack in advanced metering
[78] G. Creech and J. Hu, “Generation of a new ids test dataset: Time to infrastructure,” IEEE Transactions on Smart Grid, vol. 6, no. 5, pp. 2435–
retire the kdd collection,” in 2013 IEEE Wireless Communications and 2443, Sept 2015.
Networking Conference (WCNC), April 2013, pp. 4487–4492. [98] R. Mitchell and I. Chen, “Behavior-rule based intrusion detection systems
[79] A. I. Abubakar, H. Chiroma, S. A. Muaz, and L. B. Ila, “A review for safety critical smart grid applications,” IEEE Transactions on Smart
of the advances in cyber security benchmark datasets for evaluating Grid, vol. 4, no. 3, pp. 1254–1263, Sept 2013.
data-driven based intrusion detection systems,” Procedia Computer [99] P. Jokar and V. C. M. Leung, “Intrusion detection and prevention for
Science, vol. 62, pp. 221 – 227, 2015, proceedings of the 2015 zigbee-based home area networks in smart grids,” IEEE Transactions on
International Conference on Soft Computing and Software Engineering Smart Grid, vol. 9, no. 3, pp. 1800–1811, May 2018.
(SCSE’15). [Online]. Available: http://www.sciencedirect.com/science/ [100] Y. Kabalci, IEEE 802.15.4 Technologies for Smart Grids. Singapore:
article/pii/S1877050915025788 Springer Singapore, 2019, pp. 531–550. [Online]. Available: https:
[80] Y. Li, R. Qiu, and S. Jing, “Intrusion detection system using online //doi.org/10.1007/978-981-13-1768-2_15
sequence extreme learning machine (os-elm) in advanced metering in- [101] Z. Alliance and H. P. Alliance, “Smart energy profile 2.0 technical
frastructure of smart grid,” PloS one, vol. 13, no. 2, p. e0192216, 2018. requirements document,” ZigBee-105553, Apr, vol. 24, 2010.
[81] G.-B. Huang, D. H. Wang, and Y. Lan, “Extreme learning machines: [102] M. Attia, H. Sedjelmaci, S. M. Senouci, and E. Aglzim, “A new intrusion
a survey,” International Journal of Machine Learning and Cybernetics, detection approach against lethal attacks in the smart grid:temporal and
vol. 2, no. 2, pp. 107–122, Jun 2011. [Online]. Available: https: spatial based detections,” in 2015 Global Information Infrastructure and
//doi.org/10.1007/s13042-011-0019-y Networking Symposium (GIIS), Oct 2015, pp. 1–3.
[82] I. S. S. D. Archive, “Cer smart metering project,” {www.ucd.ie/issda/ [103] T. H. Morris, B. A. Jones, R. B. Vaughn, and Y. S. Dandass, “Determinis-
CER-electricity}. tic intrusion detection rules for modbus protocols,” in 2013 46th Hawaii
International Conference on System Sciences, Jan 2013, pp. 1773–1781.
[83] P. Chen, S. Yang, J. A. McCann, J. Lin, and X. Yang, “Detection of false
[104] “Snort,” https://www.snort.org/.
data injection attacks in smart-grid systems,” IEEE Communications
[105] W. Park and S. Ahn, “Performance comparison and detection analysis
Magazine, vol. 53, no. 2, pp. 206–213, Feb 2015.
in snort and suricata environment,” Wireless Personal Communications,
[84] N. Boumkheld, M. Ghogho, and M. E. Koutbi, “Intrusion detection
vol. 94, no. 2, pp. 241–252, May 2017. [Online]. Available:
system for the detection of blackhole attacks in a smart grid,” in 2016 4th
https://doi.org/10.1007/s11277-016-3209-9
International Symposium on Computational and Business Intelligence
[106] T. Fleming and H. Wilander, “Network intrusion and detection: An
(ISCBI), Sept 2016, pp. 108–111.
evaluation of snort,” 2018.
[85] T. Issariyakul and E. Hossain, “Introduction to network simulator 2
[107] H. Li, G. Liu, W. Jiang, and Y. Dai, “Designing snort rules to detect
(ns2),” in Introduction to Network Simulator NS2. Springer, 2012, pp.
abnormal dnp3 network data,” in 2015 International Conference on
21–40.
Control, Automation and Information Sciences (ICCAIS), Oct 2015, pp.
[86] C. Perkins, E. Belding-Royer, and S. Das, “Ad hoc on-demand distance 343–348.
vector (aodv) routing,” Internet Engineering Task Force, Tech. Rep., [108] E. Hodo, S. Grebeniuk, H. Ruotsalainen, and P. Tavolato, “Anomaly
2003. detection for simulated iec-60870-5-104 trafiic,” in Proceedings of the
[87] I. Ullah and Q. H. Mahmoud, “An intrusion detection framework for the 12th International Conference on Availability, Reliability and Security,
smart grid,” in 2017 IEEE 30th Canadian Conference on Electrical and ser. ARES ’17. New York, NY, USA: ACM, 2017, pp. 100:1–100:7.
Computer Engineering (CCECE), April 2017, pp. 1–5. [Online]. Available: http://doi.acm.org/10.1145/3098954.3103166
[88] A. Shiravi, H. Shiravi, M. Tavallaee, and A. A. Ghorbani, “Toward [109] P. Matoušek, “Description and analysis of iec 104 protocol,” Faculty
developing a systematic approach to generate benchmark datasets for of Information Technology, Brno University o Technology, Tech. Rep.,
intrusion detection,” Computers & Security, vol. 31, no. 3, pp. 357 – 374, 2017.
2012. [Online]. Available: http://www.sciencedirect.com/science/article/ [110] A. Liaw, M. Wiener et al., “Classification and regression by randomfor-
pii/S0167404811001672 est,” R news, vol. 2, no. 3, pp. 18–22, 2002.
[89] “Intrusion detection evaluation dataset (iscxids2012),” {http://www.unb. [111] N. Goldenberg and A. Wool, “Accurate modeling of modbus/tcp
ca/cic/datasets/ids.html}. for intrusion detection in scada systems,” International Journal of
[90] Y. Zhao and Y. Zhang, “Comparison of decision tree methods for finding Critical Infrastructure Protection, vol. 6, no. 2, pp. 63 – 75,
active objects,” Advances in Space Research, vol. 41, no. 12, pp. 1955 – 2013. [Online]. Available: http://www.sciencedirect.com/science/article/
1959, 2008. [Online]. Available: http://www.sciencedirect.com/science/ pii/S1874548213000243
article/pii/S027311770700796X [112] “Wireshark,” https://www.wireshark.org/.
[91] F. A. A. Alseiari and Z. Aung, “Real-time anomaly-based distributed [113] U. Banerjee, A. Vashishtha, and M. Saxena, “Evaluation of the capabili-
intrusion detection systems for advanced metering infrastructure utilizing ties of wireshark as a tool for intrusion detection,” International Journal
stream data mining,” in 2015 International Conference on Smart Grid and of computer applications, vol. 6, no. 7, 2010.
Clean Energy Technologies (ICSGCE), Oct 2015, pp. 148–153. [114] V. Ndatinya, Z. Xiao, V. R. Manepalli, K. Meng, and Y. Xiao, “Network
[92] V. Gulisano, M. Almgren, and M. Papatriantafilou, “Metis: A two-tier forensics analysis using wireshark,” International Journal of Security and
intrusion detection system for advanced metering infrastructures,” in Networks, vol. 10, no. 2, pp. 91–106, 2015.
International Conference on Security and Privacy in Communication [115] “Pcapy,” https://www.secureauth.com/labs/open-source-tools/pcapy.
Networks, J. Tian, J. Jing, and M. Srivatsa, Eds. Cham: Springer [116] “Impacket,” https://www.secureauth.com/labs/open-source-tools/
International Publishing, 2015, pp. 51–68. impacket.
[93] M. Stonebraker, U. Çetintemel, and S. Zdonik, “The 8 requirements of [117] S. D. Anton, S. Kanoor, D. Fraunholz, and H. D. Schotten, “Evaluation
real-time stream processing,” SIGMOD Rec., vol. 34, no. 4, pp. 42–47, of machine learning-based anomaly detection algorithms on an industrial
Dec. 2005. [Online]. Available: http://doi.acm.org/10.1145/1107499. modbus/tcp data set,” in Proceedings of the 13th International
1107504 Conference on Availability, Reliability and Security, ser. ARES 2018.
[94] R. Berthier and W. H. Sanders, “Specification-based intrusion detection New York, NY, USA: ACM, 2018, pp. 41:1–41:9. [Online]. Available:
for advanced metering infrastructures,” in 2011 IEEE 17th Pacific Rim http://doi.acm.org/10.1145/3230833.3232818
22 VOLUME 4, 2016
[118] A. Lemay and J. M. Fernandez, “Providing SCADA network [140] K. Wong, C. Dillabaugh, N. Seddigh, and B. Nandy, “Enhancing suricata
data sets for intrusion detection research,” in 9th Workshop on intrusion detection system for cyber security in scada networks,” in 2017
Cyber Security Experimentation and Test (CSET 16). Austin, TX: IEEE 30th Canadian Conference on Electrical and Computer Engineering
USENIX Association, 2016. [Online]. Available: https://www.usenix. (CCECE), April 2017, pp. 1–5.
org/conference/cset16/workshop-program/presentation/lemay [141] J. T. Sørensen and M. G. Jaatun, “An analysis of the manufacturing
[119] P. Cunningham and S. J. Delany, “k-nearest neighbour classifiers,” Mul- messaging specification protocol,” in Ubiquitous Intelligence and Com-
tiple Classifier Systems, vol. 34, no. 8, pp. 1–17, 2007. puting, F. E. Sandnes, Y. Zhang, C. Rong, L. T. Yang, and J. Ma, Eds.
[120] A. K. Jain, “Data clustering: 50 years beyond k-means,” Pattern Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 602–615.
Recognition Letters, vol. 31, no. 8, pp. 651 – 666, 2010, award [142] Y. Kwon, H. K. Kim, Y. H. Lim, and J. I. Lim, “A behavior-based
winning papers from the 19th International Conference on Pattern intrusion detection technique for smart grid infrastructure,” in 2015 IEEE
Recognition (ICPR). [Online]. Available: http://www.sciencedirect.com/ Eindhoven PowerTech, June 2015, pp. 1–6.
science/article/pii/S0167865509002323 [143] C. Kriger, S. Behardien, and J.-C. Retonda-Modiya, “A detailed analysis
[121] P. Wang, I. Liao, K. Kao, and J. Huang, “An intrusion detection method of the goose message structure in an iec 61850 standard-based substation
based on log sequence clustering of honeypot for modbus tcp protocol,” automation system,” International Journal of Computers Communica-
in 2018 IEEE International Conference on Applied System Invention tions & Control, vol. 8, no. 5, pp. 708–721, 2013.
(ICASI), April 2018, pp. 255–258. [144] Y. Yang, H.-Q. Xu, L. Gao, Y.-B. Yuan, K. McLaughlin, and S. Sezer,
[122] M. Baykara and R. Das, “A novel honeypot based security approach “Multidimensional intrusion detection system for iec 61850-based scada
for real-time intrusion detection and prevention systems,” Journal networks,” IEEE Transactions on Power Delivery, vol. 32, no. 2, pp.
of Information Security and Applications, vol. 41, pp. 103 – 116, 1068–1078, 2017.
2018. [Online]. Available: http://www.sciencedirect.com/science/article/ [145] M. Kabir-Querrec, S. Mocanu, J.-M. Thiriet, and E. Savary, “Power
pii/S2214212616303295 utility automation cybersecurity: Iec 61850 specification of an intrusion
[123] A. Jicha, M. Patton, and H. Chen, “Scada honeypots: An in-depth analysis detection function,” in 25th European Safety and Reliability Conference
of conpot,” in 2016 IEEE Conference on Intelligence and Security (ESREL 2015), 2015.
Informatics (ISI), Sept 2016, pp. 196–198. [146] H. Yoo and T. Shon, “Novel approach for detecting network anomalies
[124] W. H. E. Day and H. Edelsbrunner, “Efficient algorithms for for substation automation based on iec 61850,” Multimedia Tools and
agglomerative hierarchical clustering methods,” Journal of Classification, Applications, vol. 74, no. 1, pp. 303–318, 2015.
vol. 1, no. 1, pp. 7–24, Dec 1984. [Online]. Available: https: [147] T. K. Moon, “The expectation-maximization algorithm,” IEEE Signal
//doi.org/10.1007/BF01890115 Processing Magazine, vol. 13, no. 6, pp. 47–60, Nov 1996.
[125] “Mongodb,” https://www.mongodb.com/. [148] M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander, “Lof: identifying
[126] K. Banker, MongoDB in action. Manning Publications Co., 2011. density-based local outliers,” in ACM sigmod record, vol. 29, no. 2.
[127] Y. Yang, K. McLaughlin, S. Sezer, Y. B. Yuan, and W. Huang, “Stateful ACM, 2000, pp. 93–104.
intrusion detection for iec 60870-5-104 scada security,” in 2014 IEEE [149] U. K. Premaratne, J. Samarabandu, T. S. Sidhu, R. Beresh, and J.-C.
PES General Meeting | Conference Exposition, July 2014, pp. 1–5. Tan, “An intrusion detection system for iec61850 automated substations,”
[128] J. Hurley, A. Munoz, and S. Sezer, “Itaca: Flexible, scalable network IEEE Transactions on Power Delivery, vol. 25, no. 4, pp. 2376–2383,
analysis,” in 2012 IEEE International Conference on Communications 2010.
(ICC), June 2012, pp. 1069–1073. [150] R. Van Hauser, “Thc-hydra, 8.4 th edn, github. com,” 2017.
[129] Y. Yang, K. McLaughlin, T. Littler, S. Sezer, B. Pranggono, and H. F. [151] “Arp seringe,” http://www.securiteam.com/tools/5QP0I2AC0I.html.
Wang, “Intrusion detection system for iec 60870-5-104 based scada [152] J. Hong, C.-C. Liu, and M. Govindarasu, “Detection of cyber intrusions
networks,” in 2013 IEEE Power Energy Society General Meeting, July using network-based multicast messages for substation automation,” in
2013, pp. 1–5. Innovative Smart Grid Technologies Conference (ISGT), 2014 IEEE
[130] Z. Feng, S. Qin, X. Huo, P. Pei, Y. Liang, and L. Wang, “Snort improve- PES. IEEE, 2014, pp. 1–5.
ment on profinet rt for industrial control system intrusion detection,” in [153] Y. Yang, K. McLaughlin, L. Gao, S. Sezer, Y. Yuan, and Y. Gong,
2016 2nd IEEE International Conference on Computer and Communica- “Intrusion detection system for iec 61850 based smart substations,” in
tions (ICCC), Oct 2016, pp. 942–946. Power and Energy Society General Meeting (PESGM), 2016. IEEE,
[131] S. Simatic, “Profinet system description,” http://www.siemens.fi/ 2016, pp. 1–5.
pool/products/industry/iadt_is/tuotteet/automaatiotekniikka/teollinen_ [154] R. Khan, K. McLaughlin, D. Laverty, and S. Sezer, “Analysis of ieee
tiedonsiirto/profinet/man_pnsystem_description.pdf, 2008. c37.118 and iec 61850-90-5 synchrophasor communication frameworks,”
[132] P. Ferrari, A. Flammini, and S. Vitturi, “Performance analysis of profinet in 2016 IEEE Power and Energy Society General Meeting (PESGM),
networks,” Computer Standards & Interfaces, vol. 28, no. 4, pp. 369 July 2016, pp. 1–5.
– 385, 2006. [Online]. Available: http://www.sciencedirect.com/science/ [155] S. Pan, T. Morris, and U. Adhikari, “Developing a hybrid intrusion detec-
article/pii/S0920548905000528 tion system using data mining for power systems,” IEEE Transactions on
[133] D. Zhang, J. Wang, and H. Zhang, “Peach improvement on profinet- Smart Grid, vol. 6, no. 6, pp. 3104–3113, Nov 2015.
dcp for industrial control system vulnerability detection,” in 2015 [156] “Openpdc,” https://github.com/GridProtectionAlliance/openPDC.
2nd International Conference on Electrical, Computer Engineering [157] G. P. Alliance, “openpdc documentation.”
and Electronics. Atlantis Press, 2015. [Online]. Available: http: [158] R. Khan, A. Albalushi, K. McLaughlin, D. Laverty, and S. Sezer, “Model
//dx.doi.org/10.2991/icecee-15.2015.305 based intrusion detection system for synchrophasor applications in smart
[134] M. Baud and M. Felser, “Profinet io-device emulator based on the man- grid,” in 2017 IEEE Power Energy Society General Meeting, July 2017,
in-the-middle attack,” in 2006 IEEE Conference on Emerging Technolo- pp. 1–5.
gies and Factory Automation, Sept 2006, pp. 437–440. [159] J. Ahrenholz, C. Danilov, T. R. Henderson, and J. H. Kim, “Core: A
[135] S. Li, Y. Huang, B. Tai, and C. Lin, “Using data mining methods to real-time network emulator,” in MILCOM 2008 - 2008 IEEE Military
detect simulated intrusions on a modbus network,” in 2017 IEEE 7th Communications Conference, Nov 2008, pp. 1–7.
International Symposium on Cloud and Service Computing (SC2), Nov [160] S. Tan, W. Song, Q. Dong, and L. Tong, “Score: Smart-grid common
2017, pp. 143–148. open research emulator,” in 2012 IEEE Third International Conference
[136] S. B. Aher and L. Lobo, “Data mining in educational system using weka,” on Smart Grid Communications (SmartGridComm), Nov 2012, pp. 282–
in International Conference on Emerging Technology Trends (ICETT), 287.
vol. 3, 2011, pp. 20–25. [161] Y. Yang, K. McLaughlin, S. Sezer, T. Littler, B. Pranggono, P. Brogan,
[137] F. Cleveland, “Iec tc57 wg15: Iec 62351 security standards for the power and H. F. Wang, “Intrusion detection system for network security in
system information infrastructure,” White Paper, 2012. synchrophasor systems,” in IET International Conference on Information
[138] B. K. K. M. S. Sezer, “Towards a stateful analysis framework for smart and Communications Technologies (IETICT 2013), April 2013, pp. 246–
grid network intrusion detection,” in Proceedings of the 4th International 252.
Symposium for ICS & SCADA Cyber Security Research. BCS Learning [162] P. Li, “Selecting and using virtualization solutions: our experiences with
& Development Ltd., 2016, pp. 1–8. vmware and virtualbox,” Journal of Computing Sciences in Colleges,
[139] “Suricata,” https://suricata-ids.org/. vol. 25, no. 3, pp. 11–17, 2010.
VOLUME 4, 2016 23
[163] D. M. Laverty, R. J. Best, P. Brogan, I. A. Khatib, L. Vanfretti, and D. J. DR. PANAGIOTIS SARIGIANNIDIS is an As-
Morrow, “The openpmu platform for open-source phasor measurements,” sistant Professor in the Department of Informatics
IEEE Transactions on Instrumentation and Measurement, vol. 62, no. 4, and Telecommunications Department of Univer-
pp. 701–709, April 2013. sity of Western Macedonia, Kozani, Greece since
[164] C. COLASOFT, “Ltd.: Colasoft packet builder. 2011,” pp. 08–20, 2011. 2016. He received the B.Sc. and Ph.D. degrees in
[165] G. F. Lyon, Nmap network scanning: The official Nmap project guide to computer science from the Aristotle University
network discovery and security scanning. Insecure, 2009. of Thessaloniki, Thessaloniki, Greece, in 2001
[166] D. Kennedy, J. O’gorman, D. Kearns, and M. Aharoni, Metasploit: the
and 2007, respectively. He has published over 120
penetration tester’s guide. No Starch Press, 2011.
papers in international journals, conferences, and
[167] R. Masood, Z. Anwar et al., “Swam: Stuxnet worm analysis in metas-
ploit,” in Frontiers of Information Technology (FIT), 2011. IEEE, 2011, book chapters. He has been involved in several
pp. 142–147. national, EU and international projects. He is currently the project coor-
[168] S. Sanfilippo et al., “Hping,” 2006. dinator of the H2020 project entitled SPEAR: Secure and PrivatE smArt
[169] S. Gorman, “Electricity grid in us penetrated by spies,” The wall street gRid (H2020-DS-2016-2017/H2020-DS-SC7-2017). His research interests
journal, vol. 8, 2009. include optical and wireless telecommunications, resource allocation, in-
[170] D. Starkey, “Hacker group dragonfly takes aim ternet of things and security and privacy in smart networks.
at us power grid,” https://www.geek.com/tech/
hacker-groupdragonfly-takes-aim-at-us-power-grid-1715157/.
[171] “Risi - the repository of industrial security incidents,” http://www.
risidata.com/.
[172] M. R. Endsley, “Toward a theory of situation awareness in dynamic
systems,” Human factors, vol. 37, no. 1, pp. 32–64, 1995.
[173] B. McGuinness and L. Foy, “A subjective measure of sa: the crew
awareness rating scale (cars),” in Proceedings of the first human per-
formance, situation awareness, and automation conference, Savannah,
Georgia, vol. 16. SA Technologies, 2000, pp. 286–291.
[174] Y. Yang, H. Xu, L. Gao, Y. Yuan, K. McLaughlin, and S. Sezer,
“Multidimensional intrusion detection system for iec 61850-based scada
networks,” IEEE Transactions on Power Delivery, vol. 32, no. 2, pp.
1068–1078, April 2017.
[175] A. Carcano, A. Coletta, M. Guglielmi, M. Masera, I. N. Fovino, and
A. Trombetta, “A multidimensional critical state analysis for detecting
intrusions in scada systems,” IEEE Transactions on Industrial Informat-
ics, vol. 7, no. 2, pp. 179–186, May 2011.
[176] D. Jin, Z. Li, C. Hannon, C. Chen, J. Wang, M. Shahidehpour, and C. W.
Lee, “Toward a cyber resilient and secure microgrid using software-
defined networking,” IEEE Transactions on Smart Grid, vol. 8, no. 5, pp.
2494–2504, Sept 2017.
[177] I. Aguirre and S. Alonso, “Improving the automation of security infor-
mation management: A collaborative approach,” IEEE Security Privacy,
vol. 10, no. 1, pp. 55–59, Jan 2012.
[178] G. Cerullo, V. Formicola, P. Iamiglio, and L. Sgaglione, “Critical
infrastructure protection: having SIEM technology cope with network
heterogeneity,” CoRR, vol. abs/1404.7563, 2014. [Online]. Available:
http://arxiv.org/abs/1404.7563
[179] Z. Li, M. Shahidehpour, and F. Aminifar, “Cybersecurity in distributed
power systems,” Proceedings of the IEEE, vol. 105, no. 7, pp. 1367–1388,
July 2017.
[180] K. Boroojeni, M. H. Amini, A. Nejadpak, T. DragiÄ eviÄĞ, S. S. Iyen-
gar, and F. Blaabjerg, “A novel cloud-based platform for implementation
of oblivious power routing for clusters of microgrids,” IEEE Access,
vol. 5, pp. 607–619, 2017.
PANAGIOTIS RADOGLOU-GRAMMATIKIS
received the Diploma degree (5 years) from the
Dept. of Informatics and Telecommunications
Eng., University of Western Macedonia, Greece,
in 2016. He is now a Ph.D. student in the same
department. His main research interests are in
the area of information security and mainly focus
on intrusion detection, vulnerability research and
applied cryptography. Currently, he is working as
a research associate at the University of Western
Macedonia in national and European funded research projects.
24 VOLUME 4, 2016
VOLUME 4, 2016

TABLE 2: Summary of 37 IDPSs cases in SG.
Detection
Literature work Target System Protocols Attacks Performance Dataset Software
Technique
1. Dos Attacks
2. Packet splitting
1. KDD CUP 1999
3. Command insertion
Etire SG [65]
A. Patel et al. [63] Anomaly-based Not provided 4. Shellcode mutation AUC = 0.99451 Protege [66]
ecosystem
5. Brute force attacks
2. Simulated data
6. Payload mutation
7. Duplicate Insertion
1. DoS attacks 1. CLONALG ACC
Entire SG 2. U2R attacks = [80.1%, 99.7%] NSL-KDD 1. Matlab
Y. Zhang et al. [67] Anomaly-based Not provided
ecosystem 3. R2L attacks 2. AIRS2Parallel ACC [65], [68], [69] 2. WEKA [70], [71]
4. Probing attacks = [82.1%, 98.7%]
Q.He and R.S. Entire SG
Anomaly-based Not provided Not provided TPR = 95% Not required Not provided
Blum [72] ecosystem
1. ACC, FPR, FNR, Size,
Running time, RAM-Hours
of Active Classifier
= 94.67%, 3.31%, 9.13%,
134.55 KB, 3.46 secs., 1.23E-7.
2. ACC, FPR, FNR, Size, 1. KDD CUP 1999

1. DoS attacks
Running time, RAM-Hours of [65]
M.A. Faisal et al. 2. R2L attacks
AMI Anomaly-based Not provided Leveraging Bagging = 98.33%, MOA [74]–[76]
[73] 3. U2R attacks
0.78%, 5.15%, 401.01 KB, 2. NSL-KDD
4. Probing attacks
20.92 secs., 2.22E-6. [65], [68], [69]
3. ACC, FPR, FNR, Size

Running time, RAM-Hours of
Single Classifier Drift =
97.74%, 1.07%, 6.79%,
187.30KB, 6.74 secs., 3.34E-7.
1. Exploits
2. DoS attacks
1. ACC > 90%
3. Fuzzers
R. Vijayanand [77] AMI Anomaly-based Not provided 2. TPR = 89.2% ADFA-LD [78], [79] Matlab
4. Backdoor attacks
3. TNR = 93.4%
5. Worms
6. Generic attacks
1. ACC = 97.239%
CER Smart
Y. Li et al [80] AMI Anomaly-based Not provided Not provided 2. FPR = 5.897% Not provided
Metering Project [82]
3. FNR = 3.614%
1. FPR of the first attack
False data injection = 0%
P.Y. Chen [83] AMI Anomaly-based Not provided Not required Not provided
attacks 2. FPR of the second attack
= 0.43%
1. TPR = 100%
N. Boumkheld et al. 2. ACC = 99% 1. NS2 [85]
AMI Anomaly-based AODV [86] Blackhole attacks Simulated data
[84] 3. Precision = 66% 2. WEKA [70], [71]
4. AUC = 1
25
26
1. DoS attacks
I. Ullah and 2. L2L attacks 1. Precision = 99.70%
AMI Anomaly-based Not provided ISCX2012 [88], [89] WEKA [70], [71]
H. Mahmoud [87] 3. Secure shell attacks 2. TPR = 99.60%
4. Botnet
F.A.A. Alseiari and 1. DoS attacks Figures present the values of
AMI Anomaly-based Not provided Simulated data Not provided
Z. Aung [91] 2. Port scanning TPR and FPR.
V. Gulisano et al. Energy exfiltration
AMI Anomaly-based Not provided TPR = 91% Not provided Not provided
[92] attacks
1. TPR = 100% 1. Table TstBench
1. Meter reading attacks
R. Berthier and Specification- 2. TNR = 99.57% [94]
AMI ANSI C12.22 2. Service switch Not required
W.H. Sanders [94] based 3. CPU Consumption = 0.3% 2. VirtualBox [162]
attacks
4. RAM Consumption = 10MB 3. Python
Specification- False data injection Figures present the values of
X. Liu et al. [97] AMI Not provided Not required Not provided
based attacks TPR
1. TPR = 100%
2. FPR of reckless attacks
R. Mitchell and Specification- 1. Reckless attacks ≤ 0.2%
AMI Not provided Not required Not provided
R. Chen [98] based 2. Random attacks 3. FPR of random attacks
≤ 0.6%
4. ROC curves are presented
1. Spoofing attacks
2. Radio Jamming
3. Replay attacks
P.Jokar and Specification- 4. Stenography attacks 1. Theoretical analysis
AMI 1. ZigBee Not required Matlab
V.Leung [99] based 5. Back-off 2. ROC curves are presented
manipulation

6. DoS against CFP
7. DoS against GTS
Specification- 1. Blackhole attacks 1. TPR = 90%
M. Attia et al. [102] AMI Not provided Not required Matlab
based 2. Time delay attacks 2. FPR = 6%
T.H. Morris et al. Modbus
SCADA Signature-based Not provided Not provided Not required Snort [104]–[106]
[103] [55]–[57]
1. Protocol anomalies
2. Reconnaissance
H. Li et al. [107] SCADA Signature-based DNP3 [58] attacks Not provided Not required Snort [104]–[106]
3. DoS attacks
4. Mixed attacks
VOLUME 4, 2016
VOLUME 4, 2016

1. TPR, FPR, Precision, AUC
of Naive Bayes = 0.846, 0.055,
0.907, 0.905

of IBk = 0.847, 0.300, 0.850,
0.766

of J48 = 0.917, 0.090, 0928,
0.929
1. ARP attacks 4. TPR, FPR, Precision, AUC IEC-104 dataset

E. Hodo et al. [108] SCADA Anomaly-based IEC-104 [109] 2. DoS attacks of RandomForest = 0.914, generated by the WEKA [70], [71]
3. Replay attacks 0.136, 0.919, 0.965 authors

of RandomTree = 0.894,
0.210, 0.895, 0.843

of DecisionTable = 0.917,
0.062, 0.933, 0.963

of OneR = 0.846, 0.328, 0.845,
0.759
1. ACC = 100%
2. Precision = 100% 1. Wireshark
N. Goldenberg and Anomaly- Modbus 3.TPR = 100% Real datasets [112]–[114]
SCADA Not Provided
A. Wool [111] based [55]–[57] 4. TNR = 100% generated by authors 2. Pcapy [115]
5. FPR = 0% 3. Impacket [116]
6. FNR = 0%
1. ACC of SVM with DS1,
DS2 and DS3 is 100%, 100%
and 99.99% respectively
2. ACC of Random Forest

with DS1, DS2 and DS3 is
100%, 99.99% and 99.99%
S.D. Anton et al. Anomaly- Modbus Lemay and
SCADA Not provided Not provided
[117] based [55]–[57] Fernandez [118]
3. ACC of KNN with DS1,
DS2 and DS3 is 99.7%, 99.9%
and 99.9%.
4. ACC of k-means with DS1,

DS2 and DS3 is 98.1%, 55.62%
and 63.36%
1. TPR of reconnaissance 1. Conpot [123],
1. Reconnaissance
P.H. Wang et al. Anomaly- Modbus attacks = 90% Data from a 2. Python 2.7
SCADA attacks
[121] based [55]–[57] 2. TPR of DoS attacks = honeypot 3. MongoDB
2. DoS attacks
95.12% [125], [126]
27
28
1. ACC = 100%
1. Packet injection 2. Precision = 100%
Specification- attacks 3. TPR = 100%
Y. Yang et al. [127] SCADA IEC-104 [109] Not required ITACA [128]
based 2. Replay attacks 4. TNR = 100%
3. Data manipulation 5. FPR = 0%
6. FNR = 0%
1. Unauthorized read
commands
2. Unauthorized reset
commands
3. Unauthorized remote
control and adjustment 1. ACC = 100%
commands 2. Precision = 100%
4. Spontaneous packets 3. TPR = 100%
Y. Yang et al. [129] SCADA Hybrid IEC-104 [109] Not required Snort [104]–[106]
storm 4. TNR = 100%
5. Unauthorized 5. FPR = 0%
interrogation commands 6. FNR = 0%
6. Buffer overflows
7. Unauthorized
broadcast requests
8. IEC-104 port
communication
1. Reconnaissance
attacks
Numerical results are not
Z.Feng et al. [130] SCADA Hybrid Profinet 2. DoS attacks Not required Snort [104]–[106]
provided
3. MiTM attacks
4. Protocol anomalies

1. Reconnaissance 1. ACC of j48 = 99.8361%
attacks 2. ACC of 1st neural network
2. Response injection = 97.4185% 1. Wireshark
Anomaly- Modbus Simulated dataset
S.C. Li et al. [135] SCADA attacks 3. ACC of 2nd neural network [112]–[114]
based [55]–[57] generated by authors
3. Command injection = 97.4603% 2. WEKA [70], [71]
attacks 4. ACC of 3rd neural network
4. DoS attacks = 97.3876%
MMS [141] /
Active power limitation Two examples that were Suricata
B. Kang et al. [138] Substation Signature-based IEC 61850 Not required
attacks detected. [105], [139], [140]
[41], [42]
1. MMS [141] / 1. DoS attacks
IEC 61850 2. Port scanning
1. FPR = 0%
[41], [42] 3. Portable executable Real data from a
Specification- 2. FNR = 1.1% Wireshark
Y. Kwon et al. [142] Substation attacks substation in South
based 3. TPR = 98.9% [112]–[114]
2. GOOSE [143] 4. GOOSE attacks Korea
4. Precision = 100%
/ IEC 61850 5. MMS attacks
[41], [42] 6. SNMP attacks
VOLUME 4, 2016
VOLUME 4, 2016

1. MMS [141] /
IEC 61850
[41], [42]
2. GOOSE [143] 1. DoS attacks 1. ITACA [128]

Specification- Real data from a
Y. Yang et al. [144] Substation / IEC 61850 2. MiTM attacks Not provided 2. Wireshark
based substation in China
[41], [42] 3. Packet injection [112]–[114]
3. SMV [42] /
IEC 61850
[41], [42]
GOOSE [143]
M. Kabir-Querrec et Specification-
Substation / IEC 61850 Not Provided Not provided Not required Not Provided
al. [145] based
[41], [42]
1. MMS [141] /
IEC 61850
[41], [42]
H. Yoo and T. Shon Real data from a
Substation Anomaly-based Not Provided FPR = [1%, 6%] WEKA [70], [71]
[146] substation
2. GOOSE [143]
/ IEC 61850
[41], [42]
1. DoS attacks
2. Traffic analysis 1. Snort [104]–[106]
U. Premaratne et al. IEC 61850 Real data from a
Substation Hybrid attacks Not provided 2. THC Hydra [150]
[149] [41], [42] substation
3. Password cracking 3. Seringe [151]
attacks
1. GOOSE [143]
/ IEC 61850 1. Wireshark
[41], [42] [112]–[114]
Specification- 1. DoS attacks
J. Hong et al. [152] Substation FPR= 1.61 × 10−4 Not required 2. Colasoft Packet
based 2. Replay attacks
2. SMV [42] / Builder [164]
IEC 61850 3. Nmap [165]
[41], [42]
1. MMS [141] /
IEC 61850
[41], [42]
2. GOOSE [143] 1. DoS attacks 1. ITACA [128]

Specification- Real data from a
Y. Yang et al. [153] Substation / IEC 61850 2. MiTM attacks Not provided 2. Wireshark
based substation in China
[41], [42] 3. Packet injection [112]–[114]
3. SMV [42] /
IEC 61850
[41], [42]
1. Single line-to-ground
faults
1. Snort [104]–[106]
2. Replay attacks
S.Pan et al. [155] Synchrophasor Hybrid Not provided ACC = 90.4% Simulated data 2. OpenPDC
3. Command injection
[156], [157]
attacks
4. Disable relay attacks
29
30
1. ARP spoofing attacks

2. Port scanning
3. GPS spoofing 1. NRL core
IEEE C37.118 4. Packet drop attacks [159], [160]
R.Khan et al. [158] Synchrophasor Hybrid Not provided Not required
[154] 5. Replay attacks 2. OpenPMU [163]
6. Command injection 3. C/C++
attacks
7. Physical attacks
1. ITACA [128]
1. Reconnaissance
2. Nmap [165]
Specification- IEEE C37.118 attacks
Y. Yang et al. [161] Synchrophasor FPR= 0% Not required 3. Metasploit
based [154] 2. MiTM attacks
[166], [167]
3. DoS attacks
4. hping [168]

VOLUME 4, 2016

Securing The Smart Grid A Comprehensive

Uploaded by

Copyright:

Available Formats

Securing The Smart Grid A Comprehensive

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Securing The Smart Grid A Comprehensive

Uploaded by

Copyright:

Available Formats

Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.

Digital Object Identifier 10.1109/ACCESS.2017.DOI

Securing the Smart Grid: A

I. INTRODUCTION distribution lines. On the other hand, as illustrated in Fig. 1

from now on in this paper for referring to both previous

WiMAX, PLC, Cellular, Satellite Zigbee, Z-Wave,

Layer 3 Layer 2 Layer 1

WAN Mediums NAN Mediums HAN / IAN / BAN Mediums

WiMAX PLC Cellular WiMAX Wi-Fi PLC Zigbee Z-Wave Wi-Fi

Fiber Cellular Satellite

Industrial communication concerning electricity transfer

FIGURE 3: The SG architecture in terms of communication.

sists of three kinds of rules including: a) access control rules,

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

2. ACC, FPR, FNR, Size, 1. KDD CUP 1999

3. ACC, FPR, FNR, Size

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

2. TPR, FPR, Precision, AUC

3. TPR, FPR, Precision, AUC

1. ARP attacks 4. TPR, FPR, Precision, AUC IEC-104 dataset

5. TPR, FPR, Precision, AUC

6. TPR, FPR, Precision, AUC

7. TPR, FPR, Precision, AUC

2. ACC of Random Forest

4. ACC of k-means with DS1,

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

2. GOOSE [143] 1. DoS attacks 1. ITACA [128]

2. GOOSE [143] 1. DoS attacks 1. ITACA [128]

1. ARP spoofing attacks

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

You might also like