NETWORK SECURITY

(18EC821)
Module-3

Dr. Shivashankar
Professor
Department of Electronics & Communication
Engineering
RRIT, Bangalore

Dr. Shivashankar, E&CE, RRIT 1

 Course Outcomes
After Completion of the course, student will be able to:
▪Explain network security services and mechanisms and explain security
concepts.
▪Understand the concept of Transport Level Security and Secure Socket
Layer.
▪Explain security concerns in Internet Protocol Security.
▪Explain Intruders, Intrusion detection and Malicious Software.
▪Describe Firewalls, Firewall characteristics, Biasing and Configuration.
▪Text Book:
1. Cryptography and Network Security Principles and Practice, Pearson
Education Inc., William Stallings 5th Edition, ISBN: 978-81-317-6166-3.
2. Cryptography and Network Security, Atul Kahate, TMH, 2003.
▪Reference:
▪Cryptography and Network Security, Behrouz A Forouzan, TMH, 2007.
Dr. Shivashankar, E&CE, RRIT 2
 MODULE-3: IP SECURITY
Overview of IP Security
• In 1994, the Internet Architecture Board (IAB) issued a report titled “Security
in the Internet Architecture” (RFC 1636).
• To provide security, the IAB included authentication and encryption as
necessary security features in the next-generation IP, which has been issued
as IPv6.
• It also defines the encrypted, decrypted and authenticated packets. The
protocols needed for secure key exchange and key management.
Uses of IP Security –
• IPsec can be used to do the following things:
• To encrypt application layer data.
• To provide security for routers sending routing data across the public internet.
• To provide authentication without encryption, like to authenticate that the
data originates from a known sender.
• To protect network data by setting up circuits using IPsec tunneling, as with a
Virtual Private Network(VPN) connection.
Dr. Shivashankar, E&CE, RRIT 3
 Applications of IPsec
IPsec provides the capability to secure communications across a LAN, across
private and public WANs, and across the Internet. Examples:
• Secure branch office connectivity over the Internet: A company can build a
secure virtual private network over the Internet or over a public WAN. This
enables a business to rely heavily on the Internet and reduce its need for private
networks, saving costs and network management overhead.
• Secure remote access over the Internet: An end user whose system is
equipped with IP security protocols can make a local call to an Internet Service
Provider (ISP) and gain secure access to a company network.
• Establishing extranet and intranet connectivity with partners: IPsec can be
used to secure communication with other organizations, ensuring authentication
and confidentiality and providing a key exchange mechanism.
• Enhancing electronic commerce security: Even though some Web and
electronic commerce applications have built-in security protocols, the use of
IPsec enhances that security, encrypted and authenticated, adding an additional
layer of security to whatever is provided at the application layer.

Dr. Shivashankar, E&CE, RRIT 4

 Benefits of IPsec
• When IPsec is implemented in a firewall or router, it provides
strong security that can be applied to all traffic crossing the
perimeter.
• IPsec in a firewall is resistant to bypass if all traffic from the
outside must use IP and the firewall.
• IPsec is below the Transport Layer (TCP, UDP) and so is transparent
to applications. There is no need to change software.
• IPsec can be transparent to end users. There is no need to train
users on security mechanisms, issue keying material on a per-user
basis.
• IPsec can provide security for individual users if needed. This is
useful for offsite workers and for setting up a secure virtual
subnetwork within an organization for sensitive applications

Dr. Shivashankar, E&CE, RRIT 5

 Routing Applications
IPsec can assure that
• A router advertisement (a new router advertises its presence)
comes from an authorized router.
• A neighbor advertisement (a router seeks to establish or
maintain a neighbor relationship with a router in another routing
domain) comes from an authorized router.
• A redirect message comes from the router to which the initial IP
packet was sent.
• A routing update is not fake.
• IPsec can also encrypt application layer data and provide security
for routers sending routing data across the public internet.
• IPsec can also be used to provide authentication without
encryption.

Dr. Shivashankar, E&CE, RRIT 6

 IPsec Services
IPsec provides security services at the IP layer by enabling a system to select
required security protocols, determine the algorithm(s) to use for the service(s),
and put in place any cryptographic keys required to provide the requested
services.
Two protocols are used to provide security: an authentication protocol
designated by the header of the protocol, Authentication Header (AH); and a
combined encryption/ authentication protocol designated by the format of the
packet for that protocol, Encapsulating Security Payload (ESP). RFC 4301 lists the
following services:
• Access control
• Connectionless integrity
• Data origin authentication
• Rejection of replayed packets (a form of partial sequence integrity)
• Confidentiality (encryption)
• Limited traffic flow confidentiality

Dr. Shivashankar, E&CE, RRIT 7

 IP Security Policy
▪Fundamental to the operation of IPsec is the concept of a security
policy applied to each IP packet.
▪IPsec policy is determined primarily by the interaction of two
databases, the Security Association Database (SAD) and the Security
Policy Database (SPD).

Figure 19.2 IPsec Architecture

Dr. Shivashankar, E&CE, RRIT 8
 Conti…
Security Associations
▪A key concept that appears in both the authentication and confidentiality
mechanisms for IP is the security association (SA).
▪An association is a one-way logical connection between a sender and a receiver
that affords security services to the traffic carried on it.
A security association is uniquely identified by three parameters.
1. Security Parameters Index (SPI): A bit string assigned to this SA and having
local significance only.
2. IP Destination Address: This is the address of the destination endpoint of the
SA, which may be an end-user system or a network system such as a firewall
or router.
3. Security Protocol Identifier: This field from the outer IP header indicates
whether the association is an AH or ESP security association.
Hence, in any IP packet, the security association is uniquely identified by the
Destination Address in the IPv4 or IPv6 header and the SPI in the enclosed
extension header (AH or ESP).
Dr. Shivashankar, E&CE, RRIT 9
 Security Association Database
▪Security Association Database that defines the parameters associated with each SA. A security
association is normally defined by the following parameters in an SAD entry.
• Security Parameter Index: A 32-bit value selected by the receiving end of an SA to uniquely
identify the SA.
• Sequence Number Counter: A 32-bit value used to generate the Sequence Number field in AH
or ESP headers.
• Sequence Counter Overflow: A flag indicating whether overflow of the Sequence Number
Counter should generate an auditable event and prevent further transmission of packets on this
SA
• Anti-Replay Window: Used to determine whether an inbound AH or ESP packet is a replay.
• AH Information: Authentication algorithm, keys, key lifetimes, and related parameters being
used with AH.
• ESP Information: Encryption and authentication algorithm, keys, initialization values, key
lifetimes, and related parameters being used with ESP.
• Lifetime of this Security Association: A time interval or byte count after which an SA must be
replaced with a new SA or terminated, plus an indication of which of these actions should occur.
• IPsec Protocol Mode: Tunnel, transport, or wildcard.
• Path MTU: Any observed path maximum transmission unit (maximum size of a packet that can
be transmitted without fragmentation) and aging variables.

Dr. Shivashankar, E&CE, RRIT 10

 Security Policy Database
▪IP traffic is related to specific SAs (or no SA in the case of traffic
allowed to bypass IPsec) is the nominal Security Policy Database
(SPD).
▪Each SPD entry is defined by a set of IP and upper-layer protocol
field values, called selectors.
Outbound processing obeys the following general sequence for
each IP packet.
1. Compare the values of the appropriate fields in the packet (the
selector fields) against the SPD to find a matching SPD entry, which
will point to zero or more SAs.
2. Determine the SA if any for this packet and its associated SPI.
3. Do the required IPsec processing (i.e., AH or ESP processing).

Dr. Shivashankar, E&CE, RRIT 11

 Conti…
Selectors determine an SPD entry:
• Remote IP Address: This may be a single IP address, an enumerated list
or range of addresses, or a wildcard (mask) address.
• Local IP Address: This may be a single IP address, an enumerated list or
range of addresses, or a wildcard (mask) address.
• Next Layer Protocol: The IP protocol header (IPv4, IPv6, or IPv6
Extension) includes a field (Protocol for IPv4, Next Header for IPv6 or IPv6
Extension) that designates the protocol operating over IP. If AH or ESP is
used, then this IP protocol header immediately precedes the AH or ESP
header in the packet.
• Name: A user identifier from the operating system. This is not a field in
the IP or upper-layer headers but is available if IPsec is running on the
same operating system as the user.
• Local and Remote Ports: These may be individual TCP or UDP port
values, an enumerated list of ports, or a wildcard port.
Dr. Shivashankar, E&CE, RRIT 12
 IP Traffic Processing
▪IPsec is executed on a packet-by-packet basis.
▪When IPsec is implemented, each outbound IP packet is processed by the IPsec
logic before transmission, and each inbound packet is processed by the IPsec
logic after reception and before passing the packet contents on to the next higher
layer (e.g., TCP or UDP).
Modes of Operations
OUTBOUND PACKETS
Passes through the IPsec module when IPsec is enabled in the system.
Then the following steps occur:
1. IPsec searches the SPD for a match to this packet.
2. If no match is found, then the packet is discarded and an error message is
generated.
3. If a match is found, further processing is determined by the first matching
entry in the SPD. If the policy for this packet is DISCARD, then the packet is
discarded. If the policy is BYPASS, then there is no further IPsec processing;
the packet is forwarded to the network for transmission.
Dr. Shivashankar, E&CE, RRIT 13
 Conti…
▪4. If the policy is PROTECT, then a search is made of the SAD for a matching
entry. If no entry is found, then IKE is invoked to create an SA with the
appropriate keys and an entry is made in the SA.
▪5. The matching entry in the SAD determines the processing for this packet.
Either encryption, authentication, or both can be performed, and either transport
or tunnel mode can be used.

Figure 19.3 Processing Model for Outbound Packets

Dr. Shivashankar, E&CE, RRIT 14
 Conti…
INBOUND PACKETS
An incoming IP packet triggers the IPsec processing.
The following steps occur:
1. IPsec determines whether this is an unsecured IP packet or one that has ESP or AH
headers/trailers, by examining the IP Protocol field (IPv4) or Next Header field (IPv6).
2. If the packet is unsecured, IPsec searches the SPD for a match to this packet.
3. For a secured packet, IPsec searches the SAD. If no match is found, the packet is
discarded. Otherwise, IPsec applies the appropriate ESP or AH processing. Then, the
IP header is processed and stripped off and the packet body is delivered to the next
higher layer, such as TCP.

Figure 19.4 Processing Model for Inbound Packets

Dr. Shivashankar, E&CE, RRIT 15
 Encapsulating Security Payload (ESP)
▪ESP can be used to provide confidentiality, data origin authentication,
connectionless integrity, an anti-replay service and (limited) traffic flow
confidentiality.
▪The set of services provided depends on options selected at the time of Security
Association (SA) establishment and on the location of the implementation in a
network topology.

Figure 19.5 ESP Packet Format

Dr. Shivashankar, E&CE, RRIT 16

 Conti…
Format of an ESP packet. It contains the following fields.
• Security Parameters Index (32 bits): Identifies a security association.
• Sequence Number (32 bits): A monotonically increasing counter value; this
provides an anti-replay function, as discussed for AH.
• Payload Data (variable): This is a transport-level segment (transport mode) or IP
packet (tunnel mode) that is protected by encryption.
• Padding (0 – 255 bytes): The purpose of this field is discussed later.
• Pad Length (8 bits): Indicates the number of pad bytes immediately preceding
this field.
• Next Header (8 bits): Identifies the type of data contained in the payload data
field by identifying the first header in that payload (for example, an extension
header in IPv6, or an upper-layer protocol such as TCP).
• Integrity Check Value (variable): A variable-length field (must be an integral
number of 32-bit words) that contains the Integrity Check Value computed over
the ESP packet minus the Authentication Data field.

Dr. Shivashankar, E&CE, RRIT 17

 Conti…
Encryption and Authentication Algorithms
▪The Payload Data, Padding, Pad Length, and Next Header fields are encrypted by
the ESP service. If the algorithm used to encrypt the payload requires
cryptographic synchronization data, such as an initialization vector (IV), then
these data may be carried explicitly at the beginning of the Payload Data field.
▪If included, an IV is usually not encrypted, although it is often referred to as
being part of the ciphertext.
▪The Integrity Check Value (ICV) field is optional. It is present only if the integrity
service is selected and is provided by either a separate integrity algorithm or a
combined mode algorithm that uses an ICV.
▪The ICV is computed after the encryption is performed. This order of processing
facilitates rapid detection and rejection of replayed or bogus packets by the
receiver prior to decrypting the packet, hence potentially reducing the impact of
denial of service (DoS) attacks.

Dr. Shivashankar, E&CE, RRIT 18

 Conti…
Padding
The Padding field serves several purposes:
• If an encryption algorithm requires the plaintext to be a multiple
of some number of bytes (e.g., the multiple of a single block for a
block cipher), the Padding field is used to expand the plaintext
(consisting of the Payload Data, Padding, Pad Length, and Next
Header fields) to the required length.
• The ESP format requires that the Pad Length and Next Header
fields be right aligned within a 32-bit word. Equivalently, the
ciphertext must be an integer multiple of 32 bits. The Padding field
is used to assure this alignment.
• Additional padding may be added to provide partial traffic-flow
confidentiality by concealing the actual length of the payload.

Dr. Shivashankar, E&CE, RRIT 19

 Transport and Tunnel Modes
TRANSPORT MODE ESP
Transport mode ESP is used to encrypt and optionally authenticate the data carried by IP
(e.g., a TCP segment).
Transport mode operation
1. At the source, the block of data consisting of the ESP trailer plus the entire transport-
layer segment is encrypted and the plaintext of this block is replaced with its
ciphertext to form the IP packet for transmission. Authentication is added if this option
is selected.
2. The packet is then routed to the destination. Each intermediate router needs to
examine and process the IP header plus any plaintext IP extension headers but does
not need to examine the ciphertext.
3. The destination node examines and processes the IP header plus any plaintext IP
extension headers. Then, on the basis of the SPI in the ESP header, the destination
node decrypts the remainder of the packet to recover the plaintext transport-layer
segment.

Dr. Shivashankar, E&CE, RRIT 20

 Conti…

(a) Transport-level security

(b)A virtual private network via tunnel mode

Figure 19.7 Transport-Mode versus Tunnel-Mode Encryption
Dr. Shivashankar, E&CE, RRIT 21
 Conti…

(b)Tunnel mode
(a) Transport mode

Figure 19.8: Figure 19.9 Protocol Operation for ESP

Consider a case in which an external host wishes to communicate with a host on an

internal network protected by a firewall, and in which ESP is implemented in the external
host and the firewalls.

Dr. Shivashankar, E&CE, RRIT 22

 Conti…
TUNNEL MODE ESP
▪Tunnel mode ESP is used to encrypt an entire IP packet.
▪For this mode, the ESP header is prefixed to the packet and then the packet plus
the ESP trailer is encrypted.
▪This method can be used to counter traffic analysis. Because the IP header
contains the destination address and possibly source routing directives and hop-
by-hop option information.
▪Intermediate routers would be unable to process such a packet.
▪Therefore, it is necessary to encapsulate the entire block (ESP header plus
ciphertext plus Authentication Data, if present) with a new IP header that will
contain sufficient information for routing but not for traffic analysis.
▪Whereas the transport mode is suitable for protecting connections between
hosts that support the ESP feature, the tunnel mode is useful in a configuration
that includes a firewall or other sort of security gateway that protects a trusted
network from external networks.

Dr. Shivashankar, E&CE, RRIT 23

 Security Association
• The term security association bundle refers to a sequence of SAs through
which traffic must be processed to provide a desired set of IPsec services.
• The SAs in a bundle may terminate at different endpoints or at the same
endpoints.
• Security associations may be combined into bundles in two ways:
❑ Transport adjacency: Refers to applying more than one security protocol to
the same IP packet without invoking tunneling.
❑ Iterated tunneling: Refers to the application of multiple layers of security
protocols effected through IP tunneling. This approach allows for multiple
levels of nesting, since each tunnel can originate or terminate at a different
IPsec site along the path.

Dr. Shivashankar, E&CE, RRIT 24

 Conti…
Authentication Plus Confidentiality
▪Encryption and authentication can be combined in order to transmit an IP
packet that has both confidentiality and authentication between hosts.
▪ESP WITH AUTHENTICATION OPTION: In this approach, the user first applies ESP
to the data to be protected and then appends the authentication data field.
There are actually two subcases:
▪Transport mode ESP: Authentication and encryption apply to the IP payload
delivered to the host, but the IP header is not protected.
▪Tunnel mode ESP: Authentication applies to the entire IP packet delivered to the
outer IP destination address (e.g., a firewall), and authentication is performed at
that destination.
▪TRANSPORT ADJACENCY: Another way to apply authentication after encryption
is to use two bundled transport SAs, with the inner being an ESP SA and the outer
being an AH SA.
▪TRANSPORT-TUNNEL BUNDLE: The use of authentication prior to encryption
might be preferable for several reasons.
Dr. Shivashankar, E&CE, RRIT 25
 Basic Combinations of Security Associations
▪The IPsec Architecture document lists four examples of combinations of SAs that
must be supported by compliant IPsec hosts (e.g., workstation, server) or security
gateways (e.g. firewall, router).

Fig. 10: Basic Combinations of Security Association

Dr. Shivashankar, E&CE, RRIT 26
 Conti…
▪Each SA can be either AH or ESP. For host-to-host SAs, the mode may be either transport
or tunnel; otherwise it must be tunnel mode.
▪Case 1. All security is provided between end systems that implement IPsec. For any two
end systems to communicate via an SA, they must share the appropriate secret keys.
Among the possible combinations are ate secret keys.
a. AH in transport mode
b. ESP in transport mode
c. ESP followed by AH in transport mode (an ESP SA inside an AH SA)
d. Any one of a, b, or c inside an AH or ESP in tunnel mode
Case 2. Security is provided only between gateways (routers, firewalls, etc.) and no hosts
implement IPsec.
Case 3. This builds on case 2 by adding end-to-end security. The same combinations
discussed for cases 1 and 2 are allowed here. The gateway-to-gateway tunnel provides
either authentication, confidentiality, or both for all traffic between end systems.
Case 4. This provides support for a remote host that uses the Internet to reach an
organization’s firewall and then to gain access to some server or workstation behind the
firewall.

Dr. Shivashankar, E&CE, RRIT 27

 Internet Key Exchange
▪The key management portion of IPsec involves the determination and distribution of
secret keys.
▪A typical requirement is four keys for communication between two applications: transmit
and receive pairs for both integrity and confidentiality.
▪The IPsec Architecture document mandates support for two types of key management:
• Manual: A system administrator manually configures each system with its own keys and
with the keys of other communicating systems. This is practical for small, relatively static
environments.
• Automated: An automated system enables the on-demand creation of keys for SAs and
facilitates the use of keys in a large distributed system with an evolving configuration.
The default automated key management protocol for IPsec is referred to as
ISAKMP/Oakley and consists of the following elements:
• Oakley Key Determination Protocol: Oakley is a key exchange protocol based on the
Diffie-Hellman algorithm but providing added security. Oakley is generic in that it does not
dictate specific formats.
• Internet Security Association and Key Management Protocol (ISAKMP): ISAKMP provides
a framework for Internet key management and provides the specific protocol support,
including formats, for negotiation of security attributes.

Dr. Shivashankar, E&CE, RRIT 28

 Key Determination Protocol
▪IKE key determination is a refinement of the Diffie-Hellman key exchange
algorithm.
▪There is prior agreement on two global parameters:
q, a large prime number; and α, a primitive root of .
A selects a random integer XA as its private key and transmits to B its public key
YA= αXA. Similarly, B selects a random integer XB as its private key and transmits to
A its public key YB= αXB mod q.
▪Each side can now compute the secret session key:
K = (YB) XA mod q = (YA) XB mod q = αXAXB mod q
The Diffie-Hellman algorithm has two attractive features:
• Secret keys are created only when needed. There is no need to store secret keys
for a long period of time, exposing them to increased vulnerability.
• The exchange requires no pre-existing infrastructure other than an agreement
on the global parameters.

Dr. Shivashankar, E&CE, RRIT 29

 Conti…
FEATURES OF IKE KEY DETERMINATION
The IKE key determination algorithm is characterized by five
important features:
1. It employs a mechanism known as cookies to thwart clogging
attacks.
2. It enables the two parties to negotiate a group; this, in essence,
specifies the global parameters of the Diffie-Hellman key exchange.
3. It uses nonces to ensure against replay attacks.
4. It enables the exchange of Diffie-Hellman public key values.
5. It authenticates the Diffie-Hellman exchange to thwart man-in-
the-middle attacks.

Dr. Shivashankar, E&CE, RRIT 30

 Conti…
Three different authentication methods can be used with IKE key
determination:
▪Digital signatures: The exchange is authenticated by signing a
mutually obtainable hash; each party encrypts the hash with its
private key. The hash is generated over important parameters, such
as user IDs and nonces.
▪Public-key encryption: The exchange is authenticated by
encrypting parameters such as IDs and nonces with the sender’s
private key.
▪Symmetric-key encryption: A key derived by some out-of-band
mechanism can be used to authenticate the exchange by symmetric
encryption of exchange parameters.

Dr. Shivashankar, E&CE, RRIT 31

 Header and Payload Formats
IKE HEADER FORMAT
• An IKE message consists of an IKE header followed by one or more payloads.
• All of this is carried in a transport protocol.
• The specification dictates that implementations must support the use of UDP for the
transport protocol.

Figure 19.12 IKE Formats

Dr. Shivashankar, E&CE, RRIT 32

 Conti…
Figure 19.12a shows the header format for an IKE message. It consists of the
following fields.
▪Initiator SPI (64 bits): A value chosen by the initiator to identify a unique IKE
security association (SA).
▪Responder SPI (64 bits): A value chosen by the responder to identify a unique
IKE SA.
▪Next Payload (8 bits): Indicates the type of the first payload in the message;
payloads are discussed in the next subsection.
▪ Major Version (4 bits): Indicates major version of IKE in use.
▪ Minor Version (4 bits): Indicates minor version in use.
▪Exchange Type (8 bits): Indicates the type of exchange.
▪ Flags (8 bits): Indicates specific options set for this IKE exchange. Three bits are
defined so far.
▪ Message ID (32 bits): Used to control retransmission of lost packets and
matching of requests and responses.
▪ Length (32 bits): Length of total message (header plus all payloads) in octets.
Dr. Shivashankar, E&CE, RRIT 33
 Conti…
IKE PAYLOAD TYPES
• All IKE payloads begin with the same generic payload header.
• The Next Payload field has a value of 0 if this is the last payload in
the message; otherwise its value is the type of the next payload.
• The Payload Length field indicates the length in octets of this
payload, including the generic payload header.
• The critical bit is 0 if the sender wants the recipient to skip this
payload if it does not understand the payload type code in the
Next Payload field of the previous payload.
• It is set to 1 if the sender wants the recipient to reject this entire
message if it does not understand the payload type.

Dr. Shivashankar, E&CE, RRIT 34

 Objective Questions
▪1. IPSec is designed to provide security at the _________
a) Transport layer
b) Network layer
c) Application layer
d) Session layer
▪2. In tunnel mode, IPSec protects the ______
a) Entire IP packet
b) IP header
c) IP payload
d) IP trailer
▪3. Which component is included in IP security?
a) Authentication Header (AH)
b) Encapsulating Security Payload (ESP)
c) Internet key Exchange (IKE)
d) All of the mentioned
▪4. WPA2 is used for security in _______
a) Ethernet
b) Bluetooth
c) Wi-Fi
d) Email
▪5. An attempt to make a computer resource unavailable to its intended users is called ______
a) Denial-of-service attack
b) Virus attack
c) Worms attack
d) Botnet process

Dr. Shivashankar, E&CE, RRIT 35

 Conti…
▪6. Extensible authentication protocol is authentication framework frequently used in ______
a) Wired personal area network
b) Wireless networks
c) Wired local area network
d) Wired metropolitan area network
▪7. Pretty good privacy (PGP) is used in ______
a) Browser security
b) Email security
c) FTP security
d) WiFi security
▪8. PGP encrypts data by using a block cipher called ______
a) International data encryption algorithm
b) Private data encryption algorithm
c) Internet data encryption algorithm
d) Local data encryption algorithm
▪9. When a DNS server accepts and uses incorrect information from a host that has no authority giving that information,
then it is called _________
a) DNS lookup
b) DNS hijacking
c) DNS spoofing
d) DNS authorizing
▪9. When a DNS server accepts and uses incorrect information from a host that has no authority giving that information,
then it is called _________
a) DNS lookup
b) DNS hijacking
c) DNS spoofing
d) DNS authorizing
Dr. Shivashankar, E&CE, RRIT 36
Thank You

Dr. Shivashankar, E&CE, RRIT 37