Authentication and Key Agreement Protocol in Hybrid Edge–Fog–Cloud Computing Enhanced by 5G Networks

Abstract

The Internet of Things (IoT) has revolutionized connected devices, with applications in healthcare, data analytics, and smart cities. For time-sensitive applications, 5G wireless networks provide ultra-reliable low-latency communication (URLLC) and fog computing offloads IoT processing. Integrating 5G and fog computing can address cloud computing’s deficiencies, but security challenges remain, especially in Authentication and Key Agreement aspects due to the distributed and dynamic nature of fog computing. This study presents an innovative mutual Authentication and Key Agreement protocol that is specifically tailored to meet the security needs of fog computing in the context of the edge–fog–cloud three-tier architecture, enhanced by the incorporation of the 5G network. This study improves security in the edge–fog–cloud context by introducing a stateless authentication mechanism and conducting a comparative analysis of the proposed protocol with well-known alternatives, such as TLS 1.3, 5G-AKA, and various handover protocols. The suggested approach has a total transmission cost of only 1280 bits in the authentication phase, which is approximately 30% lower than other protocols. In addition, the suggested handover protocol only involves two signaling expenses. The computational cost for handover authentication for the edge user is significantly low, measuring 0.243 ms, which is under 10% of the computing costs of other authentication protocols.

1. Introduction

The edge–fog–cloud (three-tier) computing environment is a distributed computing architecture that combines the capabilities of cloud computing, fog computing, and edge computing to provide a comprehensive and flexible framework for processing, storing, and managing data and services. This environment takes advantage of the strengths of each computing paradigm to address a wide range of applications. Cloud computing provides a scalable infrastructure for storing and processing data and running applications. Cloud resources offer extensive computational power and storage capacity, making them suitable for tasks requiring significant resources and complex data analysis [1]. Fog computing extends the cloud capabilities to the network edge, closer to the data source and end-users. Fog nodes, which include routers, switches, and gateways, process and filter data locally, reducing latency and supporting real-time applications. Fog computing is beneficial for scenarios where low-latency interactions and immediate responses are critical [2]. Edge computing brings computation closer to the data source, often directly on the devices themselves. This minimizes the need for data transmission to distant data centers, reducing latency and bandwidth usage. Edge computing is ideal for applications demanding rapid data processing and quick decision making, such as autonomous vehicles and robotics [3].

Security in an edge–fog–cloud computing environment refers to the set of measures, practices, and technologies to protect data, resources, applications, and communication within the context of these distributed computing paradigms. One of the key aspects of security in an edge–fog–cloud computing environment is authentication, establishing robust authentication processes to confirm entities’ identity and determine their access rights based on authorization policies [4]. User authentication involves verifying an identity asserted by or on behalf of a system entity [5,6]. In the edge–fog–cloud environment, authentication is critical in ensuring secure communication, protecting sensitive data, and mitigating the risks associated with unauthorized access or malicious activities [7]. This environment encompasses a wide range of entities, including cloud servers, fog nodes, edge devices, and users, each with unique characteristics and requirements. The distributed nature of the edge–fog–cloud environment poses challenges to authentication. Due to the mobility of edge users and the low-latency requirements, authentication protocols must be fast to adhere to this requirement. The convergence of fog, cloud, and edge computing significantly benefits data processing, storage, and decision making. However, a robust and high-speed communication infrastructure is required for this convergence to be effective. The 5G network, with its ultra-fast data transfer rates, low latency, and high capacity, provides the necessary capabilities to meet the demanding communication requirements of fog, cloud, and edge computing environments [8]. One of the key advantages of 5G network connectivity is its ability to support massive machine-type communications (mMTC) and ultra-reliable and low-latency communications (URLLC). mMTC enables the efficient handling of a large number of IoT devices, sensors, and actuators, which is a main characteristic of edge and fog computing environments [9]. These devices can generate vast amounts of data to be transmitted and processed in real time. Furthermore, the 5G network’s mMTC capabilities ensure reliable and efficient communication between these devices and fog/cloud resources. Fog computing brings security and authentication challenges that must be carefully addressed [10]. Security and authentication challenges in fog computing arise due to the distributed and heterogeneous nature of the environment, where a variety of devices, communication protocols, and data processing locations are involved. These challenges must be addressed to ensure the confidentiality, integrity, and availability of data and services [11]. Addressing these challenges requires a comprehensive security strategy that combines encryption, authentication, access controls, and monitoring mechanisms. It also involves the collaboration of fog computing stakeholders to develop standardized security protocols and best practices that suit the unique characteristics of the fog computing paradigm [12].

This work aims to design a lightweight mutual Authentication and Key Agreement protocol tailored for the edge–fog–cloud environment, specifically integrated with the 5G network. Hereafter, we will call it the “3-ier AKA” protocol. This protocol provides secure and stateless authentication mechanisms that address the unique challenges posed by the distributed nature of the cloud, fog, and edge computing paradigms. This involves designing mechanisms that allow for the seamless transfer of authentication credentials and cryptographic keys, ensuring continuous service without compromising security while accommodating the dynamic nature of the 5G network and the unique characteristics of the edge–fog–cloud computing. Incorporating a lightweight cryptographic technique with the Advanced Encryption Standard (AES), [13] aims to enhance authentication’s overall security and performance in this complex computing environment.

To achieve this objective, this work has the following main contributions:

• Analyzing the security requirements and constraints of the edge–fog–cloud environment, considering factors such as resource limitations, network latency, and potential attacks during first authentication and handover scenarios. A detailed analysis is given in Section 2.2.
• Based on this analysis, lightweight cryptographic techniques and protocols are designed and developed to enable secure and efficient authentication. The protocol design is explained in Section 3.
• The designed protocol is further integrated to support handover authentication, enabling the mobility characteristic of fog computing and minimizing authentication delays. The handover protocol design is presented in Section 4.
• The designed protocols are evaluated through simulations to assess their security, performance, and scalability in edge–fog–cloud scenarios. The detailed evaluation is presented in Section 5.

The rest of this paper is organized as follows: Section 2 presents a background and a review of related work. Section 3 presents the proposed model, which is derived based on the characteristics of the edge–fog–cloud computing environment and its requirements. Section 4 presents the proposed mutual authentication protocols. Section 5 evaluates the proposed protocol. Section 6 analyzes the features of the proposed security protocols. Finally, Section 7 presents the conclusions and future work.

2. Background and Related Work

2.1. Background

Fog computing is an emerging paradigm that extends cloud computing capabilities to the network edge, closer to end-users and devices [14]. It leverages the concept of the three-tier architecture, which consists of three distinct layers: the cloud, the fog, and the edge [15,16,17]. The cloud represents centralized data centers with high computational and storage capabilities. The cloud tier constitutes a collection of robust servers housed within the physical infrastructure, designed to efficiently manage, store, and process vast volumes of data. This tier offers seamless integration with the edge tier. Nevertheless, a significant challenge arises due to its limitations in meeting the low-latency demands of certain applications. The cloud boasts substantial processing and caching capacities, enabling users and applications to access its resources from anywhere at any time. Fog computing is characterized as a dynamic environment where an extensive array of diverse wireless and occasionally self-governing devices collaboratively communicate with each other and the network. This collaboration enables them to independently execute storage and processing tasks without relying on external entities. The fog encompasses intermediary devices such as routers, gateways, and switches. Fog nodes possess computational power and storage capabilities, facilitating seamless interaction and resource-sharing. The edge layer comprises the devices at the network periphery, including sensors, smartphones, smart vehicles, and IoT devices. Edge nodes exhibit constrained compute, storage, and networking capabilities. In the edge tier of the Internet of Things (IoT) architecture, IoT devices play a vital role in sensing diverse events, executing constrained tasks, and relaying the unprocessed data they collect to the fog tier. While fog computing offers several advantages, it also introduces unique security challenges and vulnerabilities. Some common vulnerabilities in fog computing include unauthorized access, data breaches, denial-of-service (DoS) attacks, compromised intermediary devices, and insecure communication channels [18,19]. The dynamic nature of fog computing environments and the heterogeneity and large-scale deployment of devices make them susceptible to various security threats, as follows:

• Fog computing involves processing and storing data at the edge devices, which increases the risk of unauthorized access and data breaches. Since these devices may not have robust security mechanisms, sensitive data could be exposed to potential attackers [20].
• Fog computing has limited physical resources, including processing power and memory. This limitation makes it challenging to implement strong security measures, leaving resources more vulnerable to attacks [10].
• The heterogeneous nature of fog computing resources and the interoperability of resources can lead to cyber-attackers exploiting the resources, gaining unauthorized access, and disrupting services [14].
• Fog computing involves data processing and storage at the edge, which opens up opportunities for data tampering and integrity breaches.
• Data integrity in a decentralized environment becomes a significant concern [21].

These vulnerabilities necessitate the development of robust authentication mechanisms to mitigate potential risks and safeguard sensitive data in fog computing environments.

The 5G (fifth-generation) architecture is designed to provide higher data rates, lower latency, increased capacity, and improved user experience compared to its predecessor, 4G/LTE. It is based on a flexible and virtualized network infrastructure, enabling the efficient delivery of a wide range of services, including enhanced mobile broadband (eMBB), massive machine-type communications (mMTC), and ultra-reliable low-latency communications (URLLC) [22]. eSIM, short for Embedded Subscriber Identity Module, is a technology that enables the use of a digital SIM card directly embedded within a device, such as a smartphone, tablet, or wearable, without needing a physical SIM card [23]. The 5G architecture is characterized by the following key components [24,25,26,27]:

• User equipment (UE): The mobile devices used by end-users, such as smartphones, tablets, and IoT devices.
• Radio access network (RAN): The RAN connects UEs to the core network. It includes the base stations, antennas, and other radio equipment.
• Core network (CN): The core network handles data processing, authentication, and service delivery. It is a virtualized network that can be optimized for different use cases.
• Network slicing: 5G supports network slicing, allowing the creation of multiple virtual networks with specific performance characteristics to cater to different types of services.

In the context of 5G, fog computing is integrated into the network through multi-access edge computing (MEC). MEC nodes are deployed at the edge of the 5G network, close to end-users [28]. These nodes can host applications, services, and virtualized network functions, enabling data processing and service delivery closer to the point of data generation. As shown in Figure 1, MEC serves as a critical intermediary component by positioning compute resources at base stations and roadside units, which provides low-latency services and supports real-time applications, which is a core characteristic of fog computing, particularly in applications such as augmented reality (AR), virtual reality (VR), autonomous vehicles, industrial automation, and real-time gaming [29,30].The 5G network employs several authentication protocols to ensure secure communication between the user equipment (UE) and the network. Some of the key authentication protocols used in 5G are:

• Authentication and Key Agreement (AKA) [31]: AKA is a challenge–response-based protocol used during the initial connection setup between the UE and the 5G core network. It ensures that only legitimate UEs are granted access to the network.
• Extensible Authentication Protocol (EAP) [32]: EAP is an authentication framework that supports multiple authentication methods. It uses various authentication mechanisms based on the user’s and network’s specific needs.
• Transport Layer Security (TLS) [33]: TLS is a cryptographic protocol that secures data transmission between the UE and network elements. It ensures privacy and integrity during communication.
• Secure Authentication Vector (AV) [31]: This protocol generates authentication vectors used by the AKA protocol for mutual authentication between the UE and the network.
• Subscription Concealed Identifier (SUCI) [31]: The SUCI protects the user’s permanent identity by concealing it behind temporary identifiers during authentication procedures.

These authentication protocols work together to establish a secure connection within the 5G network. They play a vital role in ensuring the integrity and security of the communication between the user and the network components.

2.2. Related Work

The characteristics of fog computing, as specified in Section 2.1, make existing cloud computing authentication protocols inadequate to apply to fog computing [34]. Zhong et al. [35] address securing communication and privacy in fog-based vehicular ad hoc networks (VANETs). They propose an approach for secure and lightweight conditional privacy-preserving authentication. The paper aims to establish secure communication while preserving privacy on VANETs as traditional authentication methods are inefficient and may expose sensitive information, such as vehicle ID, location data, movement path, and the scope of everyday routines. The proposed approach assumes fog-based VANETs, where vehicles and fog nodes communicate. The proposed mechanism employs conditional privacy, where vehicles disclose only necessary information for authentication while preserving anonymity. Cryptographic techniques like bilinear pairings and elliptic curve cryptography (ECC) are used for secure and efficient authentication, minimizing computation and communication overhead. Their results demonstrate a balance between security and efficiency in VANETs. The approach is suitable for resource-constrained vehicular environments. However, scalability aspects and the impact of dynamic network conditions are not addressed.

In [36], Rudri et al. proposed a mutual authentication based on elliptic curve cryptography (ECC) and one-way hash functions. These functions focus on the challenge of establishing mutual authentication between fog nodes and end-users in fog computing environments. The proposed approach establishes secure mutual authentication by leveraging these lightweight cryptographic techniques while minimizing computational and communication overheads. The scheme undergoes a security analysis, demonstrating protection against known attacks. Validation using the security protocol animator (SPAN) of the automated validation of Internet security protocols and applications (AVISPA) tool confirms its effectiveness. However, this protocol requires the edge user to store an extra identity (ID), and no authentication is presented between the fog and the cloud.

The protocol used in [37] is a lightweight authentication protocol designed for resource-constrained devices in the Industrial Internet of Things (IIoT). The IIoT ecosystem faces numerous security vulnerabilities, including industrial espionage and sabotage. The work designed an authentication protocol to operate within the IIoT environment. The environment has sensors, networks, and services that connect and control production systems. It assumes the presence of resource-constrained devices, which may have computational power, memory, and energy efficiency limitations. The authentication protocol uses lightweight operations, including xor, addition, subtraction, and a hash function. The protocol minimizes communication overhead by facilitating authentication with four message exchanges between the participating entities. Furthermore, a performance and security comparison with state-of-the-art protocols reveals its performance efficiency for resource-constrained IIoT devices. The protocol achieves higher security levels comparable to computationally expensive schemes.

Refs. [38,39] raised mutual authentication in the Internet of Vehicles (IoV) environment. Han et al. [38] proposed an anonymous authentication scheme based on fog computing for vehicular ad hoc networks (VANETs). The problem lies in the privacy concerns of using real identities for vehicle authentication in VANETs. Existing privacy-protection schemes rely on anonymous authentication but face challenges related to network congestion and updating anonymous information, resulting in poor real-time performance and potential key leakage. The proposed scheme operates within the VANET environment, assuming the presence of vehicles, roadside units (RSUs), and a trusted authority (TA). The focus is on improving the authentication process and reducing communication burdens between vehicles and RSUs. Their designed protocol has an authority named local authority (LA) responsible for managing vehicles’ anonymous information. Utilizing a third-party authority will lead to elevated expenses, as regular commissions must be paid to these entities. Also, there is no authentication between roadside units. In [39], Chen et al. proposed a mutual authentication protocol for the IoV environment. This protocol can tackle various security concerns, such as identity compromise, identity theft, and replay attacks. Each vehicular user must register with a trust authority (TA), and then TA issues a smart card. The vehicular user can use the smart card to authenticate with the roadside unit. However, it is significant to note that specific vulnerabilities, such as distributed denial-of-service attacks, are not explicitly addressed in their study. Once the attacker sends multiple authentication requests to TA, TA must compare the dynamic login identity (DIDV) and value CV to the stored value. After this, the TA creates a random integer and a session key and saves them to its storage. The approach will cause a waste of storage and does not deal with the risks of denial-of-service attacks.

Blockchain mechanisms are used for anonymous authentication in [40]. With the rapid advancements in modern vehicles and distributed fog services, the expansion of vehicular fog services (VFSs) is becoming increasingly important, necessitating their presence across multiple geographically dispersed data centers. Consequently, the need for cross-data center authentication arises. However, traditional cross-data center authentication models are ill suited for the specific scenario of high-speed moving vehicles accessing VFSs. These models disregarded user privacy and failed to meet the time constraints of driving vehicles. In this protocol, only the fog node closest to the vehicle is required to authenticate vehicles, after which the fog node broadcasts the authentication result to other fog nodes and records the results on the blockchain. Therefore, this paper did not consider a fog node compromise attack. Once a fog node is controlled or compromised, the reported result on the blockchain could be misleading.

In [41], Ibrahim et al. proposed a mutual authentication scheme called Octopus for the edge–fog–cloud network architecture, which utilizes a master secret key for new users to authenticate themselves to the fog server. However, the scheme has a limitation: it openly transmits the user’s identity over a public channel, compromising user anonymity. Also, Octopus is primarily designed for stationary smart cards and devices, which increases the risk of interference from masquerading servers. In a large-scale fog computing environment, reusing the same master password can pose a significant security risk.

Cloud computing and IoT convergence have limitations for low-latency and mobile applications. Fog computing bridges this gap by bringing computation closer to end devices. However, remote and unprotected fog nodes require secure solutions, especially in IoT healthcare systems. In [42], Jia et al. proposed a fog-driven IoT healthcare system authentication key agreement protocol. Fog nodes connect cloud data centers and end devices, aiming for efficient and secure healthcare services. It considers resource-constrained fog nodes, mobile healthcare devices, and low-latency communication requirements. The protocol leverages bilinear pairings to establish secure cryptographic keys among the entities involved, ensuring authentication and secure communication. It also employs an Authenticated Key Agreement (AKA) protocol, which involves three components: a fog node, a cloud server, and sensors. It introduces a security model, provides formal security proof, and defends against common attacks. The performance evaluation considers communication and computation costs. The results demonstrate the protocol’s secure and efficient authentication and key exchange for fog-driven IoT healthcare systems. However, there are some limitations to this protocol. The attacker can perform a password guessing attack. Additionally, it is considerably resource expensive.

Dewanta et al. [43] propose a mutual handover authentication in a vehicular network environment. The paper focuses on the challenge of secure fog computing service handover in vehicular networks. It aims to establish mutual authentication between vehicles and fog nodes (FNs) to ensure the integrity and privacy of vehicular network systems. The environment consists of vehicles, fog nodes deployed on roadside units (RSUs), and a cloud server (CS) that facilitates authentication. The scheme utilizes one-way hash functions and exclusive or operations to ensure its lightweight nature. During the login and service request phases, the cloud server distributes credentials for on-the-road authentication between vehicles and fog nodes. During the handover process, the vehicle and fog node can perform mutual authentication using login credentials. The approach achieves computational efficiency by providing faster computation and reducing the total message size compared to previous authentication schemes in similar environments. This scheme’s validation using the SPAN software based on AVISPA confirms its effectiveness in achieving mutual authentication goals and its resilience against replay and man-in-the-middle attacks. However, this paper does not discuss how CS distributes user credentials to specific fog nodes.

Ref. [44] introduces a novel anonymous handover authentication scheme for fog computing. The paper addresses the problem of secure handover authentication for mobile devices in fog computing environments. Handovers, where mobile devices transition between different fog access points, require authentication mechanisms that ensure secure communication while maintaining efficiency and preserving user privacy. In this protocol, edge users and fog nodes have to register in a registration authority (RA), then retrieve their unique identity and compute the corresponding pseudo-identity. After edge users log in and authenticate with one fog node, they move to another fog node, and the pre-negotiation between the old and new fog nodes happens. Then, the edge user can authenticate with a new fog node. The encryption methods used in this protocol are all lightweight, such as concatenation and bitwise XOR operation. The proposed FogHA scheme leverages symmetric trivariate polynomials to provide low-latency authentication while ensuring security, user anonymity, and resistance against known attacks. The authors analyze the security of FogHA using the real-or-random (ROR) model. The analysis demonstrates the semantic security of the scheme, indicating its resilience against potential attacks. An informal security analysis displays FogHA’s ability to resist various known attacks, including mutual authentication, replay attacks, man-in-the-middle attacks, and impersonation attacks.

Yang et al. [45] proposed a threshold mutual authentication protocol that supports fast handover. The work focuses on the problem of secure and efficient access authentication in vehicular networks. Existing authentication protocols often lack consideration for attacks like single points of failure and fail to reduce authentication delays. The authors aim to tackle these challenges and provide a decentralized authentication architecture that enhances security and efficiency. The paper assumes a vehicular network environment comprising a registration server (RS), edge nodes (ENs) such as roadside units (RSUs) and base stations (BSs), and vehicles. The RS is considered a trusted party responsible for registration and revocation. Vehicles are assumed to be potentially malicious, while attackers could compromise ENs. The communication channel between vehicles and ENs is vulnerable to various attacks. The paper introduces the edge-assisted decentralized authentication (EADA) architecture, which delegates the authentication capability from the RS to distributed ENs. The proposed protocol consists of two authentication scenarios: Auth-I and Auth-II. In Auth-I, vehicles are collaboratively authenticated by a subset of ENs using identity-based signature techniques. The involved ENs are efficiently authenticated in batches by the vehicle. For Auth-II, a vehicle with a valid token can achieve fast handover authentication by utilizing the token as a private credential with the nearest EN, reducing authentication delays significantly. The evaluation of the proposed protocol’s performance reports significant reductions in authentication delays. The proposed EADA architecture and threshold mutual authentication protocol address security requirements such as mutual authentication between vehicles and ENs, secure token generation, and resistance against attacks. However, increasing the number of vehicles will lead to an increase in EN storage.

The previous works on authentication protocols in fog computing, vehicular ad hoc networks, Internet of Things (IoT), TLS 1.3, and 4G/5G networks provide valuable insights and inspiration for developing secure and efficient authentication mechanisms. These works highlight the importance of addressing different computing environments’ unique challenges and requirements, such as resource constraints, mobility, privacy concerns, and scalability. They also demonstrate the application of various cryptographic techniques, including bilinear pairings, elliptic curve cryptography (ECC), one-way hash functions, and blockchain, to achieve secure authentication while minimizing computational and communication overhead.

The study in [46] proposed a fog-computing-based e-learning architecture. The authors extended the learning environment from the cloud to the network edge. The approach enhanced the performance of learning data assessments and reduced the encryption burden on user devices by offloading part of the encryption tasks to fog nodes. It enforces specified policies for data access in the learning context by encrypting courses and exams using various cryptographic techniques. The proposed approach demonstrated that the architecture achieves data confidentiality, fine-grained access control, collusion resistance, and enforceability. Furthermore, the evaluations indicate that the solution is efficient, particularly regarding encryption computation costs.

The work in [47] proposed a lattice-based incremental signature scheme to support an efficient gradational authentication approach for modified data in a fog environment. Fog nodes are allowed to authenticate changes without accessing the original message and signature. This eliminates the need for the edge device to store them locally. The approach supports multi-block and combined incremental operations. The proposed approach reduces computational overhead while maintaining reasonable key and signature lengths. The analysis of the proposed approach indicates that it can withstand known attacks and effectively conserves the limited resources of fog nodes.

Overall, previous works provide a foundation for designing authentication protocols that address the specific challenges and requirements of fog computing, IoT, vehicular networks, and 5G environments. They emphasize the need for secure, lightweight, and efficient authentication mechanisms that ensure privacy, scalability, and resilience against various known attacks. A summary of the aforementioned related work is presented in

Table 1. Literature review summary.

Study	Focus	Main Contributions	Limitations	Mobility	Domain
Zhong et al. [35]	Establishes secure communication while preserving privacy in vehicular ad hoc networks.	Minimizes computational and communication overhead.	Scalability and the impact of dynamic network conditions are not addressed.	Yes	Vehicular ad hoc networks
Rudri et al. [36]	A mutual authentication based on elliptic curve cryptography and hash functions in fog computing.	Minimizes computation and communication overhead. Lightweight cryptographic.	Requires the edge user to store an extra identity, and no authentication between the fog and the cloud.	No	Generic
Lara et al. [37]	A lightweight authentication protocol designed for resource constrained devices in the Industrial Internet of Things (IIoT).	Minimizes communication overhead. Performance efficiency for resource constrained IIoT devices.	Enhanced security for IoV for identity compromise and replay attacks. It stores a key pool for each sensor node. The storage cost is significant when the number of nodes increases.	No	Industrial Internet of Things
Ibrahim et al. [41]	A mutual authentication for the edge–fog–cloud network utilizes a master secret key for new users to authenticate themselves to the fog server.	Computationally efficient, even in the existence of a large number of nodes.	Transmits a user’s identity over a public channel, compromising user anonymity. Reusing the same master password can pose a significant security risk.	No	Generic
Jia et al. [42]	Authentication key agreement protocol for fog-driven IoT healthcare system. The protocol involving three components: fog node, cloud server, and sensors.	Low latency. Leverages bilinear pairings to establish secure keys among the entities involved, ensuring authentication.	An attacker can perform a password guessing attack. Computationally expensive.	No	Healthcare
Dewanta, et al. [43]	A mutual handover authentication in a vehicular network environment. It establishes mutual authentication between vehicles and fog nodes.	Ensure the integrity and privacy of vehicular network. Providing faster computation. Reduces the total message size.	It distributes user credentials to specific fog nodes.	Yes	Vehicular network
Guo et al. [44]	Anonymous handover authentication scheme for fog computing.	Lightweight. Low-latency authentication.	Improved communication and computation cost.	Yes	Generic
Yang, et al. [45]	Threshold mutual authentication protocol that supports fast handover.	Reductions in authentication delays.	It increases the number of vehicles, leading to an increase in edge node storage.	Yes	Vehicular networks
Amor, et al. [46]	Developed an access control in fog-assisted e-learning using cryptographic approaches.	High efficiency. Low complexity. Time efficient.	Large data size and hence high network cost.	No	eLearning
Wang et al. [47]	A lightweight, secure authentication key exchange AKE.	Computational efficiency. Low storage usage. Low communication costs.	Does not consider all the environment characteristics.	Yes	Generic

. The table clearly shows the lack of security mechanisms that consider the characteristics of the three-tier architecture. Hence, this work aims to develop a secure and efficient method for mutual authentication and session key agreement in the edge–fog–cloud three-tier architecture, with a particular focus on enabling quick handover. In addition, the use of 5G technology in a stateless control environment provides protection for our three-tier protocol against various security threats, such as the distributed denial-of-service attack.

3. The Proposed 3-Tier-AKA Model

This section describes the characteristics of the edge–fog–cloud 3-tier environment. The term “3-tier” includes a distributed computing encompassing cloud computing in the top tier, fog computing in the intermediary tier, and edge computing in the foundational tier, as shown in Figure 1. Each of these tiers is pivotal in facilitating streamlined and scalable computation, storage, and data processing, catering to various applications. This section delves into this architecture’s fundamental attributes and constituent elements while investigating scenarios that underline the importance of authentication.

3.1. Environment Characteristics

Figure 1 illustrates the edge–fog–cloud 3-tier architecture. This section presents an in-depth analysis of each tier, elucidating their distinct characteristics using the notation presented in

Table 2. Notation for 3-Tier AKA model.

Notation	Description
$\delta$	A set of edge devices
D	Number of edge devices represented as $\delta =\{{\delta }_1,{\delta }_2,\cdots ,{\delta }_d\}$, where $1\le d\le D$
$\zeta$	A set of fog nodes
Z	Number of fog nodes represented as $\zeta =\{{\zeta }_1,{\zeta }_2,\cdots ,{\zeta }_z\}$, where $1\le z\le Z$
$\gamma$	A set of cloud data centers
C	Number of cloud data centers used as $\gamma =\{{\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_c\}$, where $1\le c\le C$
u	A set of 5G service providers
S	A temporary secret key for each entity
K	A secret key generated by 5G service provider during the registration phase for each entity
O	A security token generated by 5G service provider during the registration phase for each entity

.

The edge tier has the following characteristics:

• Set of D mobile edge devices represented as $\delta =\{{\delta }_1,{\delta }_2,\cdots ,{\delta }_d\}$, where $1\le \mathit{d}\le \mathit{D}$.
• Mobility: Edge devices can move from one place to another [48].
• Interoperability: Edge devices may depend on their operation with other heterogeneous devices and service architectures. The edge tier exhibits heterogeneity due to variations in device architectures, communication, and network configurations [48].

The fog tier has the following characteristics:

• Set of Z fog nodes, represented as $\zeta =\{{\zeta }_1,{\zeta }_2,\cdots ,{\zeta }_z\}$, where $1\le \mathit{z}\le \mathit{Z}$.
• Low latency and real-time interactions: Fog nodes close to the network edge collect, process, and store sensor and device data. This enables low latency and meets the needs of real-time interactions, particularly for latency-sensitive applications [14].
• Heterogeneity [14]: Fog nodes are available in various forms and can be deployed as physical or virtual nodes in diverse environments. They encompass high-performance servers, edge routers, gateways, access points, base stations, and more. These hardware platforms exhibit distinct computation and storage capabilities, run various operating systems (OSs), and support different software applications [14].
• Interoperability: Fog nodes are geographically distributed and interoperate in executing tasks to achieve the required quality of service. This includes interoperation among multiple fog nodes and devices with cloud computing [14].

The cloud tier has the following characteristics:

• Set of C cloud data centers, represented as $\gamma =\{{\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_c\}$, where $1\le \mathit{c}\le \mathit{C}$.
• Service-oriented: Cloud computing services are categorized into infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Users can utilize these services without owning hardware or knowing data locations. Examples include cloud storage, Google App Engine, and online gaming [49].
• On-demand self-service: Consumers can provision computing resources (such as virtual machines, storage, and applications) as needed without requiring human intervention from the service provider [50].
• Broad network access: Cloud resources are accessible over the network and can be accessed by various devices with Internet connectivity, such as laptops, smartphones, and tablets [50].
• Resource pooling: Cloud providers use multi-tenant models to pool and share resources among multiple users, with resources dynamically allocated based on demand. Users typically do not have control over the physical location of the resources [50].
• Rapid elasticity: Cloud resources can be scaled up or down quickly to accommodate changes in demand. This elasticity allows users to access additional resources during peak periods and release them when no longer needed [50].
• Measured service: Cloud systems automatically monitor and track resource usage, enabling users to be billed based on their consumption.

The entities within the 5G network environment have the following assumptions:

• The 5G network backhaul establishes a connection between fog nodes and the cloud data center. To ensure communication, a temporary secret key is assigned for the cloud’s initial entry into the environment.
• Each edge device and fog node has an Electronic Subscriber Identity Module (eSIM) and a temporary secret key S stored in the eSIM.
• After the registration phase, the entity (i.e., edge device, fog node, or cloud server) receives a secret key K and a token O, which are generated by the 5G service provider $u_v$.

3.2. Authentication within the Edge–Fog–Cloud Environment

Within the edge–fog–cloud 3-tier environment, various communication scenarios occur, involving direct communication between edge devices and fog nodes and the interactions between fog nodes themselves and between fog nodes and the cloud. These communication channels must be secured to protect sensitive data and maintain the system’s integrity [7]. Authentication plays a crucial role in the edge–fog–cloud computing environment due to several key needs and challenges:

• Data security: Data are processed and stored across various tiers, from edge devices to fog nodes and cloud servers. Authentication ensures that only authorized entities can access and manipulate sensitive data, reducing the risk of unauthorized data breaches or leaks [51].
• Resource access control: Different architecture tiers have varying levels of resources and capabilities. Authentication helps control access to these resources based on user roles, permissions, and the specific needs of applications, ensuring optimal resource utilization [51].
• Seamless handovers: Devices moving between different tiers, such as from edge to fog or fog to cloud, require smooth handovers without interrupting services. Authentication enables seamless handovers by ensuring the device is authenticated in the environment before communication [52].
• Dynamic mobility: Devices in this environment can be mobile, moving across different fog nodes. Authentication mechanisms are needed to handle dynamic mobility patterns without disrupting services [53].
• Latency and real-time requirements: Applications in fog–edge environments have stringent latency requirements. Authentication mechanisms must be fast and efficient to avoid introducing unacceptable delays [54].
• Adversarial threats: Distributed systems are susceptible to various threats, including man-in-the-middle attacks and impersonation. Authentication mechanisms need to counteract these threats effectively [51].
• Authentication overhead: Introducing authentication processes can lead to communication overhead. Balancing the need for security with the performance impact of authentication is a challenge [43].

This work proposes a mutual authentication protocol. It ensures that the communicating entities (edge devices, fog nodes, and the cloud) authenticate each other’s identities before initiating any data exchange. Given the mobility of edge devices, a mutual security protocol is designed for the handover process. When an edge device transitions to a new fog node, it must undergo mutual identity verification to establish a secure connection. This verification step ensures that the edge device and the new fog node mutually authenticate each other’s identities before enabling communication.

The proposed implementation of a mutual identity verification protocol for handover in the edge–fog–cloud 3-tier environment reinforces the security measures. It addresses the challenges posed by mobile edge devices. Note that by verifying the identities of both parties involved, the protocol guarantees that only authorized edge devices can connect to new fog nodes, reducing the risk of unauthorized access and potential security vulnerabilities. For example, in a smart grid environment, edge devices monitor and control the distribution of electricity in real time. Data integrity and encryption are critical to protecting against cyber-attacks that could disrupt the power supply.

In addition, the 3-Tier AKA protocol suite enables seamless and secure communication, ensuring uninterrupted access to services and resources during the handover process. This enhances the environment’s overall reliability, efficiency, and integrity, simplifying the process of moving between fog nodes without causing interruptions or disruptions.

4. The Proposed 3-Tier AKA Mutual Authentication and Key Agreement (AKA) Protocol

The 3-Tier AKA protocol is structured into four main phases:

• Initialization: Responsible for generating the temporary secret key and master key.
• Registration: Identifies edge devices, fog nodes, and cloud data centers within the 3-tier architecture.
• Authentication and Key Agreement: Ensures the verification process between edge devices, fog nodes, and cloud data centers, and facilitates the creation and distribution of session keys for secure communication.
• Handover verification: Focuses on validating the edge user and the new fog node during the handover process. This phase ensures seamless and secure transitions during handovers.

Table 3. The edge–fog–cloud 3-Tier AKA protocol notation.

Notation	Description
$S_{{\delta }_d}$	A temporary secret key S for edge device ${\delta }_d$
$S_{{\zeta }_z}$	A temporary secret key S for fog node ${\zeta }_z$
$S_{{\gamma }_c}$	A temporary secret key S for cloud data center ${\gamma }_c$
$K_{{\delta }_d}$	A secret key K generated by 5G service provider during the registration phase for edge device ${\delta }_d$
$K_{{\zeta }_z}$	A secret key K generated by 5G service provider during the registration phase for fog node ${\zeta }_z$
$K_{{\gamma }_c}$	A secret key K generated by 5G service provider during the registration phase for cloud data center ${\gamma }_c$
$O_{{\delta }_d}$	A security token O generated by 5G service provider during the registration phase of the edge device ${\delta }_d$
$O_{{\zeta }_z}$	A security token O generated by 5G service provider during the registration phase of the fog node ${\zeta }_z$
$O_{{\gamma }_c}$	A security token O generated by 5G service provider during the registration phase of cloud data center ${\gamma }_c$
$M{S}_{u_v}$	The master secret key of the 5G service provider $u_v$
T	A timestamp T is the current time that both participants record during the session.
$S{K}_{{\delta }_d{\zeta }_z}$	A generated session key to be used by the edge device ${\delta }_d$ and the fog node ${\zeta }_z$
$S{K}_{{\delta }_d{\gamma }_c}$	A generated session key to be used by the edge device ${\delta }_d$ and the cloud data center ${\gamma }_c$
$S{K}_{{\zeta }_z{\gamma }_c}$	A generated session key to be used by the fog node ${\zeta }_z$ and the cloud data center ${\gamma }_c$
$R_{{\delta }_d}$	Response message from 5G service provider to edge device ${\delta }_d$
$R_{{\zeta }_z}$	Response message from 5G service provider to fog node ${\zeta }_z$
$R_{{\gamma }_c}$	Response message from 5G service provider to cloud data center ${\gamma }_c$
$M_{{\delta }_d{\zeta }_z}$	Messages between edge device ${\delta }_d$ and the fog node ${\zeta }_z$
$M_{{\delta }_d{\gamma }_c}$	Messages between edge device ${\delta }_d$ and the cloud data center ${\gamma }_c$
$M_{{\gamma }_c{\zeta }_z}$	Messages between fog node ${\zeta }_z$ and the cloud data center ${\gamma }_c$
$E(p,k)$	The encryption of the plaintext p with the encryption key k using the AES-128 encryption technique.
$D(c,k)$	The decryption of the ciphertext c with the encryption key k using the AES-128 encryption technique.
‖	Concatenates two or more strings, sequences, or values together in a specific order to create a longer string.

complements the modeling notation presented in Table 2 and displays the extended notation utilized in the protocol design.

4.1. Initialization Phase

Figure 2 shows the UML activity diagram for the initialization phase. In the initialization phase, each entity (the edge devices ${\delta }_d$, fog nodes ${\zeta }_z$, and cloud data centers) requests an eSIM from the 5G service provider $u_v$. The eSIM of the edge device includes the device ID (${\delta }_d$ ID) and a temporary secret key $S_{{\delta }_d}$. The fog node’s eSIM includes the node ID (${\zeta }_z$ ID) and a temporary secret key $S_{{\zeta }_z}$. Similarly, the cloud data center’s eSIM includes its ID (${\gamma }_c$ ID) and a temporary secret key $S_{{\gamma }_c}$. The 5G service provider $u_v$ maintains a master secret key $M{S}_{u_v}$. This key does not leave the $u_v$, and there will be one master key per application. The 5G service provider also knows these temporary secret keys.

4.2. Registration Phase

In the registration phase, an entity registers with the 5G service provider. This entity can be an edge device, fog node, or cloud data center. Figure 3 and Figure 4 show the registration protocol of the edge device ${\delta }_d$. The following steps describe the details of this protocol. The registration protocols for the fog nodes and the cloud data centers should follow the same protocol steps.

Step 1: 

The edge device starts the session by sending a hello message, including the device’s ID, i.e., ${\delta }_d$ ID, to register with the 5G service provider $u_v$, as shown in line 1 of Figure 4.

Step 2: 

The 5G service provider $u_v$ receives the hello message, generates a secret key $K_{{\delta }_d}$ and computes a security token $O_{{\delta }_d}$. This token is the encryption (using AES) of the device ID along with the generated secret key $K_{{\delta }_d}$ using the master key of the service provider $M{S}_{u_v}$, denoted by $E(“{\delta }_{d}ID”\Vert K_{{\delta }_d},M{S}_{u_v})$, as shown in line 2 in Figure 4. In line 3, $u_v$ computes a response message $R_{{\delta }_d}=E(K_{{\delta }_d}\Vert O_{{\delta }_d},S_{{\delta }_d})$, to send the generated secret key and the token to the edge device, where $S_{{\delta }_d}$ is the temporary secret key of the edge device.

Step 3: 

The edge device ${\delta }_d$ receives the response message $R_{{\delta }_d}$ from the 5G service provider $u_v$. ${\delta }_d$ decrypts the $R_{{\delta }_d}$ to retrieve and store the secret key $K_{{\delta }_d}$ and the token $O_{{\delta }_d}$, using its temporary secret key $S_{{\delta }_d}$ as follows: $D(K_{{\delta }_d}\Vert O_{{\delta }_d},S_{{\delta }_d})$. Then, in line 4 in Figure 4, the edge device forgets the temporary secret key $S_{{\delta }_d}$ to prevent future replay attacks, and the 5G service provider forgets the temporary and generated device secret keys to keep the protocol stateless as much as possible and to prevent denial-of-service attacks.

This step concludes the registration phase, during which the edge device has a permanent secret key and a sealed security token that only the 5G service provider can open. Note that the service provider does not store any information about the registered edge device.

4.3. Authentication and Key Agreement Phase (Edge Device and Fog Node Authentication Protocol)

In this phase, if a registered edge device ${\delta }_d$ is going to join the network, the Authentication and Key Agreement process between edge device ${\delta }_d$, fog node ${\zeta }_z$, and 5G service provider $u_v$ is performed. Figure 5 and Figure 6 show the steps of the Authentication and Key Agreement protocol to accomplish mutual authentication between fog nodes and edge devices and generate a common session key for future security services.

Step 1: 

The edge device ${\delta }_d$ computes a self-authentication message $proofMe$, which is the encryption of the current timestamp T along with a string literal “${\delta }_{d}to{\zeta }_z$” using the edge secret key $K_{{\delta }_d}$, $proofMe=E(T\Vert “{\delta }_{d}to{\zeta }_z$”, $K_{{\delta }_d})$. The timestamp ensures the refresh of the message and prevents the replay attacks, whereas the string literal requests from the 5G service provider that the edge device ${\delta }_d$ would like to communicate with the fog node ${\zeta }_z$. Then, ${\delta }_d$ transmits $proofMe$ and the security token $O_{{\delta }_d}$ to 5G service provider $u_v$. See lines 1 and 2 in Figure 6.

Step 2: 

The 5G service provider $u_v$ receives the self-authentication message $proofMe$ and the token $O_{{\delta }_d}$ from edge device ${\delta }_d$. Then, $u_v$ decrypts the received token to recognize the identity of the edge device, i.e., the ${\delta }_{d}ID$, and to retrieve the secret key of ${\delta }_d$, i.e., $K_{{\delta }_d}$. $u_v$ decrypts the received $proofMe$ using $K_{{\delta }_d}$, and verifies that T is within the current time skew. If the timestamp is verified, then ${\delta }_d$ is authenticated to $u_v$ and generates a session key $S{K}_{{\delta }_d{\zeta }_z}$ to be used between edge device ${\delta }_d$ and fog node ${\zeta }_z$. However, if the timestamp verification fails, $u_v$ closes the session as presented in lines 2 and 3 in Figure 6. In lines 4 and 5, from Figure 6, $u_v$ encrypts the string “${\delta }_{d}to{\zeta }_z$” along with the generated session key using their master secret key $M{S}_{u_v}$, which is called $proo{f}_{{\delta }_d}$. Note that $proo{f}_{{\delta }_d}$ is a sealed value of the generated session key $M{S}_{{\delta }_d{\zeta }_z}$ that is not saved in the 5G service provider to make it a stateless entity. As shown in line 5 in Figure 6, the retrieved timestamp, the string “${\delta }_{d}to{\zeta }_z$”, the session key $S{K}_{{\delta }_d{\zeta }_z}$, and the $proo{f}_{{\delta }_d}$ are encrypted by the secret key $K_{{\delta }_d}$ to form the message response $R_{{\delta }_d}$, i.e., $R_{{\delta }_d}=E(T\Vert “{\delta }_{d}to{\zeta }_z”\Vert S{K}_{{\delta }_d{\zeta }_z}\Vert proo{f}_{{\delta }_d},K_{{\delta }_d})$. Then, $u_v$ sends $R_{{\delta }_d}$ to the edge device ${\delta }_d$.

Step 3: 

Once edge device ${\delta }_d$ receives the response message $R_{{\delta }_d}$, as shown in line 5 from Figure 6, it decrypts $R_{{\delta }_d}$, and first, verifies the value of the timestamp to authenticate the 5G service provider $u_v$ that verified device ${\delta }_d$ and created a secret session key to be used between ${\delta }_d$ and the fog node ${\zeta }_z$. Note that the value $proo{f}_{{\delta }_d}$ is sent to the edge device, not to the fog node, to avoid involving the fog node in the protocol until the edge device decides to. Now, the edge device decrypts $R_{{\delta }_d}$ to retrieve the session key $S{K}_{{\delta }_d{\zeta }_z}$ and $proo{f}_{{\delta }_d}$. Then, it stores $S{K}_{{\delta }_d{\zeta }_z}$ in its memory and computes a message to the fog node ${\zeta }_z:M_{{\delta }_d{\zeta }_z}=E(T\Vert “{\delta }_{d}to{\zeta }_z”,S{K}_{{\delta }_d{\zeta }_z})$, where T is the timestamp at the current time of line 6 in Figure 6. Finally, ${\delta }_d$ transmits $M_{{\delta }_d{\zeta }_z}$ and $proo{f}_{{\delta }_d}$ to ${\zeta }_z$.

Step 4: 

Fog node ${\zeta }_z$ receives the message $M_{{\delta }_d{\zeta }_z}$ and $proo{f}_{{\delta }_d}$ from edge device ${\delta }_d$ to indicate that an edge device wishes to connect with the fog node ${\zeta }_z$. At this point, the fog node depends on the 5G service provider $u_v$ to validate the request and obtain a secret key to be used between the edge device and the fog node. Therefore, the fog node authenticates itself to the $u_v$ by sending a self-authentication message $proofMe=E(T,K_{{\zeta }_z})$ along with the value $proo{f}_{{\delta }_d}$ and the token $O_{{\zeta }_z}$, as shown in lines 7 and 8 in Figure 6.

Step 5: 

The 5G service provider receives the token $O_{{\zeta }_z}$ by which it recognizes the sender ${\zeta }_z$. It uses the secret key $K_{{\zeta }_z}$ to verify the received $proofMe$ and authenticate the fog node. Then, $u_v$ decrypts the received $roo{f}_{{\delta }_d}$, checks the text “${\delta }_{d}to{\zeta }_z$”, and retrieves the session key $S{K}_{{\delta }_d{\zeta }_z}$. Finally, $u_v$ computes an “OK” response message $R_{{\zeta }_z}=E(T\Vert S{K}_{{\delta }_d{\zeta }_z},K_{{\zeta }_z})$ and transmits $R_{{\zeta }_z}$ to ${\zeta }_z$, as shown in lines 9 and 10 from Figure 6. Note that $u_v$ forgets $O_{{\delta }_d}$, $O_{{\zeta }_z}$, $K_{{\delta }_d}$, $K_{{\zeta }_z}$, $S{K}_{{\delta }_d{\zeta }_z}$, $proofMe$, and $proo{f}_{{\delta }_d}$ after sending $R_{{\zeta }_z}$.

Step 6: 

Once the fog node ${\zeta }_z$ receives the response message $R_{{\zeta }_z}$ from the 5G service provider $u_v$, it decrypts $R_{{\zeta }_z}$, verifies the value of the timestamp to authenticate the 5G service provider $u_v$, retrieves the session key $S{K}_{{\delta }_d{\zeta }_z}$, and stores it in its memory. Then, in line 11 from Figure 6, ${\zeta }_z$ decrypts the received message $M_{{\delta }_d{\zeta }_z}$ (in line 6, in Figure 6) to recognize the string “${\delta }_{d}to{\zeta }_z$” and the identity of the requesting edge device, who was ${\delta }_d$. At this point, ${\zeta }_z$ authenticates ${\delta }_d$. In order for ${\zeta }_z$ to authenticate itself to ${\delta }_d$, it computes and sends the “OK” message $M_{{\zeta }_z{\delta }_d}=E(T\Vert “{\zeta }_{z}to{\delta }_d$”, $S{K}_{{\delta }_d{\zeta }_z})$ to ${\delta }_d$.

Step 7: 

The edge device ${\delta }_d$ decrypts the received message $M_{{\zeta }_z{\delta }_d}$, using the session key $S{K}_{{\delta }_d{\zeta }_z}$, as shown in line 12 in Figure 6. The edge device ${\delta }_d$ authenticates ${\zeta }_z$ if the string literal is “${\zeta }_{z}to{\delta }_d$” and the retrieved timestamp T is within the time skew of the timestamp at line 6 in Figure 6. If successful, the mutual Authentication and Key Agreement process is completed. Note that the real identity of the edge device and the fog node were concealed during this protocol.

4.4. Authentication and Key Agreement Phase (Fog Node and Fog Node Authentication Protocol)

During this phase, a fog node ${\zeta }_1$ aims to establish a secure communication channel with another fog node ${\zeta }_2$ with the presence of the 5G service provider $u_v$. The structure of the fog node-to-fog node Authentication and Key Agreement protocol closely resembles the structure of the edge device-to-fog node protocol described in Section 4.3. Therefore, Figure 7 presents the protocol representation for the fog node-to-fog node protocol, depicting its methodology, accompanied by annotations for each procedural step.

4.5. Authentication and Key Agreement Phase (Fog Node and Cloud Authentication Protocol)

In this phase, a fog node ${\zeta }_z$ establishes a secure communication channel with a cloud data center ${\gamma }_c$ with the presence of the 5G service provider $u_v$. The structure of the fog node-to-cloud data center Authentication and Key Agreement protocol closely resembles the structure of the edge device-to-fog node protocol described in Section 4.3. As a result, Figure 8 presents the protocol representation for the fog node to cloud data center protocol, depicting its methodology and including annotations for each procedural step.

4.6. Edge Device Handover Authentication Phase

During this phase, when an edge device ${\delta }_d$ relocates from one fog node ${\zeta }_1$ to another fog node ${\zeta }_2$, a handover process is initiated. The edge device ${\delta }_d$ undergoes mutual authentication with the new fog node ${\zeta }_2$. The next steps explain the details of this Authentication and Key Agreement protocol during the handover process. They set up mutual authentication between the target fog nodes and edge devices and created a shared session key for future security services, as shown in Figure 9.

Step 1: 

The fog node ${\zeta }_1$ performs the fog-to-fog node mutual authentication protocol and generates a secure session key $S{K}_{{\zeta }_1{\zeta }_2}$, as specified in line 1 in Figure 9.

Step 2: 

Fog node ${\zeta }_1$ generates a secure random key, denoted as $S{K}_{{\delta }_d{\zeta }_2}$, intended for utilization by the target fog node ${\zeta }_2$ and edge device ${\delta }_d$. It then proceeds to generate two handover messages, namely, $M_{{\zeta }_1{\zeta }_2}$ and $M_{{\zeta }_1{\delta }_d}$. The content of message $M_{{\zeta }_1{\zeta }_2}$ consists of the encryption of the present timestamp T along with a string literal “${\zeta }_{1}to{\zeta }_2$” and $S{K}_{{\delta }_d{\zeta }_2}$ using the secret key $K_{{\zeta }_1{\zeta }_2}$, $M_{{\zeta }_1{\zeta }_2}=E(T\Vert “{\zeta }_{1}to{\zeta }_2$”$\Vert S{K}_{{\delta }_d{\zeta }_2},S{K}_{{\zeta }_1{\zeta }_2})$. That is, the message $M_{{\zeta }_1{\delta }_d}$ includes the encrypted timestamp T, the string literal “${\zeta }_{1}to{\delta }_d$”, and the secret key $S{K}_{{\delta }_d{\zeta }_2}$. It is written as $M_{{\zeta }_1{\delta }_d}=E(T\Vert “{\zeta }_{1}to{\delta }_d$” $\Vert S{K}_{{\delta }_d{\zeta }_2},S{K}_{{\zeta }_1{\delta }_d})$. Subsequently, ${\zeta }_1$ transmits $M_{{\zeta }_1{\zeta }_2}$ to ${\zeta }_2$ and $M_{{\zeta }_1{\delta }_d}$ to ${\delta }_d$. Then, ${\zeta }_1$ forgets $S{K}_{{\delta }_d{\zeta }_2}$. See lines 2, 3, 4, and 5 in Figure 9.

Step 3: 

Edge device ${\delta }_d$ and the target fog node ${\zeta }_2$ receive the handover message from fog node ${\zeta }_1$. Then, both ${\delta }_d$ and ${\zeta }_2$ possess a shared session key $S{K}_{{\delta }_d{\zeta }_2}$ to establish a secure channel.

5. Performance Evaluation

The 3-Tier AKA protocol performance analysis and evaluation are presented in this section. The primary objective of this evaluation is to assess the protocol’s efficiency across the following metrics: computational, signaling, communication, and storage costs. A comparative analysis is also conducted against existing protocols to ascertain the protocol’s superiority and identify areas for improvement. The computational cost is examined by identifying the resources required to execute the protocol’s operations, evaluating the time complexity, and understanding the protocol’s computational demands. Furthermore, the signaling cost of the protocol is assessed. This entails analyzing the overhead incurred during the signaling process, such as message exchanges. The communication cost of the protocol is examined, especially the amount of data exchanged between network nodes during protocol execution. An assessment is made to determine the protocol’s efficiency in utilizing network resources. Lastly, the storage cost of the protocol is also examined. This involves analyzing the storage requirements to support the protocol’s operation. To provide a robust evaluation, a comparative analysis is performed against existing well-established alternative protocols. This analysis enables the identification of areas where the protocol outperforms existing solutions and areas where further enhancements can be made. For the evaluation of costs, we suppose that the length of the random number $L_R$ is 128 bits, the AES encryption/decryption block size $L_{AES}$ is 128 bits, the key length of AES $L_K$ is 128 bits, the key length of 5G-AKA/4G EPS-AKA $L_{AKA-K}$ is 256 bits, the length of hash function $SHA-256$ $L_H$ is 256 bits, the identity, temporary identity and anonymous identity $L_{ID}$ are all of length 128 bits, and the length of sequence number SQN in 5G-AKA $L_{SQN}$ is 48 bits. The timestamp $L_{TS}$ is 32 bits. The length of the symmetric polynomial $L_{SP}$ is 384 bits. The NTRU encryption/decryption block size $L_{NTRU}$ is 160 bits.

In [55], identity $L_{I{D}^{*}}$ and timestamp $L_{T{S}^{*}}$ use 64 bits each. The elliptic curve point $L_{ECCp}$ is 320 bits. The hash function $L_{H^{*}}$ is 160 bits. The symmetric key $L_{SyK}$ is 128 bits for encryption.

In TLS 1.3, the master key size $L_{MK}$ is 384 bits, the pre-master key size $L_{PMK}$ is 256 bits, and the client/server random number $L_{C/SR}$ has a size of 256 bits. During the ECDHE process, the client/server private key length $L_{C/SPrK}$ is 256 bits, and the public key length $L_{C/SPuK}$ is 520 bits. The session resumption key size $L_{SRk}$ is 256 bits. In general, the size of the ClientHello or ServerHello messages depends on how many extensions the message has, such as the server name and session ticket. The maximum size of a ClientHello message or ServerHello message is limited to 524,280 bits. We suppose that the ClientHello message has one extension, and its size is 480 bits $L_{CH}$. The ServerHello message also has one extension, and its size is 320 bits $L_{SH}$. The finished message has a fixed length of 288 bits $L_F$. Also, the certificate has a length of 512 bits $L_{Cert}$, the certificate verify length is 512 bits $L_{CertV}$, and the signed certificate timestamp size is 64 bits $L_T$.

5.1. Experimental Setup

We describe the experimental setup in this subsection, which we used to implement and test the 3-Tier AKA protocol against the well-established alternatives mentioned above. The experiments were conducted using Google Colab [56], and Jupyter Notebook [57] provides a powerful and flexible environment for implementing and testing the proposed cryptographic protocol. By leveraging the capabilities of Python and its libraries, we were able to perform complex cryptographic operations and secure communications efficiently. The hardware configuration, featuring an Intel (R) Xeon (R) CPU at 2.20 GHz, ensures adequate computational power to handle the cryptographic operations required in our experiments. We utilized Python as the programming language for the experiments, utilizing its extensive libraries for cryptographic operations and network communications.

Table 4. The utilized Python libraries and their brief descriptions.

Library	Description
time	For measuring and managing time-related operations.
hashlib	For cryptographic hashing functions.
ecies, ecies.utils	For Elliptic Curve Integrated Encryption Scheme (ECIES) functionalities.
binascii	For converting between binary and ASCII.
Crypto.Cipher, Crypto.Random, Crypto.Hash	For various cryptographic operations from the PyCryptodome library.
cryptography.hazmat.primitives	For low-level cryptographic primitives.
os	For interacting with the operating system.
ssl	For creating secure sockets.
socket	For network communications.
sympy	For symbolic mathematics, useful in cryptographic algorithm implementations.

shows the Python libraries utilized in our setup.

The chosen setup allows for reproducibility and ease of collaboration, making it an ideal choice for cryptographic research and development.

5.2. Authentication between Entities

5.2.1. Computational Cost

The need to assess the efficiency and feasibility of the proposed protocol motivates the comparison of our designed protocol with established standards like 5G-AKA, 4G EPS-AKA, and TLS 1.3 in terms of computational costs.

In the proposed authentication protocol, only AES-128 encryption technology is used. In the 5G-AKA protocol, the encryption technology used to encrypt user ID is the Elliptic Curve Integrated Encryption Scheme (ECIES). In 4G EPS-AKA, it uses a hash-based message authentication code key-derivation function (HKDF). In the Transport Layer Security (TLS 1.3) protocol, we chose $TLS\text{\_}AES\text{\_}128\text{\_}GCM\text{\_}SHA256$ as the cipher suite. The CPU (Central Processing Units) running time simulation results are shown in

Table 5. The running time of each operation.

Operation	Time (ms)
Hash function $\left(T_H\right)$	0.107
ECIES encryption ($T_{ECIES\_E}$)	1.166
ECIES decryption ($T_{ECIES\_D}$)	1.164
AES encryption ($T_{AES\_E}$)	0.217
AES decryption ($T_{AES\_D}$)	0.243
ECDHE processing ($T_{ECDHE}$)	1.535
HKDF processing ($T_{HKDF}$)	0.578
Verify certificate ($T_{VerC}$)	0.138

. Figure 6, Figure 8 and Figure 10, Figure 11 and Figure 12 show the Authentication and Key Agreement phase of each protocol.

In 3-Tier AKA, presented in Figure 6, the edge user needs to compute two AES encryptions in lines 1 and 6 and perform two AES decryptions after lines 5 and 12. For the total computational cost, in the 5G network, it performs AES decryption in lines 3 and 9, and AES encryption in lines 4, 5, and 10. In the fog node, there are two AES decryptions in lines 6 and 10, and two encryptions in lines 7 and 12. The total computational cost is seven AES encryptions and eight AES decryption operations.

In the 5G-AKA process, presented in Figure 11, the edge user first calculates its Subscription Concealed Identifier (SUCI) or Globally Unique Temporary Identity (GUTI) using Elliptic Curve Integrated Encryption Scheme (ECIES). It then transmits this identifier to the security anchor function (SEAF) in line 1. Moving to line 2, SEAF forwards the authentication message, along with the serving network ID, to the authentication server function (AUSF). In line 3, AUSF further sends the authentication message to either the unified data management (UDM) or the authentication credential repository and processing function (ARPF). Subsequently, in the next line, UDM/ARPF generates an authentication response message. Upon receiving the authentication response message after line 5, the edge user needs to compute the response, which involves performing seven hash functions. This computation adds to the total computational cost. On the other hand, the 5G network decrypts the SUCI and calculates the authentication messages using nine hash functions. The authentication process is in line 9, the edge user authenticates the 5G core network (SEAF, AUSF, UDM, ARPF). In lines 11 and 13, the SEAF and AUSF, respectively, authenticate the edge user, completing the mutual authentication between the entities involved in the 5G-AKA process.

In the 4G EPS-AKA (Evolved Packet System Authentication and Key Agreement), presented in Figure 11, the process shares similarities with 5G-AKA, but there is a notable difference in the initial steps. Unlike 5G-AKA, there is no ID encryption process at the beginning of 4G EPS-AKA.

Table 6. Abbreviations used in 5G-AKA protocol.

Notation	Description
SUCI	Subscription Concealed Identifier
GUTI	Globally Unique Temporary Identity
SUPI	Subscription Permanent Identifier
SNid	Serving network ID
AV	Authentication vector
AUTH	Authentication token
RES	Response Token
XRES	Expected response token
HXRES	Hash of the expected response token
$K_{AUSF}$	Key used to derive other keys for authentication and encryption
$K_{SEAF}$	Anchor key (in 5G, for the security anchor function)

shows abbreviations used in the 5G-AKA protocol.

Table 7. Abbreviations in 4G EPS-AKA.

Notation	Description
GUTI	Globally Unique Temporary Identity
IMSI	International Mobile Subscriber Identity
XRES	Expected response token
AUTH	Authentication token
RRC	Radio resource control
SNid	Serving network ID
AV	Authentication vector
RES	Response token
NAS	Non-access stratum

shows abbreviations used in the 4G EPS-AKA protocol.

In TLS 1.3, presented in Figure 12, the authentication and key exchange process involves several steps. First, the client computes the shared pre-master key, and then, calculates the master key. These pieces of information are then forwarded to the server in line 2. Upon receiving the ClientHello message from the client, the server computes the shared pre-master key and uses the HKDF (HMAC-based extract-and-expand key derivation function) to derive the master key. Lines 3, 4, and 5 represent optional steps for authentication, which may or may not be used depending on the specific configuration. Once the client receives the ServerHello message in line 6, it verifies the server’s certificate and generates a certificate message (line 7). Subsequently, after line 9, the server verifies the client’s certificate, completing the mutual authentication process. In summary, TLS 1.3 facilitates secure communication between the client and server by exchanging cryptographic keys and verifying certificates to establish a trusted connection. The mutual authentication ensures that both parties can be confident in each other’s identities during the communication session. The computational costs of several protocols are shown in

Table 8. Computational cost comparison.

Protocol	Edge User Computational Cost (ms)	Total Computational Cost (ms)
3-Tier AKA	$2T_{AES\_E}+2T_{AES\_D}=0.920$	$7T_{AES\_E}+9T_{AES\_D}=3.706$
5G-AKA [31]	$T_{ECIES\_E}+7T_H=1.915$	$T_{ECIES\_E}+T_{ECIES\_D}+15T_H=3.935$
4G EPS-AKA [31]	$8T_H=0.856$	$16T_H=1.712$
TLS 1.3 [58]	$T_{ECDHE}+T_{HKDF}+T_{VerC}=2.251$	$2T_{ECDHE}+2T_{HKDF}+2T_{VerC}=4.502$

. The 4G EPS-AKA only costs 0.856 ms in the edge tier. However, edge user IDs are not anonymized during the authentication process. Compared to other protocols, 3-Tier AKA has the highest efficiency.

5.2.2. Signaling Cost

In the context of a communication protocol involving an edge device, a 5G service provider, and a fog node, this section compares different signaling messages involved in various protocols. The primary focus is on the number of signaling messages exchanged.

Table 9. Signaling cost.

Protocol	Signaling Cost
3-Tier AKA	6
5G-AKA [31]	9
4G EPS-AKA [31]	8
TLS 1.3 [58]	3

and Figure 13 summarize the findings of this comparison. The proposed protocol involves six different types of signaling messages that are exchanged between the edge device, the 5G service provider, and the fog node. These signaling messages serve different purposes, including authentication requests, response messages, and key agreement messages. The widely used TLS 1.3 protocol requires only three signaling messages. However, the size of each message in TLS 1.3 is comparatively larger than the messages used in the other protocols being considered. The 5G-AKA protocol involves nine signaling messages for its operation. This higher number of messages can increase communication overhead and potentially cause higher latency. Similarly, the 4G EPS-AKA protocol requires eight signaling messages. Again, this higher number of messages might lead to increased overhead. The protocol proposed in the paper, referred to as 3-Tier AKA, stands out in this comparison. It requires only six signaling messages for its operation, which is fewer than the 5G-AKA and 4G EPS-AKA. This reduced number of signaling messages in 3-Tier AKA makes it more efficient in terms of communication overhead and potentially contributes to lower latency compared to the other authentication mechanisms.

5.2.3. Communication Cost

The total communication cost of the proposed protocol and other related protocols during the Authentication and Key Agreement phase is analyzed and summarized in

Table 10. Total communication cost (bits).

Protocol	Communication Cost (Bits)
3-Tier AKA	$10L_{AES}=1280$
5G-AKA [31]	$6L_{ID}+L_{SQN}+L_{AKAK}+8L_H+2L_R=3376$
4G EPS-AKA [31]	$4L_{ID}+2L_R+2L_{SQN}+7L_H+2L_{AKAK}=3168$
TLS 1.3 [58]	$2L_T+2L_{Cert}+2L_{CertV}+L_{CH}+L_{SH}+L_F=3264$

and Figure 14 as follows:

• 3-Tier AKA: In lines 2, 5, 6, 8, 10, and 12, there are a total of 10 AES encrypted messages, as shown in Figure 6.
• 5G-AKA: In lines 1 and 11, there are a total of six IDs involved during the communication process. During line 3, it includes a sequence number, a random number, two hash functions, and a key. Then, in line 5, there is a random number and two hash functions transferred. In line 7 and line 9 of Figure 10, there are a total of four hash functions transferred.
• 4G EPS-AKA: In the 4G EPS-AKA, it is similar to 5G-AKA, except there is no ID encryption process in the beginning, as shown in Figure 11.
• TLS 1.3: In line 1 of Figure 12, the client sends a ClientHello message to the server. Once the server receives the message, it sends a ServerHello message, certification, certification timestamp, and certification verify back to the client. After the client authenticates the server, in line 3 of Figure 12, it transfers its certification, certification timestamp, and certification verify back to the server.
• 3-Tier AKA only performs 10 AES encryption/decryption operations. Hence, the overall communication cost of 3-Tier AKA amounts to 1280 bits, significantly lower than that of other protocols.

5.2.4. Storage Cost

In the proposed protocol, the edge device only needs to store its identity ${\delta }_{d}ID$, a token $O_{{\delta }_d}$, a secret key $K_{{\delta }_d}$, and a session key $K_{{\delta }_d{\zeta }_z}$. Fewer storage requirements mean more edge users can be accommodated. The edge user storage cost of 3-Tier AKA is 640 bits, which indicates that this protocol is more lightweight when compared to the other three protocols. The edge device storage cost after the Authentication and Key Agreement phases is presented in

Table 11. Storage cost (bits).

Protocol	Storage Cost (Bits)	5G/4G Network/Server (Bits)
3-Tier AKA	$2L_{ID}+2L_K+L_{AES}=640$	$L_K=128$
5G-AKA [31]	$2L_{ID}+2L_{AKAK}+L_{SQN}=688$	$2L_{ID}+2L_{AKAK}+L_{SQN}=688$
4G EPS-AKA [31]	$2L_{ID}+3L_{AKAK}+L_{SQN}=816$	$2L_{ID}+3L_{AKAK}+L_{SQN}=816$
TLS 1.3 [58]	$\begin{array}{c}{L}_T+L_{Cert}+L_{CertV}+L_{SRk}+L_{C/SR}+L_{MK}+\\ L_{C/SPrK}+L_{C/SPuK}=3033\end{array}$	$\begin{array}{c}{L}_T+L_{Cert}+L_{CertV}+L_{SRk}+L_{C/SR}+L_{MK}+\\ L_{C/SPrK}+L_{C/SPuK}=3033\end{array}$

and Figure 15.

5.3. Handover Authentication

5.3.1. Computational Cost

In our proposed authentication protocol, only AES-128 encryption technology is used. In FogHA [44], the encryption technology used is based on t-degree symmetric polynomials. Based on references [59,60], the computation time of a symmetric polynomial message authentication code $T_{MAC}$ closely aligns with the running time of a cryptographic hash function, particularly when the entity’s identity consists of 128 bits. Specifically, when polynomial t degree is 100, calculating a t-degree symmetric polynomial $T_{SP}$ takes approximately 16 times longer than computing $T_{MAC}$ [44]. Number theory research unit (NTRU) encryption is approximately 20 times slower than an AES implementation [61]. The symmetric encryption/decryption key used in [55] is 128 bits. Then, the symmetric encryption/decryption operation time will be the same as AES encryption/decryption. The CPU (central processing unit) running time simulation results are shown in

Table 12. Computational cost (handover).

Operation	Time (ms)
Hash function $T_H$	0.107
AES encryption $T_{AES\_E}$	0.217
AES decryption $T_{AES\_D}$	0.243
T-degree symmetric polynomial $T_{SP}$	1.712
NTRU encryption $T_{NTRU\_E}$	4.340
NTRU decryption $T_{NTRU\_D}$	4.860
Elliptic curve point multiplication operation $T_{ECC}$	5.227
Symmetric encryption operation $T_{SE\_E}$	0.217
Symmetric decryption operation $T_{SE\_D}$	0.243

. This simulation is performed on Google Colab (Intel(R) Xeon(R) CPU @ 2.20 GHz). Figure 9, Figure 16, Figure 17 and Figure 18 show the Authentication and Key Agreement phase of each protocol.

In 3-Tier AKA, shown in Figure 9, the mutual authentication among fog nodes has been previously executed. Hence, this aspect of the cost is not included in the handover authentication section. The edge user is required to carry out one AES decryption after line 4. As for the old fog node, it necessitates two AES encryptions in lines 3 and 4 of Figure 9. The target fog node performs one AES decryption after line 3. The overall computational cost encompasses two AES encryptions and two AES decryptions.

In the FogHA protocol, shown in Figure 16, the handover process begins with the first fog access point (FAPk1) generating a timestamp and a random number in line 1. Subsequently, FAPk1 proceeds to authenticate FAPk2 in line 8 of Figure 16. During this exchange, a temporary session key is established between the FAPs, denoted in lines 4 and 8. Once the temporary session key is in place, FAPk1 sends it to the mobile device in line 9. The mobile device then computes the authentication message and forwards it to the second fog access point (FAPk2) in line 11. In line 13 of Figure 16, FAPk2 performs the authentication of the mobile device. As part of this process, FAPk2 generates the FAP-MD session key, as described in line 14. With FAPk2’s authentication successful, the mobile device verifies FAPk2 and proceeds to calculate the session key in line 17, finalizing the handover procedure in the FogHA protocol. The mobile device of the edge user needs to execute eight hash functions and one t-degree symmetric polynomial computation in lines 10 and 17. As for the first fog access point, it performs three hash functions and two t-degree symmetric polynomial computations in lines 7 and 8. Meanwhile, the second fog access point involves line 4 in the handover pre-negotiation process and lines 13 and 14 in the handover process, requiring 10 hash functions and three t-degree symmetric polynomial computations. Overall, this protocol necessitates a total of 21 hash functions and six t-degree symmetric polynomial executions.

In the quantum-resistant handover authentication protocol, presented in Figure 17, the process unfolds as follows: During lines 1 and 2, the foreign agent (FA) computes the authentication message and sends it to the home agent (HA). Subsequently, in line 3, HA generates its authentication message and transmits it back to FA. In line 5, both HA and FA mutually authenticate each other, leading to the generation of a temporary session key shared between them. Moving to line 6, the mobile device (MD) calculates its authentication message and transmits it to FA for verification. Upon successful authentication of MD by FA in line 9, FA proceeds to generate a new session key. It then securely forwards the encrypted session key and relevant parameters to MD. In line 13, MD performs verification of FA’s identity and calculates the new session key, effectively completing the quantum-resistant handover authentication protocol. The mobile device performs two NTRU (number theory research unit) encryptions and one NTRU decryption in lines 6 and 13, respectively. Additionally, it involves seven hash functions. Considering the overall computational cost, there are five NTRU encryptions, five NTRU decryptions, and 23 hash functions in total.

In Liu et al.’s scheme, shown in Figure 18, the user initiates the process by generating a timestamp, elliptic curve point parameters, and encrypted ID. These are then transmitted to low-Earth-orbit satellites (LEOs) in line 3. Subsequently, LEOs forward the message and its ID to the network control center (NCC) in the following line. Upon receiving the message, NCC first verifies the identity of LEOs in line 5. In line 6, NCC proceeds to verify the user’s identity. Upon successful verification, NCC calculates the session key and the response message in line 7. Once the user receives the response message, they performs a verification of NCC’s identity in line 11. Subsequently, in line 11, the user generates their session key. To ensure the session keys held by both the user and NCC are identical, the message Z is transmitted to NCC in lines 12 and 13. NCC then checks the session key in line 14, ensuring consistency and completing the authentication and session key establishment process. Lines 1 and 2 involve the user inserting their smart card into a card reader, where they perform two elliptic curve point multiplication operations, four hash function operations, and one symmetric encryption operation. In line 11, the user executes one elliptic curve point multiplication operation and two hash function operations. As for the network control center (NCC), lines 6 and 7 require one symmetric decryption operation, five hash function operations, and three elliptic curve point multiplication operations. In line 14, the NCC performs one hash function operation.

Table 13. Abbreviations in FogHA.

Notation	Description
FAPk	kth fog access point
MDi	Mobile devices of user i
PID	Pseudo identities
APK	Pre-negotiation temporary key
$\Delta T$	Maximum transmission delay
TC	Credentials of user
TID	Temporary identity
SK	Session key

shows the abbreviations used in the FogHA protocol.

Table 14. Abbreviations in quantum-resistant handover authentication protocol.

Notation	Description
$FA$	Foreign agent
$HA$	Home agent
$MD$	Mobile device
$L_m$, $L_{m,HA}$, $L_{m,FA}$, p, r	Polynomial
h, F	Public/private key of entity
$h_{SC}$	Trusted third system center public key
$TS$	Timestamp
$SK$	Session key
${MID}_i$, ${MPW}_i$	Hashed ID and password

shows the abbreviations used in the quantum-resistant handover authentication protocol.

Table 15. Abbreviations in Liu et al.’s scheme.

Notation	Description
$U_i$	$ith$ user
$LEOs$	Low-Earth-orbit satellite
$NCC$	Network control center
$ID$, $pw$	Entity’s identity, password
$v_i$, $k_i$, f	Parameters stored in a smart card
G	A base point over $Ep(a,b)$ with prime order n
$P_{NCC}$	The public key of $NCC$
x	The private key of $NCC$
$SK$	Session key
t, $t^{\prime }$, $T_i$	Timestamp, current time, user i registration time

shows abbreviations used in Liu et al.’s scheme.

The computational costs of several protocols are shown in

Table 16. Storage cost (bits).

Protocol	Edge User Computational Cost (ms)	Total Computational Cost (ms)
3-Tier AKA	$T_{AE{S}_D}=0.243$	$2T_{AE{S}_E}+2T_{AE{S}_D}=0.920$
FogHA [44]	$8T_H+T_{SP}=2.568$	$21T_H+6T_{SP}=12.519$
Quantum-resistant handover authentication protocol [62]	$2T_{NTR{U}_E}+T_{NTR{U}_D}+7T_H=14.289$	$5T_{NTR{U}_E}+5T_{NTR{U}_D}+23T_H=48.461$
Liu et al.’s scheme [55]	$3T_{ECC}+T_{S{E}_E}+6T_H=16.540$	$6T_{ECC}+T_{S{E}_E}+T_{S{E}_D}+12T_H=33.106$

. Liu et al.’s scheme has the highest costs in the edge tier. Compared to other protocols, 3-Tier AKA has the highest efficiency.

5.3.2. Signaling Cost

In the context of a communication protocol involving an edge device, a 5G service provider, and a fog node, this section compares different signaling messages exchanged among these entities. The focus is on evaluating the efficiency and effectiveness of various protocols in terms of the number of signaling messages and overall performance. The findings are presented in

Table 17. Signaling cost (handover).

Protocol	Signaling Cost
3-Tier AKA	2
FogHA [44]	5
Zhang et al.’s Quantum-resistant handover authentication protocol [62]	4
Liu et al.’s scheme [55]	6

and Figure 19, which outline the characteristics of different protocols. The proposed protocol stands out for its efficiency. It involves only two signaling messages that are exchanged between the edge device, 5G service provider, and a fog node. These two messages serve the purposes of handover and key agreement. In contrast, another protocol proposed by Liu et al. [55] is highlighted as having the highest signaling cost, involving six signaling messages. This comparison suggests that Liu et al.’s scheme might result in higher communication overhead and potentially increased latency. Comparing the proposed protocol with two other schemes, namely, FogHA and quantum-resistant handover authentication protocols, the designed protocol has lower signaling costs. Specifically, FogHA [44] involves five signaling messages, and the quantum-resistant protocol [62] involves four. In this regard, the proposed protocol stands out as having the smallest signaling costs among the three. Consequently, the proposed 3-Tier AKA protocol is the optimal choice among the evaluated options. Its key advantage lies in its minimal signaling costs compared to other protocols.

5.3.3. Communication Cost

The communication cost of the proposed protocol when compared with other existing protocols is as follows:

• 3-Tier AKA: In lines 3 and 4 of Figure 9, there are a total of two AES encrypted messages transmitted.
• FogHA: The first fog access point initiates handover authentication communication by sending pre-negotiation information to the second fog access point in line 2 of Figure 16. Once this exchange is completed, the first fog access point proceeds to transmit the pre-negotiation temporary key to the mobile device after line 5 of Figure 16. Subsequently, in line 11 and line 15 of Figure 16, authentication messages are transmitted between the second fog access point and the mobile device. The total communication costs consist of five IDs, seven hash functions, one random number, and four timestamps.
• Quantum-resistant handover authentication protocol: The messages exchanged between entities primarily consist of hash function operations and NTRU encrypted messages. In lines 2, 4, 7, and 11 of Figure 17, there are total communication costs of five NTRU encrypted messages, nine hash function operations, and two timestamps.
• In Liu et al.’s scheme, the authentication message is initially sent from the user to a low-Earth-orbit satellite (LEOS). Subsequently, the LEOS appends its ID to the message and forwards it to the network control center (NCC). In lines 8 and 9 of Figure 18, the response message is sent back to the user. Lines 12 and 13 of Figure 18 involve the session key agreement process. The protocol includes two symmetric encryption operations, four elliptic curve point multiplication operations, four timestamps, three IDs, and four hash function operations transmitted in total.
• 3-Tier AKA only performs two AES encryption operations. Hence, the overall communication cost of 3-Tier AKA amounts to 256 bits, significantly lower than that of other protocols.

  Table 18. Total communication cost (bits) (handover).

  Protocol	Signaling Cost
  3-Tier AKA	$2L_{AES}=256$
  FogHA [44]	$5L_{ID}+7L_H+L_R+4L_{TS}=2688$
  Quantum-resistant handover authentication protocol [62]	$5L_{NTRU}+9L_H+2L_{TS}=3168$
  Liu et al.’s scheme [55]	$3L_{I{D}^{*}}+4L_{H^{*}}+4L_{EC{C}_p}+2L_{SyK}+4L_{T{S}^{*}}=2624$

  and Figure 20 show the total communication cost of our protocol and other related schemes during the handover authentication phase.

5.3.4. Storage Cost

In the proposed protocol, the edge device only needs to store its identity ${\delta }_{d}ID$, fog node identity ${\zeta }_{2}ID$, a token $O_{{\delta }_d}$, a secret key $K_{{\delta }_d}$, and a session key $K_{{\delta }_d{\zeta }_2}$. Less storage requirements mean more edge users can be accommodated. The edge user storage cost of 3-Tier AKA is 640 bits, which indicates that this protocol is more lightweight than the other three protocols. The edge device storage cost after the handover phase is shown in

Table 19. Storage cost (bits).

Protocol	Storage Cost (Bits)	5G/4G Network/Server (Bits)
3-Tier AKA	$2L_{ID}+2L_K+L_{AES}=640$	$L_K=128$
FogHA [44]	$4L_{ID}+6L_H+3L_{SP}=3200$	$2L_{ID}+2L_{SP}+2L_H=1536$
Zhang et al.’s Quantum-resistant handover authentication protocol [62]	$3L_{ID}+4L_H+2L_R+3L_{NTRU}=1952$	$L_{ID}+4L_{NTRU}+4L_H=1792$
Liu et al.’s scheme [55]	$L_{H^{*}}+L_K+2L_{I{D}^{*}}+L_{EC{C}_p}=736$	$L_{T{S}^{*}}+2L_{H^{*}}+3L_K+3L_{I{D}^{*}}+L_{EC{C}_p}=1280$

and Figure 21.

6. Security and Feature Analysis

Security and feature analysis refers to evaluating and examining the security aspects and functional features of a particular system, technology, or piece of software. This section presents the security and feature analysis of the proposed 3-Tier AKA protocol presented in Section 4.

6.1. Security Analysis

The security analysis focuses on identifying and evaluating potential security risks, threats, and vulnerabilities associated with the 3-Tier AKA. It involves assessing the effectiveness of security measures implemented within the protocol when facing common attacks, such as spoofing attacks, information disclosure, denial of service, and elevation of privilege. The aim is to identify any weaknesses or potential areas of exploitation that could compromise 3-Tier AKA’s security.

• Data integrity and tampering attack: Data integrity in authentication protocols ensures data remain uncorrupted and untampered, while tampering attacks involve unauthorized modification of message exchanges for malicious purposes, thereby preventing data tampering. The proposed protocol is designed to prevent message leakage. Each message sent between participants is encrypted. Once the attacker modifies the plaintext sent among the protocol participants, the receiver cannot decrypt it with the specified secret key. This modification will be discovered immediately. The following scenarios depict the potential consequences when a message undergoes modifications during Authentication and Key Agreement phase message transmission; see Figure 6. In step1, if the token $O_{{\delta }_d}$ is modified, the 5G service provider $u_v$ will not be able to retrieve the accurate secret key $K_{{\delta }_d}$ due to it being encrypted by $u_v$ master secret key $S_{u_v}$. If $proofMe$ is modified, $u_v$ cannot obtain the time stamp, compare it to the current one, or obtain the correct user ID. Moreover, it is highly improbable for an adversary to create a legitimate $O_{{\delta }_d}$ since its creation involves the utilization of $S_{u_v}$, which represents the master key of the 5G service provider. During step 2, it is important to note that the adversary does not know the edge user’s secret key $K_{{\delta }_d}$. Hence, any attempts to modify “${\delta }_{d}to{\zeta }_z$”, $K_{{\delta }_d{\zeta }_z}$, or $proo{f}_{{\delta }_d}$ would be unfeasible for the adversary. Similar to step 2, the adversary is unable to tamper with “${\delta }_{d}to{\zeta }_z$” or the timestamp in step 3. Suppose the adversary modifies the transmission message in the fourth step. In that case, the 5G service provider will be unable to retrieve any valid information due to using the master secret key. Then, the authentication fails. It is crucial to emphasize that safeguarding data integrity is not solely reliant on the authentication protocol but is a vital aspect to be considered across the entire system’s design and implementation.
• Spoofing: A security attack where an attacker pretends to be a specific edge device to deceive the fog node (the victim) into revealing the key information, or vice versa. In this protocol, if a malicious attacker M wants to pretend to be a legitimate user, M must calculate a self-authentication message $proofMe=E(T\Vert “{\delta }_{m}to{\zeta }_z$”, $K_{{\delta }_m})$ and a token $O_{{\delta }_m}=E(“{\delta }_{m}ID$” $\Vert K_{{\delta }_m},S_{u_v})$, where $K_{{\delta }_m}$ is a faked secret key for M and $S_{u_v}$ is the master key of the 5G service provider. The attacker can generate a fake $proofMe$ message, but not the token, because $S_{u_v}$ is known only by the 5G service provider. Therefore, the attacker will send a token that is recorded from previous sessions as a fake token instead. However, the 5G network will drop and close the session because the retrieved secret key in the received token does not match the attacker faked secret key for M. Therefore, the proposed protocol can prevent spoofing attacks.
• Man-in-the-middle attack (MitM): This is a cybersecurity attack where an attacker intercepts and potentially alters communications between two parties, allowing them to capture, manipulate, or eavesdrop on the exchanged information. In the proposed protocol, if a malicious actor labeled as M captures the message sent from an edge device to a 5G network and wants to obtain the data, the acquisition of the master key $S_{u_v}$ becomes imperative. This key serves as a mean to decrypt authentication messages and any additional messages intended for M. The absence of the master key renders M incapable of accessing any information. The response message sent from the 5G service provider to edge device ${\delta }_d$ is encrypted by the secret key of ${\delta }_d$. If M does not know the secret key, M is unable to access or alter the data. The session key encrypts messages transmitted between the edge device and the fog node $K_{{\delta }_d{\zeta }_z}$. For M to illicitly acquire the data after intercepting the information, it is imperative that M possesses the session key. The above scenarios illustrate that the designed protocol can resist MitM attacks.
• Replay attacks: These involve an attacker maliciously re-transmitting captured data without altering the content, causing security vulnerabilities and compromising system integrity and authenticity, without altering the data themselves. The proposed protocol is designed to guarantee the freshness of messages. For example, in the mutual authentication protocol between the edge device and fog node, the 5G service provider $u_v$ receives the self-authentication message $proofMe$ and the token $O_{{\delta }_d}$ from edge device ${\delta }_d$. Then, $u_v$ decrypts the received $proofMe$ using $K_{{\delta }_d}$, and verifies that T is within the current time skew. The message $M_{{\delta }_d{\zeta }_z}$ includes a text “${\delta }_{d}to{\zeta }_z$”. This text ensures that this message’s direction is from the edge device ${\delta }_d$ to fog node ${\zeta }_z$. The malevolent attacker is unable to dispatch the message $M_{{\delta }_d{\zeta }_z}$ to other fog nodes in order to execute a replay attack. Therefore, the proposed protocol can prevent replay attacks.
• Information disclosure: Also known as leakage or exposure, this involves unauthorized access to protocol information, potentially enabling attackers to launch further attacks like phishing emails or identity theft. The proposed protocol is designed to hide the identity of the participants, which includes the edge device, the 5G network, fog node, and the cloud server. Therefore, the attacker is unaware of who is exchanging the protocol messages. Also, all messages sent between entities are encrypted by Advanced Encryption Standard (AES) encryption technology. The adversary cannot guess the edge/fog/cloud secret keys $(K_{{\delta }_d},K_{{\zeta }_z},K_{{\gamma }_c})$ or 5G network master secret key $S_{u_v}$ to illegally obtain the user ID or session key.
• Denial of service (DoS): This is a security attack where an attacker overwhelms a system with traffic, requests, or data, disrupting its normal functioning. The most common scenario involves a system requiring high storage capacity, making it vulnerable to a DoS attack. The proposed protocol is designed to eliminate the need for the 5G service provider to remember/store the edge devices’ IDs or keys and remain stateless, eliminating the possibility of a DoS attack.
• Elevation of privilege (EOP): Here, an attacker gains access to a system. This attack aims to escalate their privileges from a low-level user, i.e., edge device’s account, to a higher level, i.e., 5G administrator role, allowing them to gain access to sensitive data such as the 5G network master secret key. This means a legitimate edge device could perform this attack by exploiting vulnerabilities in the 5G security system to gain administrative access to a system and reveal information about the 5G service provider $u_v$ master key. Therefore, the security level of the proposed protocol to protect against the EOP solely depends on the security level of the 5G network.

6.2. Feature Analysis

The following feature analysis examines the functional features and capabilities of 3-Tier AKA. The feature analysis aims to evaluate how well the protocol meets the desired requirements and compares its features with similar existing solutions or the industry standards.

• Hidden identities anonymous: A security mechanism designed to enable authentication while preserving the privacy and anonymity of the entities involved. It allows users to authenticate themselves without revealing their identities to the public. This protocol is useful in scenarios where privacy and anonymity are crucial, such as online transactions, communication platforms, and anonymous voting systems. The main objective of the hidden identities anonymous authentication protocol is to ensure that the authentication process does not disclose sensitive information about the users’ identities. In the proposed 3-Tier AKA, during the authentication phase, the message sent from the edge device ${\delta }_d$ to 5G service provider $u_v$ includes $proofMe=E(T\Vert “{\delta }_{d}to{\zeta }_z$”, $K_{{\delta }_d})$ and a token $O_{{\delta }_d}=E(“{\delta }_{d}ID$” $\Vert K_{{\delta }_d},S_{u_v})$. In this message, the identity of ${\delta }_d$ and target fog node ${\zeta }_z$ are encrypted by AES encryption technology. The malicious attacker M cannot acquire any ${\delta }_d$ or ${\zeta }_z$ identity information from the intercepted message. Based on this, the protocol delivers on hiding the identity feature.
• Mutual authentication: This is a bidirectional security mechanism that ensures trust and identity verification between two entities. It prevents unauthorized entities from posing as legitimate and mitigates man-in-the-middle attacks by requiring both to prove their identities. 3-Tier AKA has been designed to provide mutual authentication between the edge node and the 5G service provider, between the fog node and the 5G service provider, and between the edge device and the fog node. In Figure 6, line 3, the 5G service provider extracts the edge device key from the received token and decrypts the received proofME to check the value of T. If T matches, the current time clock is authenticated. In line 5, the edge device receives and decrypts $R_{{\delta }_d}$, if the retrieved T matches the sending T in line 1, then the edge device authenticates the 5G service provider. Similarly, in line 9, the 5G server provider authenticates the fog node, and in line 10, when the fog node decrypts $R_{{\zeta }_z}$ and finds the retrieved T matches the sending T in line 7, then the fog node authenticates the 5G service provider. In line 11, the fog node ${\zeta }_z$ retrieves the session key $K_{{\delta }_d{\zeta }_z}=D(R_{{\zeta }_z},K_{{\zeta }_z})$ to be used to decrypt and verify the message $M_{{\delta }_d{\zeta }_z}$ received in line 6. Again, the correct value of T and the text “${\delta }_{d}to{\zeta }_z$” is enough to authenticate the edge device. The edge device also authenticates the fog node when it decrypts the message $M_{{\zeta }_z{\delta }_d}$, as shown in line 12, and verifies the value of T and the text “${\zeta }_{z}to{\delta }_d$”.
• Lightweight authentication protocol: This is a security mechanism designed to verify the identity of IoT devices while minimizing computational overhead, considering resource constraints and device diversity in the environment. The following points outline the reasons why the protocol is considered lightweight. Storage resources in the proposed 3-Tier AKA: After the registration phase and authentication phase, the 5G service provider $u_v$ forgets every temporary key or secret key except its master key. Edge device ${\delta }_d$ and fog node ${\zeta }_z$ also forget the temporary secret key and only store the session key $K_{{\delta }_d{\zeta }_z}$ and secret key. In the computational resources and during the registration phase, ${\delta }_d$ only computes once when it retrieves the secret key $K_{{\delta }_d}$. Service $u_v$ only computes once for generating the secret key and the token. During the Authentication and Key Agreement phase, ${\delta }_d$ only needs to operate two AES encoding/decoding algorithms. $u_v$ takes three and ${\zeta }_z$ takes two encoding/decoding algorithms. Hence, the lightweight feature is achieved in the proposed protocol.
• Session key: This is a temporary cryptographic key used during a single communication session between two entities, ensuring secure and confidential data exchange through a key exchange or key establishment protocol. This protocol involves a series of cryptographic algorithms and techniques to securely generate and exchange the session key between the entities. After the Authentication and Key Agreement phase, edge device ${\delta }_d$, and fog node ${\zeta }_z$ will have the same session key $K_{{\delta }_d{\zeta }_z}$. In the Authentication and Key Agreement presented in Figure 6, line 5, ${\delta }_d$ receives a response message $R_{{\delta }_d}$ from service provider $u_v$. Then, ${\delta }_d$ retrieves $K_{{\delta }_d{\zeta }_z}$ by performing $D(R_{{\delta }_d},K_{{\delta }_d})$. In line 10, ${\zeta }_z$ obtains $K_{{\delta }_d{\zeta }_z}$ by using its secret key $K_{{\zeta }_z}$. Then, ${\delta }_d$ receives the message $M_{{\zeta }_z{\delta }_d}$ from ${\zeta }_z$, and computes its own ID and fog node $ID$, ${\zeta }_{z}ID$, by using $K_{{\delta }_d{\zeta }_z}$. If the session keys between ${\delta }_d$ and ${\zeta }_z$ are not the same, the secure communication channel will not be built.
• Scalability and compatibility of the system: The authentication protocol’s scalability requirements include user count, concurrent requests, network traffic volume, and system response time. It leverages 5G networks’ capabilities to handle large user requests efficiently. The protocol’s high degree of scalability ensures optimal processing without delays, and adding more edge users or fog nodes does not compromise system efficiency.

In the IoT environment, numerous users utilize a variety of devices that often originate from different manufacturers. In the designed protocol, edge users and fog nodes access the 5G network through the eSIM card. This approach ensures that, despite the diversity of devices used by edge users, they can rely on the eSIM card for seamless data transmission. Consequently, this significantly mitigates compatibility conflicts, data synchronization issues, and the requirement for additional middleware components to facilitate communication between different tiers. By leveraging the standardized eSIM technology, the protocol promotes enhanced compatibility and streamlined communication within the IoT ecosystem.

7. Conclusions and Future Work

This work proposed, developed, and evaluated a lightweight mutual authentication mechanism with handover function in an edge–fog–cloud environment, specifically focusing on integrating 5G communication technology. This work aimed to address the crucial challenges of securing communication between edge devices, fog nodes, and cloud services while accommodating the dynamic mobility of edge devices within the distributed architecture and leveraging the capabilities of 5G technology. Also, handovers involve switching a device’s connection from one access point to another. The authentication process during handovers introduces latency, which can affect real-time applications and services.

The proposed mutual authentication mechanism successfully achieved secure and efficient communication within the edge–fog–cloud environment. A mutual authentication protocol that verifies the identities and access permissions of all entities involved in the communication process is designed and developed throughout this work. Using AES cryptography and secure safe handover mechanisms, we were able to make transitions between fog nodes smooth during edge device mobility, ensuring that users could still access services and resources without interruption.

Moreover, this work enhanced the security in the edge–fog–cloud environment, mitigating the potential risks of impersonation attacks, eavesdropping, and unauthorized access. The incorporation of robust security protocols into our authentication mechanism provided a shield against potential threats, safeguarding sensitive data and ensuring privacy during communication.

The effectiveness and performance of the lightweight mutual authentication mechanism were validated through comprehensive evaluations. A comparative analysis was conducted, contrasting the proposed protocol with existing ones, including TLS 1.3, 5G-AKA, and various handover protocols. The results showcased notable benefits, including reduced computational overhead, minimal communication latency, and seamless compatibility with resource-constrained edge devices.

Further advancing the protocols introduced in this work represents a promising and valuable direction for future research. The authentication mechanism can be implemented practically in an edge–fog–cloud environment, and its performance will be thoroughly evaluated. Furthermore, the investigation of energy-efficient authentication strategies for resource-constrained edge devices aims to prolong battery life and improve overall energy efficiency in the edge–fog–cloud environment.

Additional security challenges can arise when using different operating systems deployed at the edge, fog, or cloud nodes. Hence, future directions shall study the impact of security within the three-tier architecture while utilizing heterogeneous deployment of operating systems.

It worth mentioning that the proposed protocol includes a specific requirement that each participating party must possess eSIM technology. While eSIM offers several advantages such as enhanced security, flexibility, and ease of remote provisioning, it is important to note that the current adoption rate of eSIM technology is not yet universal. This dependency on eSIM technology could be seen as a limitation, as it may restrict the immediate applicability and scalability of the protocol to environments and users who have already integrated eSIM capabilities into their devices. We recognize this as a potential shortcoming and are aware that it could impact the protocol’s broader adoption in the near term.