Split Federated Learning Empowered Vehicular Edge Intelligence: Adaptive Parellel Design and Future Directions

VEI utilizes the computing and communication resources of vehicles, combined with AI technologies. VEI relies on the effective utilization of extensive data gathered from numerous vehicles for model training. We can devide the implementation of VEI into four groups including centralized machine learning (CL), distributed collaborative FL, SL and SFL, to achieve data processing and decision-making close to the data source. The details of these four schemes are shown in Fig. 1.

CL aggregates training data at centralized locations, such as cloud data centers. However, the transmission of vast vehicular data to these centers not only strains network bandwidth but also exacerbates latency issues. Additionally, vehicular data often encompasses sensitive information, including personal data related to user information (e.g., license plate numbers, facial features, and vehicle details). Thus, the imperative to retain data on local devices emerges to safeguard user privacy.

Distributed collaborative learning emerges as a promising technical strategy under exploration by the research community to tackle these challenges of CL. Generally, distributed collaborative learning entails the joint training of a global model through collaboration, without direct access to the decentralized raw data. This approach holds significant appeal for applications seeking to leverage the wealth of data generated within a distributed IoT environment. Notably, FL and SL stand out as two representative and emerging methods within the realm of distributed collaborative learning.

FL allows multiple data owners to work together to train a shared AI model without revealing their individual data. In FL, each vehicle independently trains a local AI model using its own data, which is then aggregated with other models by a central server to form a global model, thereby preserving data privacy by keeping data on local vehicles. It’s a promising solution for dealing with data challenges in IoV. However, FL requires each vehicle to have enough resources for training AI models, which might be difficult for resource-limited vehicles, especially for complex models like deep neural networks.

SL is another distributed collaborative learning approach, the whole model is partitioned to be collaboratively trained at the vehicles and RSU. The split learning operated in three main steps. Initially, the vehicle downloads the vehicle-side model and performs forward propagation to transmits the processed data to the RSU. Subsequently, the RSU conducts forward and backward propagation of RSU-side model, then updating the RSU-side model and broadcasting the gradient associated with the cut layer back to vehicles for vehicle-side model update. Next, the updated device-side model is transferred to the next vehicle to repeat the above process until all the devices are trained. SL allows vehicles to offload part of the model training task to a RSU thus making it possible to leverage flexible resource management in computing for supporting model training. Therefore, SL may greatly facilitate the resource aspect of ubiquitous intelligence in IoV. However, the sequential vehicle-RSU collaboration in SL limits its capability of involving the IoV big data dispersed across a large number of vehicles for model training.

Combining the advantages of SL and FL, SFL can not only allows vehicles to offload part of the model training task to a RSU, but also parallel training. SFL mainly constraints three steps. Firstly, the vehicle downloads the vehicle-side model and execute forward propagation to upload the smashed data to the RSU. Secondly, the RSU perform the forward and backward propagation with received smashed data, and then broadcasts the gradients of smashed data. Finally, the updated device-side model is upload to RSU for aggregation.

In this subsection, we focus on the challenges faced by distributed machine learning in implementing VEI from different layers.

We consider a general vehicular network that includes one RSU and a set of $N$ vehicles. The data set of the vehicle $n$ is denoted as $\mathcal{D}_{n}$, and $\left|\mathcal{D}_{n}\right|$ is the number of training data samples of vehicle $n$.

The objective is to collaboratively train a global AI model that minimizes the global loss function based on the global dataset collected from all vehicles.

Where $L_{n}(\boldsymbol{\omega})$ denotes the local loss function of vehicle $n$.

The full model of vehicle $n$ in the $t$-th round $\boldsymbol{\omega}_{t}^{n,\epsilon}$ includes two non overlapping sub-models with $\epsilon$-th cut layer, represent as vehicle-side model $\boldsymbol{\omega}^{V,\epsilon}_{t}$ and RSU-side model $\boldsymbol{\omega}^{S,\epsilon}_{t}$, it can be denoted by $\boldsymbol{\omega}_{t}^{n,\epsilon}=\{\boldsymbol{\omega}^{V,\epsilon}_{t};\boldsymbol{\omega}^{S,\epsilon}_{t}\},$ and the global model update principle is as follows:

The workflow of ASFL is shown in Fig. 3. Firstly, the RSU chooses different cut layers and distributes different vehicle-side model to different vehicles according to cut layer selection strategy. Secondly, the vehicles train vehicle-side model they received over the local datasets parallel, and then send the corresponding smashed data, to the RSU. Thirdly, the RSU is supposed to have sufficient resources and can provide powerful computing capability, such that it sequentially performs forward propagation to the RSU-side model with the received smashed data to calculate the loss function respectively, and then broadcasts the gradients of smashed data. Finally, vehicles update their vehicle-side model and upload to RSU for aggregation and global whole model updating.

Because vehicles continue to move on the road and wireless transmission channel environment are unstable. Moreover, different sensors and hardwares among vehicles result in significant differences in data and computation capacities. To reduce convergence time and enhance learning accuracy, we propose a cut layer selection strategy based on data transmission rate.

We assume that all vehicles remain same time within the RSU communication range. Therefore, we only consider the transmission rates of different vehicles to choose the cut layers. As shown in Fig. 5(5(a)), the communication load of SL is significant, as it not only requires the transmission of some vehicle models but also the transmission of intermediate messages for model training. So when the vehicle’s tranmission rate is higher, we can choose a smaller split layer to reduce communication load achieving a better communication latency. We using Resnet18 as whole training model, the model has a total of 9 split points showing in the Fig. 4. The cut layer selection strategy is shown in the following:

where $r_{n}$ is the wireless transmission speed of vehicle $n$ in the $t$-th round, $\bar{R_{1}}\leq\bar{R_{2}}\leq\bar{R_{3}}\leq\bar{R_{4}}$ are the constraint speed.

The advantage of AI lies in using a large amount of local device data for training. However, in SFL, while the system can access massive data from numerous devices, these data are commonly non-IID, resulting in limitations in the learning and generalization ability of the system model. With the development of Artificial Intelligence Generated Content (AIGC) technology, we consider using it to assist in generating data to mitigate the impact of non-IID data distribution on the model, thereby improving the performance of the system. The use of data generation for training raises three key issues. Firstly, how to evaluate or measure the effectiveness of data. Secondly, considering the existence of invalid or redundant data in large-scale training datasets, how to select data to reduce communication and computational burden and avoid resource waste. Thirdly, how to balance data generation and model training performance in mobile scenarios of vehicle networks.

In SFL, the global model is divided into non-overlapping vehicle(user)-side models and RSU-side models through cut layers. The latter the cut layer is chosen, the smaller the size of the smashed data. As vehicle speed increases, the channel stability between vehicles and RSU weakens, thus selecting cut layers latter to reduce communication load. However, with the acceleration of vehicle speed, the time spent by vehicles within the RSU communication range decreases. Therefore, selecting cut layers former reduces vehicle-side model computation time. Thus, designing a cut selection strategy to balance vehicle communication and computational resources, as well as time-energy overload, to achieve minimal overall training time is a significant consideration.

Splitting strategies need to consider not only the balance between computation and communication but also the balance between privacy protection and cost. The latter the split layer, the greater the computational load on vehicles, the smaller the communication load, and the better the privacy of the smashed data (the output smashed data will be more blurred). Therefore, it is necessary to consider how to balance communication overhead and privacy for vehicles with different speeds and capabilities.

As Transformer, AIGC, and LLM technologies advance, an increasing number of intelligent vehicular networking services rely on their support. However, directly deploying large-scale models for training on vehicles is impractical. Firstly, vehicles lack sufficient computing resources to support such tasks. Secondly, running large models on vehicles consumes excessive energy, thereby impacting vehicle payload capacities. Therefore, employing the concept of distributed learning, known as split learning, to decompose Transformer architecture models into what is termed split inference, has become a noteworthy research direction. This approach can be applied in vehicular networking environments assisted by AIGC and LLM to reduce demands on vehicle-side resources while maintaining system performance and efficiency.

Distinguishing split inference from split learning is crucial. In SL, the outputs of the cut layer (smashed data) are shared during forward propagation, while only the gradients from the smashed data are transmitted back to the vehicle during backpropagation. In contrast, split inference involves sending the outputs of the cut layer to the server without requiring backpropagation.

When discussing distributed learning frameworks, it is essential to use wireless networks to transmit a large amount of intermediate training data. In this context, SFL creates a communication burden during the training process. Considering a large number of vehicles involved in each training epoch, it is crucial to allocate communication resources wisely and effectively to accelerate the implementation of SFL. Multi-objective optimization can be comprehensively explored by combining variables such as delay, energy consumption, convergence time, and learning accuracy, thereby promoting the rational resource allocation. In addition, designing incentive mechanisms for resource allocation is also a means of promoting rational resource allocation. When formulating incentive plans, not only must factors such as CPU frequency, spectrum, and energy costs be considered, but the impact of transmission interference on other users can also be considered as a cost factor. Advocacy rooted in various frameworks of game theory, contract theory, and auction theory can be proven useful in ASFL incentive design within vehicular network.

In the vehicular network, the number of vehicles within the RSU communication range varies at each time slot. In addition to designing optimization algorithms or game algorithms to minimize system overhead or maximize model performance, it is also necessary to consider the scalability of the system. When many vehicles join the network, a reasonable and high-efficient parallel design can help to avoid a linear or exponential increase in system latency, ensuring strong scalability of the system.