Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Resource Allocation Using Deep Deterministic Policy Gradient-Based Federated Learning for Multi-Access Edge Computing

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

The study focuses on utilizing the computational resources present in vehicles to enhance the performance of multi-access edge computing (MEC) systems. While vehicles are typically equipped with computational services for vehicle-centric Internet of Vehicles (IoV) applications, their resources can also be leveraged to reduce the workload on edge servers and improve task processing speed in MEC scenarios. Previous research efforts have overlooked the potential resource utilization of passing vehicles, which can be a valuable addition to MEC systems alongside parked cars. This study introduces an assisted MEC scenario where a base station (BS) with an edge server serves various devices, parked cars, and vehicular traffic. A cooperative approach using the Deep Deterministic Policy Gradient (DDPG) based Federated Learning method is proposed to optimize resource allocation and job offloading. This method enables the transfer of device operations from devices to the BS or from the BS to vehicles based on specific requirements. The proposed system also considers the duration for which a vehicle can provide job offloading services within the range of the BS before leaving. The objective of the DDPG-FL method is to minimize the overall priority-weighted task computation time. Through simulation results and a comparison with three other schemes, the study demonstrates the superiority of their proposed method in seven different scenarios. The findings highlight the potential of incorporating vehicular resources in MEC systems, showcasing improved task processing efficiency and overall system performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Fan, W., Liu, J., Hua, M., Wu, F., Liu, Y.A.: Joint task offloading and resource allocation for multi-access edge computing assisted by parked and moving vehicles. IEEE Trans. Veh. Technol. 71(5), 5314–5330 (2022)

    Article  Google Scholar 

  2. Chen, J., Wang, Q., Cheng, H.H., Peng, W., Xu, W.: A review of vision-based traffic semantic understanding in ITSs. IEEE Trans. Intell. Transp. Syst. 23(11), 19954–19979 (2022)

    Article  Google Scholar 

  3. Chen, J., Xu, M., Xu, W., Li, D., Peng, W., … Xu, H.: A flow feedback traffic prediction based on visual quantified features. IEEE Trans. Intell. Transp. Syst., 24(9), 10067–10075 (2023)

  4. Fan, W., Han, J., Yao, L., Wu, F., Liu, Y.A.: Latency-energy optimization for joint WiFi and cellular offloading in mobile edge computing networks. Comput. Netw. 181, 107570 (2020)

    Article  Google Scholar 

  5. Chen, J., Wang, Q., Peng, W., Xu, H., Li, X.,… Xu, W, Disparity-based multiscale fusion network for transportation detection. IEEE Trans. Intell. Transp. Syst. 23(10), 18855–18863 (2022)

  6. Yin, Y., Guo, Y., Su, Q., Wang, Z.: Task allocation of multiple unmanned aerial vehicles based on deep transfer reinforcement learning. Drones 6(8), 215 (2022)

    Article  Google Scholar 

  7. Fang, Z., Wang, J., Liang, J., Yan, Y., Pi, D., Zhang, H.,… Yin, G, Authority allocation strategy for shared steering control considering human-machine mutual trust level. IEEE Trans. Intell. Veh. 9(1), 2002–2015 (2024)

  8. Ma, B., Liu, Z., Dang, Q., Zhao, W., Wang, J., Cheng, Y.,… Yuan, Z, Deep reinforcement learning of UAV tracking control under wind disturbances environments. IEEE Trans. Instrum. Meas. 72, 1–13 (2023)

  9. Zhang, J., Ren, J., Cui, Y., Fu, D., Cong, J.: Multi-USV task planning method based on improved deep reinforcement learning. IEEE Internet Things J. 11(10), 18549–185672024 (2024)

    Article  Google Scholar 

  10. Sun, G., Sheng, L., Luo, L., Yu, H.: Game theoretic approach for multipriority data transmission in 5G vehicular networks. IEEE Trans. Intell. Transp. Syst. 23(12), 24672–246852022 (2022)

    Article  Google Scholar 

  11. Sun, G., Song, L., Yu, H., Chang, V., Du, X., … Guizani, M, V2V routing in a VANET Based on the autoregressive integrated moving average model. IEEE Trans. Veh. Technol. 68(1), 908–922 (2019)

  12. Sun, G., Zhang, Y., Liao, D., Yu, H., Du, X., … Guizani, M, Bus-trajectory-based street-centric routing for message delivery in urban vehicular ad hoc networks. IEEE Trans. Veh. Technol. 67(8), 7550–7563 (2018)

  13. Xu, X., Liu, W., Yu, L.: Trajectory prediction for heterogeneous traffic-agents using knowledge correction data-driven model. Inf. Sci. 608, 375–391 (2022)

    Article  Google Scholar 

  14. Zhang, X., Wang, Y., Yuan, X., Shen, Y., Lu, Z.: Adaptive dynamic surface control with disturbance observers for battery/supercapacitor-based hybrid energy sources in electric vehicles. IEEE Trans. Transp. Electrification 9(4), 5165–5181 (2023)

    Article  Google Scholar 

  15. Zhang, X., Wang, Z., Lu, Z.: Multi-objective load dispatch for microgrid with electric vehicles using modified gravitational search and particle swarm optimization algorithm. Appl. Energy. 306, 118018 (2022)

    Article  Google Scholar 

  16. Zhang, X., Lu, Z., Yuan, X., Wang, Y., Shen, X.: L2-Gain adaptive robust control for hybrid energy storage system in electric vehicles. IEEE Trans. Power Electron. 36(6), 7319–7332 (2021)

    Article  Google Scholar 

  17. Li, Q., Lin, H., Tan, X., Du, S.: Consensus for multiagent-based supply chain systems under switching topology and uncertain demands. IEEE Trans. Syst. Man. Cybernetics: Syst. 50(12), 4905–4918 (2020)

    Article  Google Scholar 

  18. Yang, H., Li, Z., Qi, Y.: Predicting Traffic Propagation Flow in Urban Road Network with Multi-Graph Convolutional Network. Complex & Intelligent Systems (2023)

    Google Scholar 

  19. Yang, H., Zhang, X., Li, Z., Cui, J.: Region-level traffic prediction based on temporal multi-spatial dependence graph Convolutional Network from GPS Data. Remote Sens. 14(2), 303 (2022)

    Article  Google Scholar 

  20. Fu, Y., Li, C., Yu, F.R., Luan, T.H., Zhao, P.: An incentive mechanism of incorporating supervision game for federated learning in autonomous driving. IEEE Trans. Intell. Transp. Syst. 24(12), 14800–14812 (2023)

    Article  Google Scholar 

  21. Xie, Y., Wang, X., Shen, Z., Sheng, Y., Wu, G.: A two-stage estimation of distribution algorithm with heuristics for energy-aware cloud workflow scheduling. IEEE Trans. Serv. Comput. 16(6), 4183–4197 (2023)

    Article  Google Scholar 

  22. Ding, C., Li, C., Xiong, Z., Li, Z., Liang, Q.: Intelligent identification of moving trajectory of autonomous vehicle based on friction nano-generator. IEEE Trans. Intell. Transp. Syst. 25(3), 3090–3097 (2024)

    Article  Google Scholar 

  23. Sun, R., Dai, Y., Cheng, Q.: An adaptive weighting strategy for multisensor integrated navigation in urban areas. IEEE Internet Things J. 10(14), 12777–12786 (2023)

    Article  Google Scholar 

  24. Wang, Y., Sun, R., Cheng, Q., Ochieng, W.Y.: Measurement quality control aided multisensor system for improved vehicle navigation in urban areas. IEEE Trans. Industr. Electron. 71(6), 6407–6417 (2024)

    Article  Google Scholar 

  25. Xiao, Z., Shu, J., Jiang, H., Min, G., Chen, H.,… Han, Z, Perception task offloading with collaborative computation for autonomous driving. IEEE J. Sel. Areas Commun. 41(2), 457–473 (2023)

  26. Dai, X., Xiao, Z., Jiang, H., Lui, J.C.S.: UAV-assisted task offloading in vehicular edge computing networks. IEEE Trans. Mob. Comput. 23(4), 2520–25342024 (2024)

    Article  Google Scholar 

  27. Dai, X., Xiao, Z., Jiang, H., Chen, H., Min, G., Dustdar, S.,… Cao, J, A learning-based approach for vehicle-to-vehicle computation offloading. IEEE Internet of Things J. 10(8), 7244–7258 (2023)

  28. Cao, B., Sun, Z., Zhang, J., Gu, Y.: Resource allocation in 5G IoV architecture based on SDN and fog-cloud computing. IEEE Trans. Intell. Transp. Syst. 22(6), 3832–3840 (2021)

    Article  Google Scholar 

  29. Jiang, H., Dai, X., Xiao, Z., Iyengar, A.K.: Joint task offloading and resource allocation for energy-constrained mobile edge computing. IEEE Transactions on Mobile Computing (2022)

  30. Qu, Z., Liu, X., Zheng, M.: Temporal-spatial quantum graph convolutional neural network based on Schrödinger approach for traffic congestion prediction. IEEE Transactions on Intelligent Transportation Systems (2022)

  31. Luo, J., Wang, G., Li, G., Pesce, G.: Transport infrastructure connectivity and conflict resolution: a machine learning analysis. Neural Comput. Appl. 34(9), 6585–6601 (2022)

    Article  Google Scholar 

  32. Zhao, X., Fang, Y., Min, H., Wu, X., Wang, W.,… Teixeira, R, Potential sources of sensor data anomalies for autonomous vehicles: An overview from road vehicle safety perspective. Expert Syst. Appl. 236 (2024)

  33. Min, H., Lei, X., Wu, X., Fang, Y., Chen, S., Wang, W.,… Zhao, X, Toward interpretable anomaly detection for autonomous vehicles with denoising variational transformer. Eng. Appl. Artif. Intell. 129 (2024)

  34. Xiao, Z., Shu, J., Jiang, H., Min, G., Chen, H.,… Han, Z, Overcoming occlusions: perception task-oriented information sharing in connected and autonomous vehicles. IEEE Net. 37(4), 224–229 (2023)

  35. Sun, G., Zhang, Y., Yu, H., Du, X., Guizani, M.: Intersection fog-based distributed routing for V2V communication in urban vehicular Ad Hoc Networks. IEEE Trans. Intell. Transp. Syst. 21(6), 2409–2426 (2020)

    Article  Google Scholar 

  36. Sun, G., Xu, Z., Yu, H., Chang, V.: Dynamic network function provisioning to enable network in box for industrial applications. IEEE Trans. Industr. Inf. 17(10), 7155–7164 (2021)

    Article  Google Scholar 

  37. Peng, Y., Zhao, Y., Hu, J.: On the role of community structure in evolution of opinion formation: a new bounded confidence opinion dynamics. Inf. Sci. 621, 672–690 (2023)

    Article  Google Scholar 

  38. Dong, J., Hu, J., Zhao, Y., Peng, Y.: Opinion formation analysis for expressed and private opinions (EPOs) models: Reasoning private opinions from behaviors in group decision-making systems. Expert Syst. Appl. 236 (2024)

  39. Xuemin, Z., Ying, R., Zenggang, X., Haitao, D., Fang, X.,… Yuan, L, Resource-constrained and socially selfish-based incentive algorithm for socially aware networks. J. Signal Process. Syst. 95(12), 1439–1453 (2023)

  40. Zhang, X., Deng, H., Xiong, Z., Liu, Y., Rao, Y., Lyu, Y.,… Li, Y, Secure routing strategy based on attribute-based trust access control in social-aware networks. J. Signal Process. Syst. 96(2), 153–168 (2024)

  41. Liao, Q., Chai, H., Han, H., Zhang, X., Wang, X., Xia, W.,… Ding, Y, An integrated multi-task model for fake news detection. IEEE Trans. Knowl. Data Eng. 34(11), 5154–5165 (2022)

  42. Mou, J., Gao, K., Duan, P., Li, J., Garg, A.,… Sharma, R, A machine learning approach for energy-efficient intelligent transportation scheduling problem in a real-world dynamic circumstances. IEEE Trans. Intell. Transp. Syst. 24(12), 15527–15539 (2023)

  43. Lyu, T., Xu, H., Zhang, L., Han, Z.: Source selection and resource allocation in wireless-powered relay networks: an adaptive dynamic programming-based approach. IEEE Internet Things J. 11(5), 8973–8988 (2024)

    Article  Google Scholar 

  44. Xiao, Y., Konak, A.: The heterogeneous green vehicle routing and scheduling problem with time-varying traffic congestion. Transp. Res. E: Logist. Transp. Rev. 88, 146–166 (2016)

    Article  Google Scholar 

  45. Yin, L., Zhuang, M., Jia, J., Wang, H.: Energy saving in flow-shop scheduling management: an improved multiobjective model based on Grey Wolf Optimization Algorithm. Mathematical Problems in Engineering (2020)

  46. Deng, Z.W., Zhao, Y.Q., Wang, B.H., Gao, W., Kong, X.: A preview driver model based on sliding-mode and fuzzy control for articulated heavy vehicle. Meccanica 57(8), 1853–1878 (2022)

    Article  MathSciNet  Google Scholar 

  47. Gong, Q., Li, J., Jiang, Z., Wang, Y.: A hierarchical integration scheduling method for flexible job shop with green lot splitting. Eng. Appl. Artif. Intell. 129 (2024)

  48. Xu, Y., Wang, E., Yang, Y., Chang, Y.: A unified collaborative representation learning for neural-network based recommender systems. IEEE Trans. Knowl. Data Eng. 34(11), 5126–5139 (2022)

    Article  Google Scholar 

  49. Zhao, J., Song, D., Zhu, B., Sun, Z., Han, J.,… Sun, Y, A human-like trajectory planning method on a curve based on the driver preview mechanism. IEEE Trans. Intell. Transp. Syst. 24(11), 11682–11698 (2023)

  50. Zhu, B., Sun, Y., Zhao, J., Han, J., Zhang, P.,… Fan, T, A critical scenario search method for intelligent vehicle testing based on the social cognitive optimization algorithm. IEEE Trans. Intell. Transp. Syst. 24(8), 7974–7986 (2023)

  51. Chen, J., Xu, M., Xu, W., Li, D., Peng, W.,... Xu, H, A flow feedback traffic prediction based on visual quantified features. IEEE Trans. Intell. Transp. Syst. 24(9), 10067–10075 (2023)

  52. Liu, Y., Fang, Z., Cheung, M.H., Cai, W., Huang, J.: Mechanism design for blockchain storage sustainability. IEEE Commun. Mag. 61(8), 102–107 (2023)

    Article  Google Scholar 

  53. Yao, Y., Zhao, B., Zhao, J., Shu, F., Wu, Y.,… Cheng, X, Anti-jamming technique for IRS aided JRC system in mobile vehicular networks. IEEE Transactions on Intelligent Transportation Systems (2024)

  54. W., Z., Y., S., Y., Z., Q., L., Y., N., T., S.,… L., P. Limited sensing and deep data mining: a new exploration of developing city-wide parking guidance systems. IEEE Intell. Transp. Syst. Mag. 14(1), 198–215 (2022)

  55. Tian, J., Wang, B., Guo, R., Wang, Z., Cao, K.,... Wang, X, Adversarial attacks and defenses for deep-learning-based unmanned aerial vehicles. IEEE Internet Things J. 9(22), 22399–22409 (2022)

  56. Yang, M., Han, W., Song, Y., Wang, Y., Yang, S.: Data-model fusion driven intelligent rapid response design of underwater gliders. Adv. Eng. Inform. 61, 102569 (2024)

    Article  Google Scholar 

  57. Rong, Y., Xu, Z., Liu, J., Liu, H., Ding, J., Liu, X.,… Gao, J, Du-Bus: A realtime bus waiting time estimation system based on multi-source data. IEEE Trans. Intell. Transp. Syst. 23(12), 24524–24539 (2022)

  58. Zheng, W., Lu, S., Cai, Z., Wang, R., Wang, L.,... Yin, L, PAL-BERT: An improved question answering model. Comput. Model. Eng. Sci. 139(3), 2729–2745 (2024)

  59. Liu, X., Wang, S., Lu, S., Yin, Z., Li, X., Yin, L.,... Zheng, W, Adapting feature selection algorithms for the classification of chinese texts. Systems 11(9), 483 (2023)

  60. Liu, X., Zhou, G., Kong, M., Yin, Z., Li, X., Yin, L.,… Zheng, W, Developing multi-labelled corpus of twitter short texts: a semi-automatic method. Systems 11(8) (2023)

  61. Cao, B., Li, Z., Liu, X., Lv, Z., He, H.: Mobility-aware multiobjective task offloading for vehicular edge computing in digital twin environment. IEEE J. Sel. Areas Commun. 41(10), 3046–3055 (2023)

    Article  Google Scholar 

  62. Cao, B., Zhang, J., Liu, X., Sun, Z., Cao, W., Nowak, R. M.,… Lv, Z, Edge–cloud resource scheduling in space–air–ground-integrated networks for internet of vehicles. IEEE Internet Things J. 9(8), 5765–5772 (2022)

  63. Li, T., Braud, T., Li, Y., Hui, P.: Lifecycle-aware online video caching. IEEE Trans. Mob. Comput. 20(8), 2624–26362021 (2021)

    Article  Google Scholar 

  64. Song, F., Liu, Y., Shen, D., Li, L., Tan, J.: Learning control for motion coordination in water scanners: toward gain adaptation. IEEE Trans. Industr. Electron. 69(12), 13428–13438 (2022)

    Article  Google Scholar 

  65. Manavi, M., Zhang, Y., Chen, G.: Resource allocation in cloud computing using genetic algorithm and neural network. In 2023 IEEE 8th International Conference on Smart Cloud (SmartCloud) (pp. 25–32). IEEE (September 2023)

  66. Alipour, P.: The BEM and DRBEM schemes for the numerical solution of the two-dimensional time-fractional diffusion-wave equations. arXiv preprint arXiv:2305.12117 (2023)

  67. Mokari, H., Firouzmand, E., Sharifi, I., Doustmohammadi, A.: Deception attack detection and resilient control in platoon of smart vehicles. In 2022 30th International Conference on Electrical Engineering (ICEE) (pp. 29–35) (May 2022)

  68. Heydarpoor, F., Karbassi, S.M., Bidabadi, N., Ebadi, M.J.: Solving multi-objective functions for cancer treatment by using Metaheuristic algorithms. Int. J. Comb. Optim. Probl. Inf., 11(3) (2020)

  69. Yang, H., Wang, Z., Song, K.: A new hybrid grey wolf optimizer-feature weighted-multiple kernel-support vector regression technique to predict TBM performance. Engineering with Computers, pp. 1–17 (2020)

Download references

Funding

No funding was obtained for this study.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Huazhong University of Science and Technology, Hubei, Wuhan, 430074, China

    Zheyu Zhou

  2. Computer Science & Artificial Intelligence Laboratory, Guangdong Institute Of Advanced Technology Chinese Academy of Sciences, Shenzheng, 518055, Guangdong, China

    Qi Wang

  3. Ulster College at Shaanxi University of Science & Technology, Xi’an, Shaanxi, 710021, China

    Jizhou Li

  4. School of Energy and Power Engineering, North University of China, Taiyuan, 030051, Shanxi, China

    Ziyuan Li

Authors
  1. Zheyu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jizhou Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ziyuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Zheyu Zhou: Conceptualization, Methodology, Formal analysis, Supervision, Writing - original draft, Writing - review & editing. Qi Wang: Investigation, Data Curation, Validation, Resources, Writing - review & editing. Jizhou Li: Writing - original draft, Writing - review & editing. Ziyuan Li: Data Curation, Validation, Resources, Writing - review & editing.

Corresponding author

Correspondence to Qi Wang.

Ethics declarations

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Z., Wang, Q., Li, J. et al. Resource Allocation Using Deep Deterministic Policy Gradient-Based Federated Learning for Multi-Access Edge Computing. J Grid Computing 22, 59 (2024). https://doi.org/10.1007/s10723-024-09774-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-024-09774-2

Keywords

Search

Navigation