Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Reinforcement learning based on routing with infrastructure nodes for data dissemination in vehicular networks (RRIN)

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Vehicular-Ad hoc Networks are extremely important due to the potential for improving road safety, traffic monitoring, and in-vehicle infotainment services. A novel Q-learning-based routing protocol named Reinforcement learning-based Routing with Infrastructure Node Data Dissemination in Vehicular Network (RRIN) is proposed to efficiently address such a dynamic network. RRIN is a routing protocol that aims to achieve low end-to-end communication latency and a high data delivery ratio. To meet the objectives, we proposed two Q-routing functions for Road Model Segment Selection (RMSS) and Intermediate Vehicle Selection (IVS). The network environment is separated into road model segments, and Road Side Units at each road junction to assist nodes in data dissemination was deployed. The exploration feature of the Q-learning algorithm allowed the vehicles to randomly explore and interact with the dynamic environment in the vehicular network. Our findings show that the proposed RRIN routing protocol is highly beneficial compared to other efficient routing protocols with high packet delivery, high throughput, and low end-to-end communication latency. Due to the exploration and exploitation phases of Q-learning, the proposed RRIN routing protocol enhances the reliability and the efficiency of the vehicular network in terms of high throughput, low communication latency, and low packet and high packet delivery ratio. For RMSS, the shortest distance and higher connectivity distribution are considered parameters; whereas, the parameters for IVS are vehicle speed difference, link reliability, moving direction, and buffer size.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The data can be available on demand.

Code availability

It can be available on demand.

References

  1. Naqvi, R., Arsalan, M., Batchuluun, G., Yoon, H., & Park, K. (2018). Deep learning-based gaze detection system for automobile drivers using a NIR camera sensor. Sensors, 18(2), 456. https://doi.org/10.3390/s18020456

    Article  Google Scholar 

  2. Naqvi, R. A., Arsalan, M., Rehman, A., Rehman, A. U., Loh, W.-K., & Paul, A. (2020). Deep learning-based drivers emotion classification system in time series data for remote applications. Remote Sensing, 12(3), 587. https://doi.org/10.3390/rs12030587

    Article  Google Scholar 

  3. Deeba, F., Dharejo, F. A., Zawish, M., Memon, F. H., Dev, K., Naqvi, R. A., Zhou, Y., & Yi, D. (2021). A novel image dehazing framework for robust vision‐based intelligent systems. International Journal of Intelligent Systems. https://doi.org/10.1002/int.22627

    Article  Google Scholar 

  4. Senouci, O., Harous, S., & Aliouat, Z. (2020). Survey on vehicular ad hoc networks clustering algorithms: Overview, taxonomy, challenges, and open research issues. International Journal of Communication Systems, 33(11), e4402. https://doi.org/10.1002/dac.4402

    Article  Google Scholar 

  5. Laberteaux, H. H. K. (2010). VANET Vehicular Applications and Inter-Networking Technologies. Wiley T. Retrieved from https://ieeexplore.ieee.org/servlet/opac?bknumber=8039897

  6. Dias, J., Rodrigues, J., Soares, V., Caldeira, J., Korotaev, V., & Proença, M. (2020). Network management and monitoring solutions for vehicular networks: A survey. Electronics, 9(5), 853. https://doi.org/10.3390/electronics9050853

    Article  Google Scholar 

  7. Ali, A., & Shah, S. A. A. (2019). VANET Clustering using whale optimization algorithm. In 2019 International symposium on recent advances in electrical engineering (RAEE) (pp. 1–5). IEEE. https://doi.org/10.1109/RAEE.2019.8886982

  8. Aadil, F., Bajwa, K. B., Khan, S., Chaudary, N. M., & Akram, A. (2016). CACONET: Ant colony optimization (ACO) based clustering algorithm for VANET. PLoS ONE, 11(5), e0154080. https://doi.org/10.1371/journal.pone.0154080

    Article  Google Scholar 

  9. Fahad, M., Aadil, F., Rehman, Z., Khan, S., Shah, P. A., Muhammad, K., Lloret, J., Wang, H., Lee, J. W., & Mehmood, I. (2018). Grey wolf optimization based clustering algorithm for vehicular ad-hoc networks. Computers & Electrical Engineering, 70, 853–870. https://doi.org/10.1016/j.compeleceng.2018.01.002

    Article  Google Scholar 

  10. An adaptive fault tolerance strategy for cloud storage. (2016). KSII Transactions on Internet and Information Systems. https://doi.org/10.3837/tiis.2016.08.005

  11. Eze, E. C., Zhang, S.-J., Liu, E.-J., & Eze, J. C. (2016). Advances in vehicular ad-hoc networks (VANETs): Challenges and road-map for future development. International Journal of Automation and Computing, 13(1), 1–18. https://doi.org/10.1007/s11633-015-0913-y

    Article  Google Scholar 

  12. Shinkawa, T., Terauchi, T., Kitani, T., Shibata, N., Yasumoto, K., Ito, M., & Higashino, T. (2006). A Technique for Information Sharing using Inter-Vehicle Communication with Message Ferrying. In 7th International conference on mobile data management (MDM’06) (pp. 130–130). IEEE. https://doi.org/10.1109/MDM.2006.23

  13. Lin, D., Kang, J., Squicciarini, A., Wu, Y., Gurung, S., & Tonguz, O. (2017). MoZo: A moving zone based routing protocol using pure V2V communication in VANETs. IEEE Transactions on Mobile Computing, 16(5), 1357–1370. https://doi.org/10.1109/TMC.2016.2592915

    Article  Google Scholar 

  14. Alsharif, N., & Shen, X. (2017). $i$CAR-II: Infrastructure-based connectivity aware routing in vehicular networks. IEEE Transactions on Vehicular Technology, 66(5), 4231–4244. https://doi.org/10.1109/TVT.2016.2600481

    Article  Google Scholar 

  15. Bilal, S., Madani, S., & Khan, I. (2011). Enhanced junction selection mechanism for routing protocol in VANETs. International Arab Journal of Information Technology, 8(4), 422–429.

    Google Scholar 

  16. Liu, J., Wan, J., Wang, Q., Deng, P., Zhou, K., & Qiao, Y. (2016). A survey on position-based routing for vehicular ad hoc networks. Telecommunication Systems, 62(1), 15–30. https://doi.org/10.1007/s11235-015-9979-7

    Article  Google Scholar 

  17. Hawbani, A., Torbosh, E., Wang, X., Sincak, P., Zhao, L., & Al-Dubai, A. (2021). Fuzzy-based distributed protocol for vehicle-to-vehicle communication. IEEE Transactions on Fuzzy Systems, 29(3), 612–626. https://doi.org/10.1109/TFUZZ.2019.2957254

    Article  Google Scholar 

  18. Hawbani, A., Wang, X., Al-Dubai, A., Zhao, L., Busaileh, O., Liu, P., & Al-qaness, M. A. A. (2021). A novel heuristic data routing for urban vehicular ad-hoc networks. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2021.3055504

    Article  Google Scholar 

  19. Zhao, L., Liu, Y., Al-Dubai, A. Y., Zomaya, A. Y., Min, G., & Hawbani, A. (2021). A novel generation-adversarial-network-based vehicle trajectory prediction method for intelligent vehicular networks. IEEE Internet of Things Journal, 8(3), 2066–2077. https://doi.org/10.1109/JIOT.2020.3021141

    Article  Google Scholar 

  20. Paul, B., Ibrahim, M., & Bikas, A. N. (2011). VANET routing protocols: Pros and cons. International Journal of Computer Applications, 20(3), 28–34. https://doi.org/10.5120/2413-3224

    Article  Google Scholar 

  21. Sabor, N., Sasaki, S., Abo-Zahhad, M., & Ahmed, S. M. (2017). A comprehensive survey on hierarchical-based routing protocols for mobile wireless sensor networks: Review, taxonomy, and future directions. Wireless Communications and Mobile Computing, 2017, 1–23. https://doi.org/10.1155/2017/2818542

    Article  Google Scholar 

  22. Alves Junior, J., & Wille, E. C. G. (2018). Routing in vehicular Ad Hoc networks: Main characteristics and tendencies. Journal of Computer Networks and Communications, 2018, 1–10. https://doi.org/10.1155/2018/1302123

    Article  Google Scholar 

  23. Perkins, C. E., & Royer, E. M. (1999). Ad-hoc on-demand distance vector routing. In Proceedings WMCSA’99. Second IEEE Workshop on mobile computing systems and applications (pp. 90–100). IEEE. https://doi.org/10.1109/MCSA.1999.749281

  24. Johnson, D. B., & Maltz, D. A. (1996). Dynamic source routing in Ad Hoc wireless networks. In T. Imielinski & H. F. Korth (Eds.), Mobile computing (pp. 153–181). Springer. https://doi.org/10.1007/978-0-585-29603-6_5

    Chapter  Google Scholar 

  25. Networks, M. (1997). Associativity-Based Routing for. Wireless Personal Communications, 103–139.

  26. Clausen, T., & Jacquet, P. (Eds.). (2003). Optimized link state routing protocol (OLSR). https://doi.org/10.17487/rfc3626

  27. Perkins, C. E., & Bhagwat, P. (1994). Highly dynamic Destination-Sequenced Distance-Vector routing (DSDV) for mobile computers. In Proceedings of the conference on Communications architectures, protocols and applications SIGCOMM ’94 (pp. 234–244). New York, New York, USA: ACM Press. https://doi.org/10.1145/190314.190336

  28. Basagni, S., Chlamtac, I., Syrotiuk, V. R., & Woodward, B. A. (1998). A distance routing effect algorithm for mobility (DREAM). In Proceedings of the 4th annual ACM/IEEE international conference on Mobile computing and networking MobiCom ’98 (pp. 76–84). New York, New York, USA: ACM Press. https://doi.org/10.1145/288235.288254

  29. Ghaffar, A., Hou, Y.-L., Liu, W.-Y., Dharejo, F. A., Zhang, H.-X., Jia, P., Yanyun, H., Liu, J., Yunjun, Z., & Nasir, Z. (2019). Two-dimensional displacement optical fiber sensor based on macro-bending effect. Optics & Laser Technology, 120, 105688. https://doi.org/10.1016/j.optlastec.2019.105688

    Article  Google Scholar 

  30. Development of protective scheme against collaborative black hole attacks in mobile Ad hoc networks. (2018). KSII Transactions on Internet and Information Systems, 12(3). https://doi.org/10.3837/tiis.2018.03.020

  31. Mili, R., & Chikhi, S. (2019). Reinforcement learning based routing protocols analysis for mobile Ad-Hoc networks (pp. 247–256). Springer.

    Google Scholar 

  32. Wu, C., Kumekawa, K., & Kato, T. (2010). Distributed reinforcement learning approach for vehicular Ad Hoc networks. IEICE Transactions on Communications, E93-B(6), 1431–1442. https://doi.org/10.1587/transcom.E93.B.1431

    Article  Google Scholar 

  33. Li, R., Li, F., Li, X., & Wang, Y. (2014). QGrid: Q-learning based routing protocol for vehicular ad hoc networks. In 2014 IEEE 33rd international performance computing and communications conference (IPCCC) (pp. 1–8). IEEE. https://doi.org/10.1109/PCCC.2014.7017079

  34. Alsharif, N., & Shen, X. S. (2014). iCARII: Intersection-based connectivity aware routing in vehicular networks. In 2014 IEEE international conference on communications (ICC) (pp. 2731–2735). IEEE. https://doi.org/10.1109/ICC.2014.6883737

  35. Ji, X., Xu, W., Zhang, C., Yun, T., Zhang, G., Wang, X., Wang, Y., Liu, B. (2019). Keep forwarding path freshest in VANET via applying reinforcement learning. In 2019 IEEE First International Workshop on Network Meets Intelligent Computations (NMIC) (pp. 13–18). IEEE. https://doi.org/10.1109/NMIC.2019.00008

  36. Wu, J., Fang, M., Li, H., & Li, X. (2020). RSU-assisted traffic-aware routing based on reinforcement learning for urban vanets. IEEE Access, 8, 5733–5748. https://doi.org/10.1109/ACCESS.2020.2963850

    Article  Google Scholar 

  37. Nazib, R. A., & Moh, S. (2020). Routing protocols for unmanned aerial vehicle-aided vehicular Ad Hoc networks: A survey. IEEE Access, 8, 77535–77560. https://doi.org/10.1109/ACCESS.2020.2989790

    Article  Google Scholar 

  38. Alrehan, A. M., & Alhaidari, F. A. (2019). Machine Learning Techniques to Detect DDoS Attacks on VANET System: A Survey. In 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS) (pp. 1–6). IEEE. https://doi.org/10.1109/CAIS.2019.8769454

  39. Doddalinganavar, S. S., Tergundi, P. V., & Patil, R. S. (2019). Survey on Deep Reinforcement Learning Protocol in VANET. In 2019 1st International Conference on Advances in Information Technology (ICAIT) (pp. 81–86). IEEE. https://doi.org/10.1109/ICAIT47043.2019.8987282

  40. Liu, J., Wang, Q., He, C., Jaffrès-Runser, K., Xu, Y., Li, Z., & Xu, Y. (2020). QMR:Q-learning based multi-objective optimization routing protocol for flying Ad Hoc networks. Computer Communications, 150, 304–316. https://doi.org/10.1016/j.comcom.2019.11.011

    Article  Google Scholar 

  41. Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT Press.

    MATH  Google Scholar 

  42. Li, M., & Li, H. (2020). Application of deep neural network and deep reinforcement learning in wireless communication. PLoS ONE, 15(7), e0235447. https://doi.org/10.1371/journal.pone.0235447

    Article  Google Scholar 

  43. Li, Y., Hu, X., Zhuang, Y., Gao, Z., Zhang, P., & El-Sheimy, N. (2020). Deep reinforcement learning (DRL): Another perspective for unsupervised wireless localization. IEEE Internet of Things Journal, 7(7), 6279–6287. https://doi.org/10.1109/JIOT.2019.2957778

    Article  Google Scholar 

  44. Ohnishi, S., Uchibe, E., Yamaguchi, Y., Nakanishi, K., Yasui, Y., & Ishii, S. (2019). Constrained deep Q-learning gradually approaching ordinary Q-learning. Frontiers in Neurorobotics. https://doi.org/10.3389/fnbot.2019.00103

    Article  Google Scholar 

  45. Killat, M., & Hartenstein, H. (2009). An empirical model for probability of packet reception in vehicular Ad Hoc networks. EURASIP Journal on Wireless Communications and Networking, 2009(1), 721301. https://doi.org/10.1155/2009/721301

    Article  Google Scholar 

  46. Qaisar, M. U. F., Wang, X., Hawbani, A., Khan, A., Ahmed, A., & Wedaj, F. T. (2020). TORP: Load Balanced Reliable Opportunistic Routing for Asynchronous Wireless Sensor Networks. In 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) (pp. 1384–1389). IEEE. https://doi.org/10.1109/TrustCom50675.2020.00186

Download references

Funding

This paper is supported by the “Fundamental Research Funds for the Central Universities Nos. WK2150110007 and WK2150110012” and the National Natural Science Foundation of China (Nos. 61772490, 61472382, 61472381, 61701322, and 61572454).

Author information

Authors and Affiliations

  1. School of Computer and Technology, University of Science and Technology of China, Hefei, 230027, Anhui, China

    Arbelo Lolai, Xingfu Wang, Ammar Hawbani, Taiyaba Qureshi, Muhammad Umar Farooq, Muhammad Mujahid & Abdul Hafeez Babar

  2. Computer Network Information Center, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, 100190, China

    Fayaz Ali Dharejo

Authors
  1. Arbelo Lolai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingfu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ammar Hawbani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fayaz Ali Dharejo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Taiyaba Qureshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Muhammad Umar Farooq
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Muhammad Mujahid
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Abdul Hafeez Babar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AL Proposed the method and wrote the manuscript; FAD and TQ provided helpful guidance on the technique and improved the writing; XW and AH provided guidance on the experimental problem and validated the results; MU and MM compiled the experimental image data. AHB helped in improving the analysis of the results.

Corresponding authors

Correspondence to Xingfu Wang or Ammar Hawbani.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lolai, A., Wang, X., Hawbani, A. et al. Reinforcement learning based on routing with infrastructure nodes for data dissemination in vehicular networks (RRIN). Wireless Netw 28, 2169–2184 (2022). https://doi.org/10.1007/s11276-022-02926-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-022-02926-w

Keywords

Search

Navigation