Abstract
Wireless underground sensor networks (WUSNs) consist of sensors that are buried in and communicate through soil medium, while the channel quality of WUSNs is greatly impacted by the underground environment, such as soil moisture and composition. Due to the precipitation and harsh weather, the underground environments change frequently, which make wireless communication in WUSNs much complicated than that in terrestrial over-the-air wireless sensor networks. To achieve reliable and energy-efficient data gathering in dynamic WUSNs, this article proposes an optimal transmission policy, where path loss of sensory data transmission, energy constraint, and network load balancing are the factors to be considered. Specifically, we capture the effect of underground environments on wireless communications, and evaluate path probability, energy consumption, and load balancing factor with respect to reliability and efficiency of transmission paths. The transmission topology can be reduced to a multi-objective and multi-constrained optimization problem and solved through an improved maximum flow minimum cost algorithm. By using reinforcement learning, we derive an adaptive transmission policy for underground sensors to efficiently use their energy and avoid transmitting sensory data in unreliable paths under a dynamic environment. Through simulations and comparison upon publicly available real data, our technique achieves more reliable wireless communication with significant reduction of packet loss, and enables more energy-efficient data gathering than other techniques, especially when soil moisture varies frequently.
Similar content being viewed by others
References
Chen C, Ding Y, Wang Z, Zhao J, Guo B, Zhang D (2020) VTracer: when online vehicle trajectory compression meets mobile edge computing. IEEE Syst J 14(2):1635–1646
Chen C, Ding Y, Xie X, Zhang S, Wang Z, Feng L (2020) TrajCompressor: an online map-matching-based trajectory compression framework leveraging vehicle heading direction and change. IEEE Trans Intell Transp Syst 21(5):2012–2028
Chen C, Zhang D, Ma X, Guo B, Wang L, Wang Y, Sha E (2017) CrowdDeliver: planning city-wide package delivery paths leveraging the crowd of taxis. IEEE Trans Intell Transp Syst 18(6):1478–1496
Chen Z, He C, He Z, Chen M (2018) BD-ADOPT: a hybrid DCOP algorithm with Best-First and Depth-First search strategies. Artif Intell Rev 50:161–199
Chowdhury AR, Pramanik A, Roy GC (2020) On wireless communication in underground mine system. In: Proceedings of the 2nd International Conference on Communication, Devices and Computing, pp 159–169
Dascalu S, Harris Jr FC, McMahon Jr M, Fritzinger E, Strachan S (2014) An overview of the Nevada Climate Change Portal. In: 7Th International Congress on Environmental Modelling and Software, vol 1, pp 75–82
Guo H, Ben B (2019) Reinforcement learning-enabled reliable wireless sensor networks in dynamic underground environments. In: IEEE Military Communications Conference, pp 646–651
Guo S, Chen C, Wang J, Liu Y, Ke X, Yu Z, Zhang D, Chiu DM (2019) ROD-Revenue: seeking strategies analysis and revenue prediction in ride-on-demand service using multi-source urban data. IEEE Transactions on Mobile Computing, pp 2202–2220
Guo W, Zhu W, Yu Z, Wang J, Guo B (2019) A survey of task allocation: contrastive perspectives from wireless sensor networks and mobile crowdsensing. IEEE Access 7:78406–78420
Hongwei H, Jingkang S, Fei W, Dongming Z, Dongmei Z (2020) Theoretical and experimental studies on the signal propagation in soil for wireless underground sensor networks. Sensor 20(9):2580
Huang Y, Chen Z, Han T, Liu X (2018) One energy-efficient random-walk topology evolution method for underground wireless sensor networks. Int J Distrib Sens Netw 14(9):155014771880062
Idrees AK, Alhussaini R, Salman MA (2020) Energy-efficient two-layer data transmission reduction protocol in periodic sensor networks of IoTs personal and ubiquitous computing
Jalal D, Ezzedine T (2018) Leaks detection and localization in water distribution network based on wireless underground sensor networks. In: International Conference on Internet of Things, Embedded Systems and Communications, pp 42–46
Ketshabetswe L, Adamu M, Mangwala M, Chuma J, Sigweni B (2019) Communication Protocols for wireless sensor networks: a survey and comparison. Heliyon 5:1–43
Kisseleff S, Akyildiz IF, Gerstacker WH (2018) Survey on advances in magnetic induction-based wireless underground sensor networks. IEEE Internet Things J 5(6):4843– 4856
Kumar] D. P., Amgoth T, Annavarapu CSR (2019) Machine learning algorithms for wireless sensor networks: a survey. Inf Fus 49:1–25
Leong AS, Ramaswamy A, Quevedo DE, Karl H, Shi L (2020) Deep reinforcement learning for wireless sensor scheduling in cyber-physical systems. Automatica 113:108759
Li K, Ni W, Abolhasan M, Tovar E (2019) Reinforcement learning for scheduling wireless powered sensor communications. IEEE Trans Green Commun Netw 3(2):264–274
Liu G, Sun Z, Jiang T (2019) Joint time and energy allocation for QoS-aware throughput maximization in MIMO-based wireless powered underground sensor networks. IEEE Trans Commun 67(2):1400–1412
Liu G, Wang Z, Jiang T (2016) Qos-aware throughput maximization in wireless powered underground sensor networks. IEEE Trans Commun 64(11):4776–4789
Chincoli M, Liotta A (2018) Self-learning power control in wireless sensor networks. Sensors 18(2):375
Qiu S, Mias C, Guo W, Geng X (2019) HS2 Railway embankment monitoring: effect of soil condition on underground signals. SN Appl 1(6):1–7
Raza U, Salam A (2020) On-site and external energy harvesting in underground wireless electronics 9(4)
Salam A, Shah S (2019) Urban underground infrastructure monitoring IoT: the path loss analysis. In: IEEE World Forum on Internet of Things, pp 398–401
Salam A, Vuran MC, Irmak S (2019) Di-Sense: In situ real-time permittivity estimation and soil moisture sensing using wireless underground communications. Comput Netw 151:31–41
Sharma R, Prakash S (2020) Enhancement of relay nodes communication approach in WSN-IoT for underground coal mine. J Inf Optim Sci 41(2):521–531
Szymanski TH (2013) Max-flow min-cost routing in a future-Internet with improved QoS guarantees. IEEE Trans Commun 61(4):1485–1497
Tam NT, Binh HTT, Dung DA, Lan PN, Vinh LT, Yuan B, Yao X (2019) A hybrid clustering and evolutionary approach for wireless underground sensor network lifetime maximization. Inf Sci 504:372–393
Tam NT, Binh HTT, Hung TH, Dung DA, Vinh LT (2019) Prolong the network lifetime of wireless underground sensor networks by optimal relay node placement. In: Applications of evolutionary computation, vol 11454, pp 439–453
Trang HTH, Dung LT, Hwang SQ (2018) Connectivity analysis of underground sensors in wireless underground sensor networks. Ad Hoc Netw 71:104–116
Wang KI, Wu S, Ivoghlian A, Salcic Z, Austin A, Zhou X (2019) LWS: A LoRaWAN wireless underground sensor network simulator for agriculture applications. In: IEEE Smartworld, Ubiquitous Intelligence Computing, Advanced Trusted Computing, Scalable Computing Communications, Cloud Big Data Computing, Internet of People and Smart City Innovation, pp 475–482
Wang S, Shin Y (2019) Efficient routing protocol based on reinforcement learning for magnetic induction underwater sensor networks. IEEE Access 7:82027–82037
Wohwe Sambo D, Forster A, Yenke BO, Sarr I, Gueye B, Dayang P (2020) Wireless underground sensor networks path loss model for precision agriculture (WUSN-PLM). IEEE Sens J 20(10):5298–5313
Xia X, Chen Z, Liu H, Wang H, Zeng F (2016) A routing protocol for multisink wireless sensor networks in underground coalmine tunnels. Sensors 16:16–32
Yu X, Han W, Zhang Z (2016) Path loss estimation for wireless underground sensor network in agricultural application. Agric Res 6:97–102
Yu X, Zhang Z, Han W (2019) Survey on transmission performance for soil wireless sensor network signal in different communication. IAENG Int J Comput 46(3):475–484
Yu XQ, Zhang ZL, Han WT (2017) Evaluation of communication in wireless underground sensor networks. IOP Conf Ser Earth Environ Sci 69:012083
Yu Z, Zhang D, Yu Z, Yang D (2015) Participant selection for offline event marketing leveraging location-based social networks. IEEE Trans Syst Man Cybern Syst 45(6):853–864
Yuan B, Chen H, Yao X (2017) Optimal relay placement for lifetime maximization in wireless underground sensor networks. Inf Sci 418:463–479
Zhou Z, Zhao D, Liu L, Hung PC (2018) Energy-aware composition for wireless sensor networks as a service. Fut Gen Comput Syst 80:299–310
Zhou Z, Zhao D, Shu L, Tsang KF (2015) A novel two-tier cooperative caching mechanism for the optimization of multi-attribute periodic queries in wireless sensor networks. Sensors 15(7):15033–15066
Zungeru AM, Mangwala M, Chuma J (2017) Optimal node placement in wireless underground sensor networks. Int J Appl Eng Res 12(20):9290–9297
Funding
This work was supported by the National Key R&D Program of China (2019YFB2101803) and by the National Natural Science Foundation of China (61772479).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhao, D., Zhou, Z., Wang, S. et al. Reinforcement learning–enabled efficient data gathering in underground wireless sensor networks. Pers Ubiquit Comput 27, 581–598 (2023). https://doi.org/10.1007/s00779-020-01443-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00779-020-01443-x