research-article

Reinforcement Learning-Based MAC Protocol for Underwater Multimedia Sensor Networks

Authors:

Nurzaman Ahmed,

Wei WeiAuthors Info & Claims

ACM Transactions on Sensor Networks (TOSN), Volume 18, Issue 3

Article No.: 37, Pages 1 - 25

https://doi.org/10.1145/3484201

Published: 19 September 2022 Publication History

Abstract

High propagation delay, high error probability, floating node mobility, and low data rates are the key challenges for Underwater Wireless Multimedia Sensor Networks (UMWSNs). In this article, we propose RL-MAC, a Reinforcement Learning (RL)–based Medium Access Control (MAC) protocol for multimedia sensing in an Underwater Acoustic Network (UAN) environment. The proposed scheme uses Transmission Opportunity (TXOP) for relay nodes in a multi-hop network for improved efficiency concerning the mobility of the relays and sensor nodes. The access point (AP) and relay nodes calculate traffic demands from the initial contention of the sensor nodes. Our solution uses Q-learning to enhance the contention mechanism at the initial phase of multimedia transmission. Based on the traffic demands, RL-MAC allocates TXOP duration for the uplink multimedia reception. Further, the Structural Similarity Index Measure (SSIM) and compression techniques are used for calculating the image quality at the receiver end and reducing the image at the destination, respectively. We implement a prototype of the proposed scheme over an off-the-shelf, low-cost hardware setup. Moreover, extensive simulation over NS-3 shows a significant packet delivery ratio and throughput compared with the existing state-of-the-art.

References

[1]

2011. Quality aware image transmission over underwater multimedia sensor networks. Ad Hoc Networks 9, 7 (2011), 1287–1301.

Digital Library

[2]

Faisal Alfouzan, Alireza Shahrabi, Seyed Mohammad Ghoreyshi, and Tuleen Boutaleb. 2016. Performance comparison of sender-based and receiver-based scheduling MAC protocols for underwater sensor networks. In 19th International Conference on Network-Based Information Systems (NBiS’16). IEEE, 99–106.

[3]

Faisal Alfouzan, Alireza Shahrabi, Seyed Mohammad Ghoreyshi, and Tuleen Boutaleb. 2017. Efficient depth-based scheduling MAC protocols for underwater sensor networks. In 9th International Conference on Ubiquitous and Future Networks (ICUFN’17). IEEE, 827–832.

[4]

Faisal Alfouzan, Alireza Shahrabi, Seyed Mohammad Ghoreyshi, and Tuleen Boutaleb. 2019. A comparative performance evaluation of distributed collision-free MAC protocols for underwater sensor networks. In SENSORNETS. 85–93.

[5]

Muhammad Awais, Nadeem Javaid, Nidal Naseer, and Muhammad Imran. 2019. Exploiting energy efficient routing protocols for void hole alleviation in IoT enabled underwater WSN. In 15th International Wireless Communications Mobile Computing Conference (IWCMC’19). 1797–1802.

[6]

Jenith L. Banluta, Lilet D. Balbuena, Nestor Michael C. Tiglao, and Jhoanna Rhodette I. Pedrasa. 2017. Comparison of contention-based MAC protocols for underwater sensor networks. In International Conference on Information Networking (ICOIN’17). IEEE, 518–523.

[7]

Fatma Bouabdallah, Raouf Boutaba, Ahmed Mehaoua, et al. 2018. Collision avoidance energy efficient multi-channel MAC protocol for underwater acoustic sensor networks. IEEE Transactions on Mobile Computing 18, 10 (2018), 2298–2314.

[8]

JungHun Byun, Jin Hoon Park, and Ohyun Jo. 2020. Deep learning based BER prediction model in underwater IoT networks. Journal of Convergence for Information Technology 10, 6 (2020), 41–48.

[9]

Cecilia Carbonelli and Urbashi Mitra. 2006. Cooperative multihop communication for underwater acoustic networks. In Proceedings of the 1st ACM International Workshop on Underwater Networks. 97–100.

Digital Library

[10]

W. Chen, Q. Guan, H. Yu, F. Ji, and F. Chen. 2021. Medium access control under space-time coupling in underwater acoustic networks. IEEE Internet of Things Journal (2021), 1–1.

[11]

Min Deng, Huifang Chen, and Lei Xie. 2018. DCO-MAC: A hybrid MAC protocol for data collection in underwater acoustic sensor networks. Sensors 18, 7 (2018), 2300.

[12]

Dmitrii Dugaev, Zheng Peng, Yu Luo, and Lina Pu. 2020. Reinforcement-learning based dynamic transmission range adjustment in medium access control for underwater wireless sensor networks. Electronics 9, 10 (2020), 1727.

[13]

Guangyu Fan, Huifang Chen, Lei Xie, and Kuang Wang. 2013. A hybrid reservation-based MAC protocol for underwater acoustic sensor networks. Ad Hoc Networks 11, 3 (2013), 1178–1192.

Digital Library

[14]

Qiang Fan, Jing Fan, Jie Li, and Xiaofang Wang. 2012. A multi-hop energy-efficient sleeping MAC protocol based on TDMA scheduling for wireless mesh sensor networks. Journal of Networks 7, 9 (2012), 1355–1361.

[15]

Firoj Gazi, Sudip Misra, Nurzaman Ahmed, Anandarup Mukherjee, and Neeraj Kumar. 2020. UnRest: Underwater reliable acoustic communication for multimedia streaming. In IEEE Global Communications Conference (GLOBECOM’20). IEEE, 1–6.

[16]

Xuan Geng and Y. Rosa Zheng. 2019. MAC protocol for underwater acoustic networks based on deep reinforcement learning. In Proceedings of the International Conference on Underwater Networks & Systems. ACM.

Digital Library

[17]

T. J. Hajenko and C. R. Benson. 2010. The high frequency underwater acoustic channel. In OCEANS. IEEE, 1–3.

[18]

Seongwon Han, Youngtae Noh, Uichin Lee, and Mario Gerla. 2019. Optical-acoustic hybrid network toward real-time video streaming for mobile underwater sensors. Ad Hoc Networks 83 (2019), 1–7.

[19]

Waheed Hyder, Adnan Nadeem, Abdul Basit, Kashif Rizwan, Kamran Ahsan, and Nadeem Mehmood. 2017. A review and classification of energy efficient MAC protocols for underwater wireless sensor network. Journal of Basic and Applied Sciences 13 (2017), 63–70.

[20]

Zhigang Jin, Qinyi Zhao, and Yishan Su. 2019. RCAR: A reinforcement-learning-based routing protocol for congestion-avoided underwater acoustic sensor networks. IEEE Sensors Journal 19, 22 (2019), 10881–10891.

[21]

Delphin Raj, Jinyoung Lee, Eunbi Ko, Soo-Young Shin, Jung-Il Namgung, Sun-Ho Yum, and Soo-Hyun Park. 2020. Underwater network management system in Internet of Underwater Things: Open challenges, benefits, and feasible solution. Electronics 9, 7 (2020).

[22]

Lutful Karim, Qusay H. Mahmoud, Nidal Nasser, Alagan Anpalagan, and Nargis Khan. 2017. Localization in terrestrial and underwater sensor-based M2M communication networks: Architecture, classification and challenges. International Journal of Communication Systems 30, 4 (2017), e2997.

[23]

Evgeny Khorov, Andrey Lyakhov, Alexander Krotov, and Andrey Guschin. 2015. A survey on IEEE 802.11 ah: An enabling networking technology for smart cities. Computer Communications 58 (2015), 53–69.

Digital Library

[24]

Hee-won Kim, Tae Ho Im, and Ho-Shin Cho. 2018. UCMAC: A cooperative MAC protocol for underwater wireless sensor networks. Sensors 18, 6 (2018), 1969.

[25]

Kieran Gerard Larkin. 2015. Structural similarity index SSIMplified: Is there really a simpler concept at the heart of image quality measurement?. arXiv preprint arXiv:1503.06680 (2015).

[26]

Meiqin Liu, Weikun Huang, Liangfang Qian, and Senlin Zhang. 2016. An improved R-MAC based MAC protocol for underwater acoustic networks. In IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC’16). IEEE, 1–6.

[27]

Robert Martin, Yibo Zhu, Lina Pu, Fei Dou, Zheng Peng, Jun-Hong Cui, and Sanguthevar Rajasekaran. 2015. Aqua-sim next generation: A NS-3 based simulator for underwater sensor networks. In Proceedings of the 10th International Conference on Underwater Networks & Systems. Springer, 1–2.

Digital Library

[28]

Heungwoo Nam and Sunshin An. 2007. An ultrasonic sensor based low-power acoustic modem for underwater communication in underwater wireless sensor networks. In International Conference on Embedded and Ubiquitous Computing. Springer, 494–504.

[29]

Sung Hyun Park, Paul Daniel Mitchell, and David Grace. 2018. Performance of the ALOHA-Q MAC protocol for underwater acoustic networks. In International Conference on Computing, Electronics & Communications Engineering (iCCECE’18). IEEE, 189–194.

[30]

Sung Hyun Park, Paul Daniel Mitchell, and David Grace. 2019. Reinforcement learning based MAC protocol (UW-ALOHA-Q) for underwater acoustic sensor networks. IEEE Access 7 (2019), 165531–165542.

[31]

Sung Hyun Park, Paul Daniel Mitchell, and David Grace. 2020. Reinforcement learning based MAC protocol (UW-ALOHA-QM) for mobile underwater acoustic sensor networks. IEEE Access (2020), 5906–5919.

[32]

Bernd-Christian Renner, Jan Heitmann, and Fabian Steinmetz. 2020. ahoi: Inexpensive, low-power communication and localization for underwater sensor networks and \(\mu\) AUVs. ACM Transactions on Sensor Networks 16, 2 (2020), 1–46.

Digital Library

[33]

Alak Roy and Nityananda Sarma. 2021. A synchronous duty-cycled reservation based MAC protocol for underwater wireless sensor networks. Digital Communications and Networks 7, 3 (2021), 385–398. DOI:.

[34]

Milica Stojanovic. 2007. On the relationship between capacity and distance in an underwater acoustic communication channel. ACM SIGMOBILE Mobile Computing and Communications Review 11, 4 (2007), 34–43.

Digital Library

[35]

William H. Thorp. 1967. Analytic description of the low-frequency attenuation coefficient. The Journal of the Acoustical Society of America 42, 1 (1967), 270–270.

[36]

SM Ullah. 2019. A robust energy efficient MAC protocol design for Underwater sensor network. Ph.D. Dissertation. Daffodil International University, Dhaka, Bangladesh.

[37]

Lijuan Wang, Xiujuan Du, Xiuxiu Liu, Chong Li, and ShengLin Lan. 2018. Design and implementation of state-based MAC protocol for UANs in real testbed. In OCEANS 2018 MTS/IEEE Charleston. IEEE, 1–5.

[38]

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612.

Digital Library

[39]

Liang Xiao, Donghua Jiang, Ye Chen, Wei Su, and Yuliang Tang. 2019. Reinforcement-learning-based relay mobility and power allocation for underwater sensor networks against jamming. IEEE Journal of Oceanic Engineering 45, 3 (2019), 1148–1156.

[40]

Xiaowen Ye, Yiding Yu, and Liqun Fu. 2022. Deep reinforcement learning based MAC protocol for underwater acoustic networks. IEEE Transactions on Mobile Computing 21, 5 (2022), 1625–1638.

[41]

Xiaowen Ye, Yiding Yu, and Liqun Fu. 2020. Deep reinforcement learning based MAC protocol for underwater acoustic networks. IEEE Transactions on Mobile Computing (2020), 1–1.

[42]

Ziwei Zhang, Wei Shi, Qiuna Niu, Ying Guo, Jingjing Wizeang, and Hanjiang Luo. 2019. A load-based hybrid MAC protocol for underwater wireless sensor networks. IEEE Access 7 (2019), 104542–104552.

[43]

Xiaoxiao Zhuo, Meiyan Liu, Yan Wei, Guanding Yu, Fengzhong Qu, and Rui Sun. 2020. AUV-aided energy-efficient data collection in underwater acoustic sensor networks. IEEE Internet of Things Journal 7, 10 (2020), 10010–10022.

Cited By

Fedorova TRyzhov VSafronov KSemenov NSulaiman S(2024)Energy-Efficient and Reliable Deployment Models for Hybrid Underwater Acoustic Sensor Networks with a Mobile GatewayJournal of Marine Science and Application10.1007/s11804-024-00444-zOnline publication date: 8-Jul-2024
https://doi.org/10.1007/s11804-024-00444-z
Tomovic SRadusinovic I(2023)DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor NetworksSensors10.3390/s2309447423:9(4474)Online publication date: 4-May-2023
https://doi.org/10.3390/s23094474
Zheng ZJiang SFeng RGe LGu C(2023)Survey of Reinforcement-Learning-Based MAC Protocols for Wireless Ad Hoc Networks with a MAC Reference ModelEntropy10.3390/e2501010125:1(101)Online publication date: 3-Jan-2023
https://doi.org/10.3390/e25010101
Show More Cited By

Index Terms

Reinforcement Learning-Based MAC Protocol for Underwater Multimedia Sensor Networks
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Embedded systems

Recommendations

Propagation modeling and MAC-layer performance in EM-based underwater sensor networks
DIVANet '14: Proceedings of the fourth ACM international symposium on Development and analysis of intelligent vehicular networks and applications

Electromagnetic (EM) waves have been considered as an alternative to acoustic communications in seawater. In this paper, we propose a propagation model for EM channels in seawater and study the MAC-layer performance in EM-based underwater sensor ...
Distributed CDMA-based MAC Protocol for Underwater Sensor Networks
LCN '07: Proceedings of the 32nd IEEE Conference on Local Computer Networks

Underwater Sensor Networks are typically distributed in nature and the nodes communicate using acoustic waves over a wireless medium. Such networks are characterized by long and variable propagation delays, intermittent connectivity, limited bandwidth ...
An OFDM based MAC protocol for underwater acoustic networks
WUWNet '10: Proceedings of the 5th International Workshop on Underwater Networks

In this paper, we investigate OFDM based MAC protocols for underwater acoustic networks. Due to the severe multi-path effects of underwater acoustic channels, the guard time between OFDM blocks becomes significant, which greatly degrades the system ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 18, Issue 3

August 2022

480 pages

ISSN:1550-4859

EISSN:1550-4867

DOI:10.1145/3531537

Editor:
Yunhao Liu
Tsinghua University, China

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 19 September 2022

Online AM: 23 February 2022

Accepted: 26 August 2021

Revised: 27 July 2021

Received: 17 April 2021

Published in TOSN Volume 18, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
797
Total Downloads

Downloads (Last 12 months)263
Downloads (Last 6 weeks)24

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fedorova TRyzhov VSafronov KSemenov NSulaiman S(2024)Energy-Efficient and Reliable Deployment Models for Hybrid Underwater Acoustic Sensor Networks with a Mobile GatewayJournal of Marine Science and Application10.1007/s11804-024-00444-zOnline publication date: 8-Jul-2024
https://doi.org/10.1007/s11804-024-00444-z
Tomovic SRadusinovic I(2023)DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor NetworksSensors10.3390/s2309447423:9(4474)Online publication date: 4-May-2023
https://doi.org/10.3390/s23094474
Zheng ZJiang SFeng RGe LGu C(2023)Survey of Reinforcement-Learning-Based MAC Protocols for Wireless Ad Hoc Networks with a MAC Reference ModelEntropy10.3390/e2501010125:1(101)Online publication date: 3-Jan-2023
https://doi.org/10.3390/e25010101
Wu CChang YTsai CHou C(2023)A Q-learning-based distributed queuing Mac protocol for Internet-of-Things networksEURASIP Journal on Wireless Communications and Networking10.1186/s13638-023-02288-72023:1Online publication date: 16-Aug-2023
https://dl.acm.org/doi/10.1186/s13638-023-02288-7
Gazi FAhmed NMisra S(2023)RE-MAC: A Hybrid MAC Protocol for Underwater Multimedia Communication SystemIEEE Systems Journal10.1109/JSYST.2022.318501517:1(840-847)Online publication date: Mar-2023
https://doi.org/10.1109/JSYST.2022.3185015
Liu ZWang X(2023)Energy-balanced routing in wireless sensor networks with reinforcement learning using greedy action chainsSoft Computing10.1007/s00500-023-08734-4Online publication date: 21-Jun-2023
https://doi.org/10.1007/s00500-023-08734-4

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents