research-article

Computation Offloading with Multiple Agents in Edge-Computing–Supported IoT

Authors:

Yan WangAuthors Info & Claims

ACM Transactions on Sensor Networks (TOSN), Volume 16, Issue 1

Article No.: 8, Pages 1 - 27

https://doi.org/10.1145/3372025

Published: 19 December 2019 Publication History

Abstract

With the development of the Internet of Things (IoT) and the birth of various new IoT devices, the capacity of massive IoT devices is facing challenges. Fortunately, edge computing can optimize problems such as delay and connectivity by offloading part of the computational tasks to edge nodes close to the data source. Using this feature, IoT devices can save more resources while still maintaining the quality of service. However, since computation offloading decisions concern joint and complex resource management, we use multiple Deep Reinforcement Learning (DRL) agents deployed on IoT devices to guide their own decisions. Besides, Federated Learning (FL) is utilized to train DRL agents in a distributed fashion, aiming to make the DRL-based decision making practical and further decrease the transmission cost between IoT devices and Edge Nodes. In this article, we first study the problem of computation offloading optimization and prove the problem is an NP-hard problem. Then, based on DRL and FL, we propose an offloading algorithm that is different from the traditional method. Finally, we studied the effects of various parameters on the performance of the algorithm and verified the effectiveness of both the DRL and FL in the IoT system.

References

[1]

Nasir Abbas, Yan Zhang, Amir Taherkordi, and Tor Skeie. 2017. Mobile edge computing: A survey. IEEE Internet of Things Journal 5, 1 (2017), 450--465.

[2]

Kofi Sarpong Adu-Manu, Nadir Adam, Cristiano Tapparello, Hoda Ayatollahi, and Wendi Heinzelman. 2018. Energy-harvesting wireless sensor networks (EH-WSNs): A review. ACM Transactions on Sensor Networks (TOSN) 14, 2 (2018), 10.

Digital Library

[3]

Ganesh Ananthanarayanan, Paramvir Bahl, Peter Bodík, Krishna Chintalapudi, Matthai Philipose, Lenin Ravindranath, and Sudipta Sinha. 2017. Real-time video analytics: The killer app for edge computing. Computer 50, 10 (2017), 58--67.

Digital Library

[4]

Bigmoyan. 2017. Keras: The Python Deep Learning library. https://keras.io.

[5]

Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konecny, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. 2019. Towards federated learning at scale: System design. Arxiv Preprint Arxiv:1902.01046 (2019).

[6]

Min Chen and Yixue Hao. 2018. Task offloading for mobile edge computing in software defined ultra-dense network. IEEE Journal on Selected Areas in Communications 36, 3 (2018), 587--597.

[7]

Xu Chen, Lei Jiao, Wenzhong Li, and Xiaoming Fu. 2016. Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans. Netw. 24, 5 (2016), 2795--2808.

Digital Library

[8]

Xianfu Chen, Honggang Zhang, Celimuge Wu, Shiwen Mao, Yusheng Ji, and Mehdi Bennis. 2018. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal (2018), 1--1.

[9]

ETSI. 2014. Mobile-Edge Computing —Introductory Technical White Paper. https://portal.etsi.org/Portals/0/TBpages/MEC/Docs/Mobile-edge_Computing_-_Introductory_Technical_White_Paper_V1%2018-09-14.pdf.

[10]

Michael R. Garey and David S. Johnson. 1978. “Strong” NP-completeness results: Motivation, examples, and implications. Journal of the ACM (JACM) 25, 3 (1978), 499--508.

Digital Library

[11]

Marco E. T. Gerards, Johann L. Hurink, and Jan Kuper. 2014. On the interplay between global DVFS and scheduling tasks with precedence constraints. IEEE Trans. Comput. 64, 6 (2014), 1742--1754.

[12]

Google. 2017. Deploy machine learning models on mobile and IoT devices. https://www.tensorflow.org/lite.

[13]

Google. 2017. Federated Learning: Collaborative Machine Learning without Centralized Training Data. https://research.googleblog.com/2017/04/federated-learning-collaborative.html.

[14]

Dennis Grewe, Marco Wagner, Mayutan Arumaithurai, Ioannis Psaras, and Dirk Kutscher. 2017. Information-centric mobile edge computing for connected vehicle environments: Challenges and research directions. In Proceedings of the Workshop on Mobile Edge Communications. ACM, 7--12.

Digital Library

[15]

Xiaoya Hu, Bingwen Wang, and Han Ji. 2013. A wireless sensor network-based structural health monitoring system for highway bridges. Computer-Aided Civil and Infrastructure Engineering 28, 3 (2013), 193--209.

[16]

S.M. Riazul Islam, Daehan Kwak, M.D. Humaun Kabir, Mahmud Hossain, and Kyung-Sup Kwak. 2015. The Internet of Things for health care: A comprehensive survey. IEEE Access 3 (2015), 678--708.

[17]

Tommi Jaakkola, Michael I. Jordan, and Satinder P. Singh. 1994. Convergence of stochastic iterative dynamic programming algorithms. In Advances in Neural Information Processing Systems. 703--710.

[18]

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM International Conference on Multimedia. ACM, 675--678.

Digital Library

[19]

Yin Jie, Ji Yong Pei, Li Jun, Guo Yun, and Xu Wei. 2013. Smart home system based on IoT technologies. In Proceedings of the 2013 International Conference on Computational and Information Sciences. IEEE, 1789--1791.

Digital Library

[20]

David S. Johnson and Michael R. Garey. 1979. Computers and Intractability: A Guide to the Theory of NP-completeness. Vol. 1. W. H. Freeman San Francisco.

[21]

M. D. Kamruzzaman, Nurul I. Sarkar, Jairo Gutierrez, and Sayan Kumar Ray. 2017. A study of IoT-based post-disaster management. In Proceedings of the 2017 International Conference on Information Networking (ICOIN). IEEE, 406--410.

[22]

Nacer Khalil, Omprakash Gnawali, Driss Benhaddou, and Jaspal Subhlok. 2018. SonicDoor: A person identification system based on modeling of shape, behavior, and walking patterns. ACM Transactions on Sensor Networks (TOSN) 14, 3--4 (2018), 27.

Digital Library

[23]

Jakub Konečnỳ, H. Brendan McMahan, Daniel Ramage, and Peter Richtárik. 2016. Federated optimization: Distributed machine learning for on-device intelligence. Arxiv Preprint Arxiv:1610.02527 (2016).

[24]

Jakub Konečnỳ, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. Arxiv Preprint Arxiv:1610.05492 (2016).

[25]

Yuyi Mao, Changsheng You, Jun Zhang, Kaibin Huang, and Khaled B. Letaief. 2017. A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys 8 Tutorials 19, 4 (2017), 2322--2358.

[26]

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017), (Fort Lauderdale, FL), (Proceedings of Machine Learning Research), Vol. 54. PMLR, 1273--1282. http://proceedings.mlr.press/v54/mcmahan17a.html.

[27]

Regulation (EU) 2016/679 of the European Parliament and of the Council. 2016. Mobile-Edge Computing -- Introductory Technical White Paper. https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1564833514064&uri==CELEX:32016R0679

[28]

Alexander Pyattaev, Kerstin Johnsson, Sergey Andreev, and Yevgeni Koucheryavy. 2015. Communication challenges in high-density deployments of wearable wireless devices. IEEE Wireless Communications 22, 1 (2015), 12--18.

Digital Library

[29]

Yongming Rao, Jiwen Lu, and Jie Zhou. 2017. Attention-aware deep reinforcement learning for video face recognition. In Proceedings of the IEEE International Conference on Computer Vision. 3931--3940.

[30]

S. Sujin Issac Samuel. 2016. A review of connectivity challenges in IoT-smart home. In Proceedings of the 2016 3rd MEC International Conference on Big Data and Smart City (ICBDSC). IEEE, 1--4.

[31]

Pararth Shah, Marek Fiser, Aleksandra Faust, J. Chase Kew, and Dilek Hakkani-Tur. 2018. Follownet: Robot navigation by following natural language directions with deep reinforcement learning. Arxiv Preprint Arxiv:1805.06150 (2018).

[32]

Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge computing: Vision and challenges. IEEE Internet of Things Journal 3, 5 (2016), 637--646.

[33]

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484.

[34]

Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT press.

[35]

Hado van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double q-learning. In Proceedings of the 13th AAAI Conference on Artificial Intelligence, 2016, (Phoenix, Arizona), AAAI Press, 2094--2100. http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12389

[36]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529--533.

[37]

Tian Wang, Guangxue Zhang, Anfeng Liu, Md Zakirul Alam Bhuiyan, and Qun Jin. 2018. A secure IoT service architecture with an efficient balance dynamics based on cloud and edge computing. IEEE Internet of Things Journal (2018).

[38]

Yuchen Yang, Longfei Wu, Guisheng Yin, Lijie Li, and Hongbin Zhao. 2017. A survey on security and privacy issues in Internet-of-Things. IEEE Internet of Things Journal 4, 5 (2017), 1250--1258.

[39]

Engin Zeydan, Ejder Bastug, Mehdi Bennis, Manhal Abdel Kader, Ilyas Alper Karatepe, Ahmet Salih Er, and Merouane Debbah. 2016. Big data caching for networking: Moving from cloud to edge. IEEE Communications Magazine 54, 9 (Sep. 2016), 36--42.

Cited By

Yaraziz MSohrabi Safa NAzad M(2024)Edge computing in IoT for smart healthcareJournal of Ambient Intelligence and Smart Environments10.3233/AIS-230009(1-30)Online publication date: 19-Jun-2024
https://doi.org/10.3233/AIS-230009
Xue HZhang DWu CJi YLong SWang CSato T(2024)Parameter Estimation-Aided Edge Server Selection Mechanism for Edge Task OffloadingIEEE Transactions on Vehicular Technology10.1109/TVT.2023.331316273:2(2506-2519)Online publication date: Feb-2024
https://doi.org/10.1109/TVT.2023.3313162
Xu ZLi DLiang WXu WXia QZhou PRana OLi H(2024)Energy or Accuracy? Near-Optimal User Selection and Aggregator Placement for Federated Learning in MECIEEE Transactions on Mobile Computing10.1109/TMC.2023.326282923:3(2470-2485)Online publication date: Mar-2024
https://doi.org/10.1109/TMC.2023.3262829
Show More Cited By

Index Terms

Computation Offloading with Multiple Agents in Edge-Computing–Supported IoT

Recommendations

Federated learning-based computation offloading optimization in edge computing-supported internet of things
ACM TURC '19: Proceedings of the ACM Turing Celebration Conference - China

Recent visualizations of smart cities, factories, healthcare system and etc. raise challenges on the capability and connectivity of massive Internet of Things (IoT) devices. Hence, edge computing is emerged to complement these capability-constrained ...
Enabling Fine-Grained Edge Offloading for IoT
SIGCOMM Posters and Demos '17: Proceedings of the SIGCOMM Posters and Demos

In this paper we make the case for IoT edge offloading, which strives to exploit the resources on edge computing devices by offloading fine-grained computation tasks from the cloud closer to the users and data generators (i.e., IoT devices). The key ...
Incentive mechanism for computation offloading using edge computing

IoT-based services benefit from cloud which offers a virtually unlimited capabilities, such as storage, processing, and communication. However, the challenges are still open for mobile users to receive computation from the cloud with satisfied quality-...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 16, Issue 1

February 2020

351 pages

ISSN:1550-4859

EISSN:1550-4867

DOI:10.1145/3368392

Editor:
Yunhao Liu
Tsinghua University, China

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 19 December 2019

Revised: 01 October 2019

Received: 01 September 2019

Accepted: 01 October 2018

Published in TOSN Volume 16, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

The Opening Fund of Acoustics Science and Technology Laboratory
China NSFC (Youth)
The National Key R&D Program of China
Huawei Innovation Research Program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

64
Total Citations
View Citations
1,432
Total Downloads

Downloads (Last 12 months)114
Downloads (Last 6 weeks)26

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yaraziz MSohrabi Safa NAzad M(2024)Edge computing in IoT for smart healthcareJournal of Ambient Intelligence and Smart Environments10.3233/AIS-230009(1-30)Online publication date: 19-Jun-2024
https://doi.org/10.3233/AIS-230009
Xue HZhang DWu CJi YLong SWang CSato T(2024)Parameter Estimation-Aided Edge Server Selection Mechanism for Edge Task OffloadingIEEE Transactions on Vehicular Technology10.1109/TVT.2023.331316273:2(2506-2519)Online publication date: Feb-2024
https://doi.org/10.1109/TVT.2023.3313162
Xu ZLi DLiang WXu WXia QZhou PRana OLi H(2024)Energy or Accuracy? Near-Optimal User Selection and Aggregator Placement for Federated Learning in MECIEEE Transactions on Mobile Computing10.1109/TMC.2023.326282923:3(2470-2485)Online publication date: Mar-2024
https://doi.org/10.1109/TMC.2023.3262829
Wang WZhang YLiu QWang TJia W(2024)Edge-Intelligence-Based Computation Offloading Technology for Distributed Internet of Unmanned Aerial VehiclesIEEE Internet of Things Journal10.1109/JIOT.2024.338389611:12(20948-20957)Online publication date: 15-Jun-2024
https://doi.org/10.1109/JIOT.2024.3383896
Peng PLin WWu WZhang HPeng SWu QLi K(2024)A survey on computation offloading in edge systems: From the perspective of deep reinforcement learning approachesComputer Science Review10.1016/j.cosrev.2024.10065653(100656)Online publication date: Aug-2024
https://doi.org/10.1016/j.cosrev.2024.100656
Liu WLei JYi PHong Y(2024)No-regret learning for repeated non-cooperative games with lossy banditsAutomatica10.1016/j.automatica.2023.111455160(111455)Online publication date: Feb-2024
https://doi.org/10.1016/j.automatica.2023.111455
Zhang HZhang JWang RHuang YZhang MShang XGao C(2024)Smart carbon monitoring platform under IoT-Cloud architecture for small cities in B5GWireless Networks10.1007/s11276-021-02756-230:5(3837-3853)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s11276-021-02756-2
Mishra RGupta H(2023)A Model Personalization-based Federated Learning Approach for Heterogeneous Participants with Variability in the DatasetACM Transactions on Sensor Networks10.1145/362997820:1(1-28)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3629978
Watt TChrysoulas CGkatzia D(2023)Edge NLP for Efficient Machine Translation in Low Connectivity Areas2023 IEEE 9th World Forum on Internet of Things (WF-IoT)10.1109/WF-IoT58464.2023.10539577(1-6)Online publication date: 12-Oct-2023
https://doi.org/10.1109/WF-IoT58464.2023.10539577
Fan WYang FWang PMiao MZhao PHuang T(2023)DRL-Based Service Function Chain Edge-to-Edge and Edge-to-Cloud Joint Offloading in Edge-Cloud NetworkIEEE Transactions on Network and Service Management10.1109/TNSM.2023.327176920:4(4478-4493)Online publication date: Dec-2023
https://doi.org/10.1109/TNSM.2023.3271769
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents