research-article

DeepSave: saving DNN inference during handovers on the edge

Authors:

Bing Bing ZhouAuthors Info & Claims

SEC '19: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing

Pages 166 - 178

https://doi.org/10.1145/3318216.3363301

Published: 07 November 2019 Publication History

Abstract

Recent advances in deep neural networks (DNNs) have substantially improved the accuracy and speed of a variety of intelligent applications, for example, real-time video classification. However, one of the challenges is how to maintain the quality of service during handovers to avoid interruptions. Inspired by the recently developed DNN partition schemes, where the DNN model inference can be partitioned and jointly processed at a mobile device and its connected edge-computing server, we propose DeepSave, a promising solution to save a large portion of consecutive video frames that cannot be handled during handovers¹. DeepSave comprises two subschemes: (1) The Frame Choosing Scheme is to determine which frames we should save during a handover, to maximize the number of saved frames while preserving the accuracy of the inferences. (2) The Last Arriving Frame Repartition Scheme, with a provable performance bound, is to handle the last frame before the end of the handover as soon as possible, so that the arriving frames after the handover can be processed as usual without causing congestion. We have built up a real-world prototype and conducted field experiments and extensive simulations, showing that DeepSave can save up to 50.98% frames during handovers, which is much more than the benchmark schemes.

References

[1]

H. Badri, T. Bahreini, D. Grosu, and K. Yang. 2018. Risk-Based Optimization of Resource Provisioning in Mobile Edge Computing. In IEEE/ACM Symposium on Edge Computing (SEC). Bellevue, WA, 328--330.

[2]

W. Bao, D. Yuan, Z. Yang, S. Wang, W. Li, B. B. Zhou, and A. Y. Zomaya. 2017. Follow Me Fog: Toward Seamless Handover Timing Schemes in a Fog Computing Environment. IEEE Communications Magazine 55, 11 (Nov 2017), 72--78.

[3]

L. Chaufournier, P. Sharma, F. Le, E. Nahum, P. Shenoy, and D. Towsley. 2017. Fast Transparent Virtual Machine Migration in Distributed Edge Clouds. In ACM/IEEE Symposium on Edge Computing (SEC). San Jose, CA.

[4]

Jianguo Chen, Kenli Li, Qingying Deng, Keqin Li, and S Yu Philip. 2019. Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing. IEEE Transactions on Industrial Informatics (2019), 1--1.

[5]

Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. 2009. Introduction to algorithms. MIT press.

[6]

Eduardo Cuervo, Aruna Balasubramanian, Dae-ki Cho, Alec Wolman, Stefan Saroiu, Ranveer Chandra, and Paramvir Bahl. 2010. MAUI: making smartphones last longer with code offload. In Proc. ACM MobiSys'10. San Francisco, CA.

Digital Library

[7]

Android Developer. [n. d.]. CONNECTIVITY_ACTION. Retrieved August 11, 2019 from https://developer.android.com/reference/android/net/ConnectivityManager.html#CONNECTIVITY_ACTION

[8]

Liming Ge. 2019. Frame Saving Scheme Prototype Implementation. Retrieved August 11, 2019 from https://greg308.github.io/frame_retention_for_video_recognition/

[9]

K. Ha, Y. Abe, T. Eiszler, Z. Chen, W. Hu, B. Amos, R. Upadhyaya, P. Pillai, and M. Satyanarayanan. 2017. You Can Teach Elephants to Dance: Agile VM Handoff for Edge Computing. In ACM/IEEE Symposium on Edge Computing (SEC). San Jose, CA.

[10]

C. Hu, W. Bao, D. Wang, and F. Liu. 2018. Dynamic Adaptive DNN Surgery for Inference Acceleration on the Edge. In Proceedings of the IEEE INFOCOM. Paris, France.

[11]

H. Jeong, I. Jeong, H. Lee, and S. Moon. 2018. Computation Offloading for Machine Learning Web Apps in the Edge Server Environment. In IEEE ICDCS. Vienna, Austria, 1492--1499.

[12]

Hyuk-Jin Jeong, Hyeon-Jae Lee, Chang Hyun Shin, and Soo-Mook Moon. 2018. IONN: Incremental Offloading of Neural Network Computations from Mobile Devices to Edge Servers. In Proceedings of the ACM Symposium on Cloud Computing. ACM, Carlsbad, CA, 401--411.

Digital Library

[13]

Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. 2017. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. In Proc. ACM ASPLOS'17. Xi'an, China.

Digital Library

[14]

Y. Liu, S. Wang, and F. Yang. 2016. Poster Abstract: A Multi-user Computation Offloading Algorithm Based on Game Theory in Mobile Cloud Computing. In IEEE/ACM Symposium on Edge Computing (SEC). Washington DC, USA, 93--94.

[15]

L. Ma, S. Yi, and Q. Li. 2017. Efficient Service Handoff Across Edge Servers via Docker Container Migration. In ACM/IEEE Symposium on Edge Computing (SEC). San Jose, CA.

[16]

Michael Nelson, Beng-Hong Lim, and Greg Hutchins. 2005. Fast transparent migration for virtual machines. In Proc. of USENIX Annual Technical Conference. Anaheim, CA.

Digital Library

[17]

Maria Rita Palattella, Ridha Soua, Abdelmajid Khelil, and Thomas Engel. 2019. Fog computing as the key for seamless connectivity handover in future vehicular networks. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing. ACM, Limassol Cyprus, 1996--2000.

Digital Library

[18]

C. Pei, Z. Wang, Y. Zhao, Z. Wang, Y. Meng, D. Pei, Y. Peng, W. Tang, and X. Qu. 2017. Why it takes so long to connect to a WiFi access point. In IEEE INFOCOM 2017 - IEEE Conference on Computer Communications. Atlanta, GA, 1--9.

[19]

G.P. Pollini. 1996. Trends in handover design. IEEE Communications Magazine 34, 3 (Mar. 1996), 82--90.

Digital Library

[20]

Xukan Ran, Haoliang Chen, Xiaodan Zhu, Zhenming Liu, and Jiasi Chen. 2018. DeepDecision: A Mobile Deep Learning Framework for Edge Video Analytics. In INFOCOM. IEEE. Honolulu, HI.

[21]

P. Ren, X. Qiao, J. Chen, and S. Dustdar. 2018. Mobile Edge Computing - a Booster for the Practical Provisioning Approach of Web-Based Augmented Reality. In IEEE/ACM Symposium on Edge Computing (SEC). Honolulu, HI, 349--350.

[22]

A. Samanta and Y. Li. 2018. Latency-Oblivious Incentive Service Offloading in Mobile Edge Computing. In IEEE/ACM Symposium on Edge Computing (SEC). Honolulu, HI, 351--353.

[23]

Mechthild Stoer and Frank Wagner. 1997. A simple min-cut algorithm. Journal of the ACM (JACM) 44, 4 (1997), 585--591.

Digital Library

[24]

Tarik Taleb and Adlen Ksentini. 2013. Follow Me Cloud: interworking federated clouds and distributed mobile networks. IEEE Network 27, 5 (Sep.-Oct. 2013), 12--19.

[25]

Surat Teerapittayanon, Bradley McDanel, and HT Kung. 2017. Distributed deep neural networks over the cloud, the edge and end devices. In IEEE ICDCS. Atlanta, GA.

[26]

Z. Zhao, K. M. Barijough, and A. Gerstlauer. 2018. DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 37, 11 (Nov 2018), 2348--2359.

Cited By

Yuan XLi NWei KXu WChen QChen HGuo S(2025)Mobility and Cost Aware Inference Accelerating Algorithm for Edge IntelligenceIEEE Transactions on Mobile Computing10.1109/TMC.2024.348415824:3(1530-1549)Online publication date: Mar-2025
https://doi.org/10.1109/TMC.2024.3484158
Wu JWang LJin QLiu F(2024)Graft: Efficient Inference Serving for Hybrid Deep Learning With SLO Guarantees via DNN Re-AlignmentIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.334051835:2(280-296)Online publication date: Feb-2024
https://doi.org/10.1109/TPDS.2023.3340518
Zheng YCui LTso FLi ZJia W(2024)DNN acceleration in vehicle edge computing with mobility-awarenessComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110607251:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110607
Show More Cited By

Index Terms

DeepSave: saving DNN inference during handovers on the edge
1. Networks
  1. Network performance evaluation
    1. Network performance modeling

Recommendations

eDeepSave: Saving DNN Inference using Early Exit During Handovers in Mobile Edge Environment
Recent advances in deep neural networks (DNNs) have substantially improved the accuracy of intelligent applications. One effective scheme known as DNN partition further improves the speed of the inference by partitioning the DNN to a mobile device and its ...
Fast and Secure Reauthentications for 3GPP Subscribers during WiMAX-WLAN Handovers

Wireless technologies such as the Wireless Local Area Network (WLAN), the Worldwide Interoperability for Microwave Access (WiMAX), and the Third-Generation (3G) mobile communications system complement each other to support a variety of services suited ...
Energy-efficient network selection with mobility pattern awareness in an integrated WiMAX and WiFi network

To provide wireless Internet access, WiFi networks have been deployed in many regions such as buildings and campuses. However, WiFi networks are still insufficient to support ubiquitous wireless service due to their narrow coverage. One possibility to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SEC '19: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing

November 2019

455 pages

ISBN:9781450367332

DOI:10.1145/3318216

General Chairs:
Songqing Chen
George Mason University
,
Ryokichi Onishi
Toyota
,
Program Chairs:
Ganesh Ananthanarayanan
Microsoft Research
,
Qun Li
College of William & Mary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

IEEE-CS\DATC: IEEE Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SEC '19

Sponsor:

SIGMOBILE

SEC '19: The Fourth ACM/IEEE Symposium on Edge Computing

November 7 - 9, 2019

Virginia, Arlington

Acceptance Rates

SEC '19 Paper Acceptance Rate 20 of 59 submissions, 34%;

Overall Acceptance Rate 40 of 100 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
712
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yuan XLi NWei KXu WChen QChen HGuo S(2025)Mobility and Cost Aware Inference Accelerating Algorithm for Edge IntelligenceIEEE Transactions on Mobile Computing10.1109/TMC.2024.348415824:3(1530-1549)Online publication date: Mar-2025
https://doi.org/10.1109/TMC.2024.3484158
Wu JWang LJin QLiu F(2024)Graft: Efficient Inference Serving for Hybrid Deep Learning With SLO Guarantees via DNN Re-AlignmentIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.334051835:2(280-296)Online publication date: Feb-2024
https://doi.org/10.1109/TPDS.2023.3340518
Zheng YCui LTso FLi ZJia W(2024)DNN acceleration in vehicle edge computing with mobility-awarenessComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110607251:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110607
Xu RRazavi SZheng R(2023)Edge Video Analytics: A Survey on Applications, Systems and Enabling TechniquesIEEE Communications Surveys & Tutorials10.1109/COMST.2023.332309125:4(2951-2982)Online publication date: Dec-2024
https://doi.org/10.1109/COMST.2023.3323091
Xue MWu HPeng GWolter K(2022)DDPQN: An Efficient DNN Offloading Strategy in Local-Edge-Cloud Collaborative EnvironmentsIEEE Transactions on Services Computing10.1109/TSC.2021.311659715:2(640-655)Online publication date: 1-Mar-2022
https://doi.org/10.1109/TSC.2021.3116597
Xie APeng Y(2022)Improving the Quality of Inference for Applications using Chained DNN Models during Edge Server Handover2022 IEEE/ACM 7th Symposium on Edge Computing (SEC)10.1109/SEC54971.2022.00079(516-520)Online publication date: Dec-2022
https://doi.org/10.1109/SEC54971.2022.00079
Ju WYuan DBao WGe LZhou B(2021)eDeepSave: Saving DNN Inference using Early Exit During Handovers in Mobile Edge EnvironmentACM Transactions on Sensor Networks10.1145/344726717:3(1-28)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3447267
Dong CHu SChen XWen W(2021)Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge ComputingIEEE Transactions on Network and Service Management10.1109/TNSM.2021.311666518:4(3973-3986)Online publication date: Dec-2021
https://doi.org/10.1109/TNSM.2021.3116665

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten