research-article

Edge Learning: The Enabling Technology for Distributed Big Data Analytics in the Edge

Authors:

Song GuoAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 54, Issue 7

Article No.: 151, Pages 1 - 36

https://doi.org/10.1145/3464419

Published: 18 July 2021 Publication History

Abstract

Machine Learning (ML) has demonstrated great promise in various fields, e.g., self-driving, smart city, which are fundamentally altering the way individuals and organizations live, work, and interact. Traditional centralized learning frameworks require uploading all training data from different sources to a remote data server, which incurs significant communication overhead, service latency, and privacy issues.

To further extend the frontiers of the learning paradigm, a new learning concept, namely, Edge Learning (EL) is emerging. It is complementary to the cloud-based methods for big data analytics by enabling distributed edge nodes to cooperatively training models and conduct inferences with their locally cached data. To explore the new characteristics and potential prospects of EL, we conduct a comprehensive survey of the recent research efforts on EL. Specifically, we first introduce the background and motivation. We then discuss the challenging issues in EL from the aspects of data, computation, and communication. Furthermore, we provide an overview of the enabling technologies for EL, including model training, inference, security guarantee, privacy protection, and incentive mechanism. Finally, we discuss future research opportunities on EL. We believe that this survey will provide a comprehensive overview of EL and stimulate fruitful future research in this field.

References

[1]

Naman Agarwal and Karan Singh. 2017. The price of differential privacy for online learning. In Proc. of ICML.

[2]

Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. 2017. QSGD: Communication-efficient SGD via gradient quantization and encoding. In Proc. of NeurIPS. 1709–1720.

[3]

Dan Alistarh, Torsten Hoefler, Mikael Johansson, Nikola Konstantinov, Sarit Khirirat, and Cédric Renggli. 2018. The convergence of sparsified gradient methods. In Proc. of NeurIPS. 5973–5983.

[4]

Mohammad Mohammadi Amiri and Deniz Gündüz. 2019. Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air. In Proc. of ISIT. IEEE, 1432–1436.

Digital Library

[5]

Ron Banner, Itay Hubara, Elad Hoffer, and Daniel Soudry. 2018. Scalable methods for 8-bit training of neural networks. In Proc. of NeurIPS. 5151–5159.

[6]

Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. 2018. Analyzing federated learning through an adversarial lens. Proc. of ICML 97 (2019), 634–643.

[7]

Peva Blanchard, Rachid Guerraoui, Julien Stainer, et al. 2017. Machine learning with adversaries: Byzantine tolerant gradient descent. In Proc. of NeurIPS. 119–129.

[8]

Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Practical secure aggregation for privacy-preserving machine learning. In Proc. of CCS. 1175–1191.

Digital Library

[9]

Alon Brutzkus, Oren Elisha, and Ran Gilad-Bachrach. 2018. Low latency privacy preserving inference. CoRR abs/1812.10659 (2018). arxiv:1812.10659.http://arxiv.org/abs/1812.10659.

[10]

Christopher Canel, Thomas Kim, Giulio Zhou, Conglong Li, Hyeontaek Lim, David G. Andersen, Michael Kaminsky, and Subramanya R. Dulloor. 2019. Scaling video analytics on constrained edge nodes. arxiv:1905.13536.http://arxiv.org/abs/1905.13536.

[11]

Chen Chen, Qizhen Weng, Wei Wang, Baochun Li, and Bo Li. 2018. Fast distributed deep learning via worker-adaptive batch sizing. In Proc. of SoCC. 521–521.

Digital Library

[12]

Chia-Yu Chen, Jungwook Choi, Daniel Brand, Ankur Agrawal, Wei Zhang, and Kailash Gopalakrishnan. 2018. Adacomp: Adaptive residual gradient compression for data-parallel distributed training. In Proc. of AAAI. 2827–2835.

[13]

Jiasi Chen and Xukan Ran. 2019. Deep learning with edge computing: A review. Proc. IEEE 107, 8 (2019), 1655–1674.

[14]

Lingjiao Chen, Hongyi Wang, Zachary Charles, and Dimitris Papailiopoulos. 2018. Draco: Byzantine-resilient distributed training via redundant gradients. Proc. of ICML 80 (2018), 902–911.

[15]

Tianyi Chen, Georgios Giannakis, Tao Sun, and Wotao Yin. 2018. LAG: Lazily aggregated gradient for communication-efficient distributed learning. In Proc. of NeurIPS. 5050–5060.

[16]

Yudong Chen, Lili Su, and Jiaming Xu. 2017. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems 1, 2 (2017), 44:1–44:25.

Digital Library

[17]

Yen-Kuang Chen, An-Yeu Wu, Magdy A. Bayoumi, and Farinaz Koushanfar. 2013. Editorial low-power, intelligent, and secure solutions for realization of Internet of Things. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 3, 1 (2013), 1–4.

[18]

Ting-Wu Chin, Ruizhou Ding, and Diana Marculescu. 2019. Adascale: Towards real-time video object detection using adaptive scaling. arXiv:1902.02910. http://arxiv.org/abs/1902.02910.

[19]

Yi-Min Chou, Yi-Ming Chan, Jia-Hong Lee, Chih-Yi Chiu, and Chu-Song Chen. 2018. Unifying and merging well-trained deep neural networks for inference stage. arXiv:1805.04980. http://arxiv.org/abs/1805.04980.

[20]

James Cipar, Qirong Ho, Jin Kyu Kim, Seunghak Lee, Gregory R. Ganger, Garth Gibson, Kimberly Keeton, and Eric Xing. 2013. Solving the straggler problem with bounded staleness. Presented as part of the 14th Workshop on Hot Topics in Operating Systems.

[21]

Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc’aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, et al. 2012. Large scale distributed deep networks. In Proc. of NeurIPS. 1223–1231.

[22]

Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. 2014. SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. Proc. of NeurIPS 2, 1646–1654.

[23]

Shuiguang Deng, Hailiang Zhao, Weijia Fang, Jianwei Yin, Schahram Dustdar, and Albert Y. Zomaya. 2020. Edge intelligence: The confluence of edge computing and artificial intelligence. IEEE Internet of Things Journal 7, 8 (Aug. 2020), 7457–7469.

[24]

Don Kurian Dennis, Chirag Pabbaraju, Harsha Vardhan Simhadri, and Prateek Jain. 2018. Multiple instance learning for efficient sequential data classification on resource-constrained devices. In Proc. of NeurIPS. 10976–10987.

[25]

Emily L. Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus. 2014. Exploiting linear structure within convolutional networks for efficient evaluation. In Proc. of NeurIPS. 1269–1277.

[26]

Guneet S. Dhillon, Kamyar Azizzadenesheli, Zachary C. Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. 2018. Stochastic activation pruning for robust adversarial defense. CoRR abs/1803.01442 (2018). arxiv:1803.01442.http://arxiv.org/abs/1803.01442.

[27]

Xiao Dong, Lei Liu, Guangli Li, Jiansong Li, Peng Zhao, Xueying Wang, and Xiaobing Feng. 2019. Exploiting the input sparsity to accelerate deep neural networks: Poster. In Proc. of PPoPP. ACM, 401–402.

Digital Library

[28]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12 (July 2011), 2121–2159.

Digital Library

[29]

Shaohan Feng, Dusit Niyato, Ping Wang, Dong In Kim, and Ying-Chang Liang. 2019. Joint service pricing and cooperative relay communication for federated learning. In Proc. of IEEE iThings/GreenCom/CPSCom/SmartData. 815–820.

[30]

Jason Ge, Zhaoran Wang, Mengdi Wang, and Han Liu. 2018. Minimax-optimal privacy-preserving sparse PCA in distributed systems. In Proc. of AISTATS, Vol. 84. PMLR, 1589–1598.

[31]

Saeed Ghadimi and Guanghui Lan. 2013. Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM Journal on Optimization 23, 4 (2013), 2341–2368.

Digital Library

[32]

Irene Giacomelli, Somesh Jha, Marc Joye, C. David Page, and Kyonghwan Yoon. 2018. Privacy-preserving ridge regression with only linearly-homomorphic encryption. In Applied Cryptography and Network Security.

[33]

Dibakar Gope, Ganesh Dasika, and Matthew Mattina. 2019. Ternary hybrid neural-tree networks for highly constrained IoT applications. arxiv:cs.LG/1903.01531. http://arxiv.org/abs/1903.01531.

[34]

Renjie Gu, Shuo Yang, and Fan Wu. 2019. Distributed machine learning on mobile devices: A survey. arXiv:1909.08329. http://arxiv.org/abs/1909.08329.

[35]

Peizhen Guo, Bo Hu, Rui Li, and Wenjun Hu. 2018. Foggycache: Cross-device approximate computation reuse. In Proc. of MobiCom. ACM, 19–34.

Digital Library

[36]

Yiwen Guo, Chao Zhang, Changshui Zhang, and Yurong Chen. 2018. Sparse DNNs with improved adversarial robustness. In Proc. of NeurIPS.

[37]

Yundi Guo, Beiji Zou, Ju Ren, Qingqing Liu, Deyu Zhang, and Yaoxue Zhang. 2019. Distributed and efficient object detection via interactions among devices, edge, and cloud. IEEE Transactions on Multimedia 21, 11 (2019), 2903–2915.

[38]

Rob Hall, Stephen E. Fienberg, and Yuval Nardi. 2011. Secure multiple linear regression based on homomorphic encryption. Journal of Official Statistics 27, 4 (2011), 669–691.

[39]

Song Han, Huizi Mao, and William J. Dally. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv:1510.00149. https://arxiv.org/abs/1510.00149.

[40]

Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. In Proc. of NeurIPS. 1135–1143.

[41]

Warren He, Bo Li, and Dawn Song. 2018. Decision boundary analysis of adversarial examples. In Proc. of ICLR.

[42]

Mikko Heikkilä, Eemil Lagerspetz, Samuel Kaski, Kana Shimizu, Sasu Tarkoma, and Antti Honkela. 2017. Differentially private Bayesian learning on distributed data. In Proc. of NeurIPS. 3226–3235.

[43]

Ehsan Hesamifard, Hassan Takabi, and Mehdi Ghasemi. 2017. CryptoDL: Deep neural networks over encrypted data. arXiv:1711.05189. http://arxiv.org/abs/1711.05189.

[44]

Geoffrey Hinton, N. Srivastava, K. Swersky, T. Tieleman, and A. R. Mohamed. 2012. COURSERA: Neural networks for machine learning. Lecture 9c: Using noise as a regularizer.

[45]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv:1503.02531. http://arxiv.org/abs/1503.02531.

[46]

Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. 2017. Deep models under the GAN: Information leakage from collaborative deep learning. In Proc. of CCS. 603–618.

Digital Library

[47]

Connor Holmes, Daniel Mawhirter, Yuxiong He, Feng Yan, and Bo Wu. 2019. GRNN: Low-latency and scalable RNN inference on GPUs. In Proc. of EuroSys. ACM, 41:1–41:16.

[48]

Junxian Huang, Feng Qian, Yihua Guo, Yuanyuan Zhou, Qiang Xu, Z. Morley Mao, Subhabrata Sen, and Oliver Spatscheck. 2013. An in-depth study of LTE: Effect of network protocol and application behavior on performance. ACM SIGCOMM Computer Communication Review 43, 4 (2013), 363–374.

Digital Library

[49]

Li Huang, Yifeng Yin, Zeng Fu, Shifa Zhang, Hao Deng, and Dianbo Liu. 2018. Loadaboost: Loss-based adaboost federated machine learning on medical data. arXiv:1811.12629. http://arxiv.org/abs/1811.12629.

[50]

Yuzhen Huang, Tatiana Jin, Yidi Wu, Zhenkun Cai, Xiao Yan, Fan Yang, Jinfeng Li, Yuying Guo, and James Cheng. 2018. Flexps: Flexible parallelism control in parameter server architecture. Proceedings of the VLDB Endowment 11, 5 (2018), 566–579.

Digital Library

[51]

Matthew Jagielski, Alina Oprea, Battista Biggio, Chang Liu, Cristina Nita-Rotaru, and Bo Li. 2018. Manipulating machine learning: Poisoning attacks and countermeasures for regression learning. In 2018 IEEE Symposium on Security and Privacy (SP’18). IEEE, 19–35.

[52]

Bargav Jayaraman, Lingxiao Wang, David Evans, and Quanquan Gu. 2018. Distributed learning without distress: Privacy-preserving empirical risk minimization. In Proc. of NeurIPS. 6343–6354.

[53]

Hyuk-Jin Jeong, Hyeon-Jae Lee, Chang Hyun Shin, and Soo-Mook Moon. 2018. IONN: Incremental offloading of neural network computations from mobile devices to edge servers. In Proc. of SoCC. 401–411.

Digital Library

[54]

Yujie Ji, Xinyang Zhang, Shouling Ji, Xiapu Luo, and Ting Wang. 2018. Model-reuse attacks on deep learning systems. In Proc. of CCS. ACM, 349–363.

Digital Library

[55]

Jiawei Jiang, Bin Cui, Ce Zhang, and Lele Yu. 2017. Heterogeneity-aware distributed parameter servers. In Proc. of SIGMOD. ACM, 463–478.

Digital Library

[56]

Sian Jin, Sheng Di, Xin Liang, Jiannan Tian, Dingwen Tao, and Franck Cappello. 2019. DeepSZ: A novel framework to compress deep neural networks by using error-bounded lossy compression. In Proc. of HPDC. 159–170.

Digital Library

[57]

Aarati Kakaraparthy, Abhay Venkatesh, Amar Phanishayee, and Shivaram Venkataraman. 2019. The case for unifying data loading in machine learning clusters. In 11th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud’19).

Digital Library

[58]

Youngsok Kim, Joonsung Kim, Dongju Chae, Daehyun Kim, and Jangwoo Kim. 2019. Layer: Low latency on-device inference using cooperative single-layer acceleration and processor-friendly quantization. In Proc. of EuroSys. 1–15.

Digital Library

[59]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. http://arxiv.org/abs/1811.12629.

[60]

Jakub Konečnỳ and Peter Richtárik. 2018. Randomized distributed mean estimation: Accuracy vs communication. Frontiers in Applied Mathematics and Statistics 4 (2018), 62.

[61]

Lei Lei, Yue Tan, Shiwen Liu, Kan Zheng, et al. 2019. Deep reinforcement learning for autonomous internet of things: Model, applications and challenges. arXiv:1907.09059. http://arxiv.org/abs/1907.09059.

[62]

Mu Li, David G. Andersen, Jun Woo Park, Alexander J. Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J. Shekita, and Bor-Yiing Su. 2014. Scaling distributed machine learning with the parameter server. In Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation (OSDI’14), 583–598.

Digital Library

[63]

Peng Li and Song Guo. 2015. Incentive mechanisms for device-to-device communications. IEEE Network 29, 4 (2015), 75–79.

Digital Library

[64]

Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2019. Federated learning: Challenges, methods, and future directions. arXiv:1908.07873. http://arxiv.org/abs/1908.07873.

[65]

Yuanzhi Li, Tengyu Ma, and Hongyang Zhang. 2017. Algorithmic regularization in over-parameterized matrix sensing and neural networks with quadratic activations. arXiv:1712.09203. https://arxiv.org/abs/1712.09203.

[66]

Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. 2017. Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In Proc. of NeurIPS. 5330–5340.

[67]

Katrina Ligett, Seth Neel, Aaron Roth, Bo Waggoner, and Steven Z. Wu. 2017. Accuracy first: Selecting a differential privacy level for accuracy constrained ERM. In Proc. of NeurIPS. 2566–2576.

[68]

Wei Yang Bryan Lim, Nguyen Cong Luong, Dinh Thai Hoang, Yutao Jiao, Ying-Chang Liang, Qiang Yang, Dusit Niyato, and Chunyan Miao. 2019. Federated learning in mobile edge networks: A comprehensive survey. arXiv:1909.11875. http://arxiv.org/abs/1909.11875.

[69]

Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J. Dally. 2017. Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv:1712.01887. http://arxiv.org/abs/1712.01887.

[70]

Fang Liu and Ness Shroff. 2019. Data poisoning attacks on stochastic bandits. arXiv:1905.06494. http://arxiv.org/abs/1905.06494.

[71]

Yang Liu, Changqiao Xu, Yufeng Zhan, Zhixin Liu, Jianfeng Guan, and Hongke Zhang. 2017. Incentive mechanism for computation offloading using edge computing: A Stackelberg game approach. Computer Networks 129 (2017), 399–409.

Digital Library

[72]

Qinyi Luo, Jinkun Lin, Youwei Zhuo, and Xuehai Qian. 2019. Hop: Heterogeneity-aware decentralized training. In Proc. of ASPLOS. 893–907.

Digital Library

[73]

Wang Luping, Wang Wei, and Li Bo. 2019. CMFL: Mitigating communication overhead for federated learning. In Proc. of ICDCS. IEEE, 954–964.

[74]

Pavel Mach and Zdenek Becvar. 2017. Mobile edge computing: A survey on architecture and computation offloading. IEEE Communications Surveys & Tutorials 19, 3 (2017), 1628–1656.

Digital Library

[75]

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arxiv:stat.ML/1706.06083. http://arxiv.org/abs/1706.06083.

[76]

Saeed Mahloujifar, Mohammad Mahmoody, and Ameer Mohammed. 2019. Data poisoning attacks in multi-party learning. In Proc. of ICML. 4274–4283.

[77]

Charles H. Martin and Michael W. Mahoney. 2018. Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning. CoRR abs/1810.01075 (2018). http://arxiv.org/abs/1810.01075.

[78]

H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. 2016. Communication-efficient learning of deep networks from decentralized data. arXiv:1602.05629. https://arxiv.org/abs/1602.05629.

[79]

H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. 2017. Communication-efficient learning of deep networks from decentralized data. In Proc. of AISTATS.

[80]

H. Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. 2017. Learning differentially private language models without losing accuracy. CoRR abs/1710.06963 (2017). arxiv:1710.06963.http://arxiv.org/abs/1710.06963.

[81]

Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2019. Exploiting unintended feature leakage in collaborative learning. In IEEE Symposium on Security and Privacy. IEEE, 691–706.

[82]

Mehdi Mohammadi, Ala Al-Fuqaha, Sameh Sorour, and Mohsen Guizani. 2018. Deep learning for IoT big data and streaming analytics: A survey. IEEE Communications Surveys & Tutorials 20, 4 (2018), 2923–2960.

Digital Library

[83]

Amir-Hamed Mohsenian-Rad, Vincent W. S. Wong, Juri Jatskevich, and Robert Schober. 2010. Optimal and autonomous incentive-based energy consumption scheduling algorithm for smart grid. In Proc. of IEEE ISGT. 1–6.

[84]

Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan, et al. 2018. Ray: A distributed framework for emerging applications. In Proc. of OSDI. 561–577.

[85]

M. G. Sarwar Murshed, Christopher Murphy, Daqing Hou, Nazar Khan, Ganesh Ananthanarayanan, and Faraz Hussain. 2019. Machine learning at the network edge: A survey. CoRR abs/1908.00080. http://arxiv.org/abs/1908.00080.

[86]

Supun Nakandala, Arun Kumar, and Yannis Papakonstantinou. 2019. Incremental and approximate inference for faster occlusion-based deep CNN explanations. In Proc. of SIGMOD. 1589–1606.

Digital Library

[87]

M. Nasr, R. Shokri, and A. Houmansadr. 2019. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE Symposium on Security and Privacy (SP’19). 739–753. https://doi.org/10.1109/SP.2019.00065

[88]

Solmaz Niknam, Harpreet S. Dhillon, and Jeffery H. Reed. 2019. Federated learning for wireless communications: Motivation, opportunities and challenges. arXiv:1908.06847. http://arxiv.org/abs/1908.06847.

[89]

Takayuki Nishio and Ryo Yonetani. 2019. Client selection for federated learning with heterogeneous resources in mobile edge. In Proc. ICC. IEEE, 1–7.

[90]

Jianmin Chen, Rajat Monga, Samy Bengio, and Rafal Józefowicz. 2016. Revisiting Distributed Synchronous (SGD). CoRR abs/1604.0098. http://arxiv.org/abs/1604.00981.

[91]

Yupeng Zhang Payman Mohassel. 2017. SecureML: A system for scalable privacy-preserving machine learning. In 2017 IEEE Symposium on Security and Privacy (SP’17). IEEE. https://doi.org/10.1109/SP.2017.12

[92]

Ju Ren, Hui Guo, Chugui Xu, and Yaoxue Zhang. 2017. Serving at the Edge: A scalable IoT architecture based on transparent computing. IEEE Network 31, 5 (2017), 96–105.

Digital Library

[93]

Ju Ren, Deyu Zhang, Shiwen He, Yaoxue Zhang, and Tao Li. 2020. A survey on end-edge-cloud orchestrated network computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet. ACM Computing Surveys 52, 6 (2020), 1–36.

Digital Library

[94]

Nicolas L. Roux, Mark Schmidt, and Francis R. Bach. 2012. A stochastic gradient method with an exponential convergence _rate for finite training sets. In Proc. of NeurIPS. 2663–2671.

[95]

Yunus Sarikaya and Ozgur Ercetin. 2019. Motivating workers in federated learning: A Stackelberg game perspective. IEEE Networking Letters 2, 1 (2019), 23–27. https://doi.org/10.1109/LNET.2019.2947144arxiv:1908.03092

[96]

Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, and Wojciech Samek. 2019. Robust and communication-efficient federated learning from non-iid data. arXiv:1903.02891. http://arxiv.org/abs/1903.02891.

[97]

Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge computing: Vision and challenges. IEEE Internet of Things Journal 3, 5 (2016), 637–646.

[98]

A. Smith, A. Thakurta, and J. Upadhyay. 2017. Is interaction necessary for distributed private learning? In 2017 IEEE Symposium on Security and Privacy (SP’17).

[99]

Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov. 2017. Machine learning models that remember too much. In Proc. of CCS.

Digital Library

[100]

Michael R. Sprague, Amir Jalalirad, Marco Scavuzzo, Catalin Capota, Moritz Neun, Lyman Do, and Michael Kopp. 2018. Asynchronous federated learning for geospatial applications. In Proc. of ECML PKDD. Springer, 21–28.

[101]

Jacob Steinhardt, Pang Wei W. Koh, and Percy S. Liang. 2017. Certified defenses for data poisoning attacks. In Proc. of NeurIPS. 3517–3529.

[102]

Sebastian U. Stich, Jean-Baptiste Cordonnier, and Martin Jaggi. 2018. Sparsified SGD with memory. In Proc. of NeurIPS. 4447–4458.

[103]

Jun Sun, Tianyi Chen, Georgios Giannakis, and Zaiyue Yang. 2019. Communication-efficient distributed learning via lazily aggregated quantized gradients. In Proc. of NeurIPS. 3365–3375.

[104]

Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar, and H. Brendan McMahan. 2017. Distributed mean estimation with limited communication. In Proc. of ICML. JMLR. org, 3329–3337.

[105]

Ilya Sutskever, James Martens, George E. Dahl, and Geoffrey E. Hinton. 2013. On the importance of initialization and momentum in deep learning. In Proc. of ICML, Vol. 28. JMLR.org, 1139–1147.

Digital Library

[106]

Rashish Tandon, Qi Lei, Alexandros G. Dimakis, and Nikos Karampatziakis. 2017. Gradient coding: Avoiding stragglers in distributed learning. In Proc. of ICML, Vol. 70. PMLR, 3368–3376.

[107]

Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, and Ji Liu. 2018. Communication compression for decentralized training. In Proc. of NeurIPS. 7652–7662.

[108]

Hanlin Tang, Xiangru Lian, Tong Zhang, and Ji Liu. 2019. Doublesqueeze: Parallel stochastic gradient descent with double-pass error-compensated compression. arXiv:1905.05957. http://arxiv.org/abs/1905.05957.

[109]

Guanhong Tao, Shiqing Ma, Yingqi Liu, and Xiangyu Zhang. 2018. Attacks meet interpretability: Attribute-steered detection of adversarial samples. In Proc. of NeurIPS.

[110]

T. Tieleman and G. Hinton. 2017. Divide the gradient by a running average of its recent magnitude. Coursera: Neural networks for machine learning. Technical Report.

[111]

Nguyen H. Tran, Wei Bao, Albert Y. Zomaya, Nguyen N. H. Minh, and Choong Seon Hong. 2019. Federated learning over wireless networks: Optimization model design and analysis. In Proc. of INFOCOM. IEEE, 1387–1395.

Digital Library

[112]

Praneeth Vepakomma, Tristan Swedish, Ramesh Raskar, Otkrist Gupta, and Abhimanyu Dubey. 2018. No Peek: A survey of private distributed deep learning. arxiv:1812.03288.http://arxiv.org/abs/1812.03288.

[113]

Raajay Viswanathan, Ganesh Ananthanarayanan, and Aditya Akella. 2016. CLARINET: WAN-Aware optimization for analytics queries. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI’16). 435–450.

[114]

Thijs Vogels, Sai Praneeth Karimireddy, and Martin Jaggi. 2019. PowerSGD: Practical low-rank gradient compression for distributed optimization. In Proc. of NeurIPS. 14236–14245.

[115]

Ashish Vulimiri, Carlo Curino, Philip Brighten Godfrey, Thomas Jungblut, Jitu Padhye, and George Varghese. 2015. Global analytics in the face of bandwidth and regulatory constraints. In Proc. of NSDI. USENIX Association, 323–336.

[116]

Benjamin W. Wah (Ed.). 2008. Wiley Encyclopedia of Computer Science and Engineering. John Wiley & Sons, Inc.

[117]

Di Wang, Changyou Chen, and Jinhui Xu. 2019. Differentially private empirical risk minimization with non-convex loss functions. In Proc. of ICML, Vol. 97. PMLR, 6526–6535.

[118]

Di Wang, Marco Gaboardi, and Jinhui Xu. 2018. Empirical risk minimization in non-interactive local differential privacy revisited. In Proc. of NeurIPS. 965–974.

[119]

Di Wang, Minwei Ye, and Jinhui Xu. 2017. Differentially private empirical risk minimization revisited: Faster and more general. In Proc. of NeurIPS. 2722–2731.

[120]

Haozhao Wang, Zhihao Qu, Song Guo, Xin Gao, Ruixuan Li, and Baoliu Ye. 2020. Intermittent pulling with local compensation for communication-efficient federated learning. CoRR abs/2001.08277. https://arxiv.org/abs/2001.08277.

[121]

Linnan Wang, Yi Yang, Renqiang Min, and Srimat Chakradhar. 2017. Accelerating deep neural network training with inconsistent stochastic gradient descent. Neural Networks 93 (2017), 219–229.

[122]

Songtao Wang, Dan Li, Yang Cheng, Jinkun Geng, Yanshu Wang, Shuai Wang, Shu-Tao Xia, and Jianping Wu. 2018. BML: A high-performance, low-cost gradient synchronization algorithm for dml training. In Proc. of NeurIPS. 4238–4248.

[123]

Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, and Kevin Chan. 2018. When edge meets learning: Adaptive control for resource-constrained distributed machine learning. In Proc. of INFOCOM. IEEE, 63–71.

Digital Library

[124]

Jianqiao Wangni, Jialei Wang, Ji Liu, and Tong Zhang. 2018. Gradient sparsification for communication-efficient distributed optimization. In Proc. of NeurIPS. 1299–1309.

[125]

Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. 2017. Terngrad: Ternary gradients to reduce communication in distributed deep learning. In Proc. of NeurIPS. 1509–1519.

[126]

Carole-Jean Wu, David Brooks, Kevin Chen, Douglas Chen, Sy Choudhury, Marat Dukhan, Kim M. Hazelwood, Eldad Isaac, Yangqing Jia, Bill Jia, Tommer Leyvand, Hao Lu, Yang Lu, Lin Qiao, Brandon Reagen, Joe Spisak, Fei Sun, Andrew Tulloch, Peter Vajda, Xiaodong Wang, Yanghan Wang, Bram Wasti, Yiming Wu, Ran Xian, Sungjoo Yoo, and Peizhao Zhang. 2019. Machine learning at facebook: Understanding inference at the edge. In Proc. of HPCA. IEEE, 331–344.

[127]

Jiaxiang Wu, Weidong Huang, Junzhou Huang, and Tong Zhang. 2018. Error compensated quantized SGD and its applications to large-scale distributed optimization. arXiv:1806.08054. http://arxiv.org/abs/1806.08054.

[128]

Liang Xiao, Yanda Li, Guoan Han, Huaiyu Dai, and H. Vincent Poor. 2017. A secure mobile crowdsensing game with deep reinforcement learning. IEEE Transactions on Information Forensics and Security 13, 1 (2017), 35–47.

[129]

Liang Xiao, Yanda Li, Xueli Huang, and XiaoJiang Du. 2017. Cloud-based malware detection game for mobile devices with offloading. IEEE Transactions on Mobile Computing 16, 10 (2017), 2742–2750.

Digital Library

[130]

Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. 2018. Zeno: Byzantine-suspicious stochastic gradient descent. CoRR abs/1805.10032 (2018). arxiv:1805.10032.http://arxiv.org/abs/1805.10032.

[131]

Eric P. Xing, Qirong Ho, Pengtao Xie, and Dai Wei. 2016. Strategies and principles of distributed machine learning on big data. Engineering 2, 2 (2016), 179–195.

[132]

Shizhen Xu, Hao Zhang, Graham Neubig, Wei Dai, Jin Kyu Kim, Zhijie Deng, Qirong Ho, Guangwen Yang, and Eric P. Xing. 2018. Cavs: An efficient runtime system for dynamic neural networks. In Proc. of USENIX ATC. 937–950.

[133]

Dejun Yang, Guoliang Xue, Xi Fang, and Jian Tang. 2012. Crowdsourcing to smartphones: Incentive mechanism design for mobile phone sensing. In Proc. of MobiCom. ACM, 173–184.

Digital Library

[134]

Dejun Yang, Guoliang Xue, Xi Fang, and Jian Tang. 2015. Incentive mechanisms for crowdsensing: Crowdsourcing with smartphones. IEEE/ACM Transactions on Networking 24, 3 (2015), 1732–1744.

Digital Library

[135]

Howard H. Yang, Zuozhu Liu, Tony Q. S. Quek, and H. Vincent Poor. 2020. Scheduling policies for federated learning in wireless networks. IEEE Transactions on Communications 68, 1 (2020), 317–333.

[136]

Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 10, 2 (2019), 1–19.

Digital Library

[137]

Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter Bartlett. 2018. Byzantine-robust distributed learning: Towards optimal statistical rates. arXiv:1803.01498. http://arxiv.org/abs/1803.01498.

[138]

Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter L. Bartlett. 2018. Defending against saddle point attack in byzantine-robust distributed learning. CoRR abs/1806.05358 (2018). arxiv:1806.05358.http://arxiv.org/abs/1806.05358.

[139]

Ryo Yonetani, Tomohiro Takahashi, Atsushi Hashimoto, and Yoshitaka Ushiku. 2019. Decentralized learning of generative adversarial networks from multi-client non-iid data. arXiv:1905.09684. http://arxiv.org/abs/1905.09684.

[140]

Naoya Yoshida, Takayuki Nishio, Masahiro Morikura, Koji Yamamoto, and Ryo Yonetani. 2019. Hybrid-FL: Cooperative learning mechanism using Non-IID data in wireless networks. arXiv:1905.07210. http://arxiv.org/abs/1905.07210.

[141]

Yue Yu, Jiaxiang Wu, and Longbo Huang. 2019. Double quantization for communication-efficient distributed optimization. In Proc. of NeurIPS. 4440–4451.

[142]

Jiawei Yuan and Shucheng Yu. 2014. Privacy preserving back-propagation neural network learning made practical with cloud computing. IEEE Transactions on Parallel and Distributed Systems 25, 1 (2014), 212–221. https://doi.org/10.1109/TPDS.2013.18

Digital Library

[143]

Yufeng Zhan, Song Guo, Peng Li, and Jiang Zhang. 2020. A deep reinforcement learning based offloading game in edge computing. IEEE Transactions on Computers 69, 6 (2020), 883–893.

[144]

Yufeng Zhan, Peng Li, Zhihao Qu, Deze Zeng, and Song Guo. 2020. A Leaning-based incentive mechanism for federated learning. IEEE Internet of Things Journal 7, 7 (2020), 6360–6368.

[145]

Yufeng Zhan, Chi Harold Liu, Yinuo Zhao, Jiang Zhang, and Jian Tang. 2019. Free market of multi-leader multi-follower mobile crowdsensing: An incentive mechanism design by deep reinforcement learning. IEEE Transactions on Mobile Computing 19, 10 (2019), 2316–2329.

[146]

Yufeng Zhan, Yuanqing Xia, and Jinhui Zhang. 2018. Incentive mechanism in platform-centric mobile crowdsensing: A one-to-many bargaining approach. Computer Networks 132 (2018), 40–52.

Digital Library

[147]

Yufeng Zhan and Jiang Zhang. 2020. An incentive mechanism design for efficient edge learning by deep reinforcement learning approach. In Proc. of INFOCOM. IEEE, 2489–2498.

Digital Library

[148]

Chaoyun Zhang, Paul Patras, and Hamed Haddadi. 2019. Deep learning in mobile and wireless networking: A survey. IEEE Communications Surveys & Tutorials 21, 3 (2019), 2224–2287.

[149]

Hantian Zhang, Jerry Li, Kaan Kara, Dan Alistarh, Ji Liu, and Ce Zhang. 2017. Zipml: Training linear models with end-to-end low precision, and a little bit of deep learning. In Proc. of ICML. JMLR. org, 4035–4043.

[150]

Hao Zhang, Zeyu Zheng, Shizhen Xu, Wei Dai, Qirong Ho, Xiaodan Liang, Zhiting Hu, Jinliang Wei, Pengtao Xie, and Eric P. Xing. 2017. Poseidon: An efficient communication architecture for distributed deep learning on clusters. In Proc. of USENIX ATC. 181–193.

[151]

Minjia Zhang, Samyam Rajbhandari, Wenhan Wang, and Yuxiong He. 2018. DeepCPU: Serving RNN-based deep learning models 10 faster. In Proc. of USENIX ATC. 951–965.

[152]

Qingchen Zhang, Laurence T. Yang, and Zhikui Chen. 2016. Privacy preserving deep computation model on cloud for big data feature learning. IEEE Transactions on Computers 65, 5 (2016), 1351–1362. https://doi.org/10.1109/TC.2015.2470255

Digital Library

[153]

Yanru Zhang, Lingyang Song, Walid Saad, Zaher Dawy, and Zhu Han. 2015. Contract-based incentive mechanisms for device-to-device communications in cellular networks. IEEE Journal on Selected Areas in Communications 33, 10 (2015), 2144–2155.

Digital Library

[154]

Jun Zhao. 2018. Distributed deep learning under differential privacy with the teacher-student paradigm. In Proc. of AAAI (AAAI Workshops), Vol. WS-18. 404–408.

[155]

Tian Zhao, Yaqi Zhang, and Kunle Olukotun. 2019. Serving recurrent neural networks efficiently with a spatial accelerator. arxiv:1909.13654.http://arxiv.org/abs/1909.13654.

[156]

Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated learning with Non-IID data. arXiv:1806.00582. http://arxiv.org/abs/1806.00582.

[157]

Kai Zheng, Wenlong Mou, and Liwei Wang. 2017. Collect at once, use effectively: Making non-interactive locally private learning possible. In Proc. of ICML, Vol. 70. PMLR, 4130–4139.

[158]

Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, and Junshan Zhang. 2019. Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proc. IEEE 107, 8 (2019), 1738–1762.

[159]

Martin Zinkevich, Markus Weimer, Lihong Li, and Alex J. Smola. 2010. Parallelized stochastic gradient descent. In Proc. of NeurIPS. 2595–2603.

Cited By

Gecer MGarbinato B(2024)Federated Learning for Mobility ApplicationsACM Computing Surveys10.1145/363786856:5(1-28)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3637868
Liu YGuo SZhan YWu LHong ZZhou Q(2024)Chiron: A Robustness-Aware Incentive Scheme for Edge Learning via Hierarchical Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.335065423:8(8508-8524)Online publication date: Aug-2024
https://doi.org/10.1109/TMC.2024.3350654
Liu LDing ZCheng DZhou X(2024)Locality-Aware and Fault-Tolerant Batching for Machine Learning on Distributed DatasetsIEEE Transactions on Cloud Computing10.1109/TCC.2024.335171612:2(370-387)Online publication date: Apr-2024
https://doi.org/10.1109/TCC.2024.3351716
Show More Cited By

Index Terms

Edge Learning: The Enabling Technology for Distributed Big Data Analytics in the Edge
1. Computing methodologies
  1. Distributed computing methodologies
  2. Machine learning

Recommendations

A Systematic Literature Review on Federated Machine Learning: From a Software Engineering Perspective

Federated learning is an emerging machine learning paradigm where clients train models locally and formulate a global model based on the local model updates. To identify the state-of-the-art in federated learning and explore how to develop federated ...
Ubiquitous intelligent federated learning privacy-preserving scheme under edge computing
Abstract
With the rapid development of artificial intelligence (AI), combining machine learning (ML) and edge computing powered by big data has become a growing trend. However, under edge computing, ML faces problems such as limited energy ...
Highlights
- Propose a architecture of federated learning with cloud–edge-device collaboration under edge computing to solve the problem of insufficient computing power ...
A federated machine learning approach to detect international revenue share fraud on the 5G edge
SAC '22: Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing

The fifth-generation (5G) of broadband cellular networks is giving rise to new paradigms of distributed computing, such as Edge Computing and Multi-access Edge Computing (MEC). The possibility of hosting Machine Learning (ML) applications close to the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 54, Issue 7

September 2022

778 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3476825

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2021

Accepted: 01 May 2021

Revised: 01 May 2021

Received: 01 March 2020

Published in CSUR Volume 54, Issue 7

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Key Research and Development Program of China
Hong Kong RGC Research Impact Fund (RIF)
Shenzhen Science and Technology Innovation Commission
Collaborative Innovation Center of Novel Software Technology and Industrialization
Hong Kong RGC General Research Fund (GRF)
Fundamental Research Funds for the Central Universities
RCN-Diku INTPART BDEM
National Natural Science Foundation of China
Hong Kong RGC Collaborative Research Fund (CRF)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
1,530
Total Downloads

Downloads (Last 12 months)278
Downloads (Last 6 weeks)30

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gecer MGarbinato B(2024)Federated Learning for Mobility ApplicationsACM Computing Surveys10.1145/363786856:5(1-28)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3637868
Liu YGuo SZhan YWu LHong ZZhou Q(2024)Chiron: A Robustness-Aware Incentive Scheme for Edge Learning via Hierarchical Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.335065423:8(8508-8524)Online publication date: Aug-2024
https://doi.org/10.1109/TMC.2024.3350654
Liu LDing ZCheng DZhou X(2024)Locality-Aware and Fault-Tolerant Batching for Machine Learning on Distributed DatasetsIEEE Transactions on Cloud Computing10.1109/TCC.2024.335171612:2(370-387)Online publication date: Apr-2024
https://doi.org/10.1109/TCC.2024.3351716
Hong ZQiu XLin JChen WYu YWang HGuo SGao W(2024)Intelligence-Endogenous Management Platform for Computing and Network ConvergenceIEEE Network10.1109/MNET.2023.332152938:4(166-173)Online publication date: Jul-2024
https://doi.org/10.1109/MNET.2023.3321529
Shi BWu Y(2024)Task Offloading and Resource Allocation Strategies Among Multiple Edge ServersIEEE Internet of Things Journal10.1109/JIOT.2023.334379311:8(14647-14656)Online publication date: 15-Apr-2024
https://doi.org/10.1109/JIOT.2023.3343793
Huda NAhmed IAdnan MAli MNaeem F(2024)Experts and intelligent systems for smart homes’ Transformation to Sustainable Smart CitiesExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122380238:PFOnline publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122380
Srirama S(2024)Distributed edge analytics in edge‐fog‐cloud continuumInternet Technology Letters10.1002/itl2.562Online publication date: 29-Jul-2024
https://doi.org/10.1002/itl2.562
Anjos JMatteussi KOrlandi FBarbosa JSilva JBittencourt LGeyer C(2023)A Survey on Collaborative Learning for Intelligent Autonomous SystemsACM Computing Surveys10.1145/362554456:4(1-37)Online publication date: 10-Nov-2023
https://dl.acm.org/doi/10.1145/3625544
Jeyaraj RBalasubramaniam AM.A. AGuizani NPaul A(2023)Resource Management in Cloud and Cloud-influenced Technologies for Internet of Things ApplicationsACM Computing Surveys10.1145/357172955:12(1-37)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3571729
Yuan LHe QChen FDou RJin HYang Y(2023)PipeEdge: A Trusted Pipelining Collaborative Edge Training based on BlockchainProceedings of the ACM Web Conference 202310.1145/3543507.3583413(3033-3043)Online publication date: 30-Apr-2023
https://doi.org/10.1145/3543507.3583413
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents