research-article

Scalable Estimator for Multi-task Gaussian Graphical Models Based in an IoT Network

Authors:

Sen WangAuthors Info & Claims

ACM Transactions on Sensor Networks (TOSN), Volume 17, Issue 3

Article No.: 23, Pages 1 - 33

https://doi.org/10.1145/3432312

Published: 21 June 2021 Publication History

Abstract

Recently, the Internet of Things (IoT) receives significant interest due to its rapid development. But IoT applications still face two challenges: heterogeneity and large scale of IoT data. Therefore, how to efficiently integrate and process these complicated data becomes an essential problem. In this article, we focus on the problem that analyzing variable dependencies of data collected from different edge devices in the IoT network. Because data from different devices are heterogeneous and the variable dependencies can be characterized into a graphical model, we can focus on the problem that jointly estimating multiple, high-dimensional, and sparse Gaussian Graphical Models for many related tasks (edge devices). This is an important goal in many fields. Many IoT networks have collected massive multi-task data and require the analysis of heterogeneous data in many scenarios. Past works on the joint estimation are non-distributed and involve computationally expensive and complex non-smooth optimizations. To address these problems, we propose a novel approach: Multi-FST. Multi-FST can be efficiently implemented on a cloud-server-based IoT network. The cloud server has a low computational load and IoT devices use asynchronous communication with the server, leading to efficiency. Multi-FST shows significant improvement, over baselines, when tested on various datasets.

References

[1]

Eshrat Arjomandi, Michael J. Fischer, and Nancy A. Lynch. 1983. Efficiency of synchronous versus asynchronous distributed systems. J. ACM 30, 3 (1983), 449–456.

Digital Library

[2]

Luigi Atzori, Antonio Iera, and Giacomo Morabito. 2010. The internet of things: A survey. Comput. Netw. 54, 15 (2010), 2787–2805.

Digital Library

[3]

Onureena Banerjee, Laurent El Ghaoui, and Alexandre d’Aspremont. 2008. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9(Mar.2008), 485–516.

Digital Library

[4]

Mohamed Ben-Daya, Elkafi Hassini, and Zied Bahroun. 2019. Internet of things and supply chain management: A literature review. Int. J. Prod. Res. 57, 15–16 (2019), 4719–4742.

[5]

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1 (2011), 1–122.

Digital Library

[6]

Hongming Cai, Boyi Xu, Lihong Jiang, and Athanasios V. Vasilakos. 2016. IoT-based big data storage systems in cloud computing: Perspectives and challenges. IEEE IoT J. 4, 1 (2016), 75–87.

[7]

Tony Cai and Weidong Liu. 2011. Adaptive thresholding for sparse covariance matrix estimation. J. Am. Stat. Assoc. 106, 494 (2011), 672–684.

[8]

Tony Cai, Weidong Liu, and Xi Luo. 2011. A constrained minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 106, 494 (2011), 594–607.

[9]

Emmanuel Candes, Terence Tao, et al. 2007. The dantzig selector: Statistical estimation when p is much larger than n. Ann. Stat. 35, 6 (2007), 2313–2351.

[10]

K. Mani Chandy and Jayadev Misra. 1981. Asynchronous distributed simulation via a sequence of parallel computations. Commun. ACM 24, 4 (1981), 198–206.

Digital Library

[11]

Tsung-Hui Chang, Mingyi Hong, Wei-Cheng Liao, and Xiangfeng Wang. 2016. Asynchronous distributed ADMM for large-scale optimization—Part I: Algorithm and convergence analysis. IEEE Trans. Sign. Process. 64, 12 (2016), 3118–3130.

Digital Library

[12]

Julien Chiquet, Yves Grandvalet, and Christophe Ambroise. 2011. Inferring multiple graphical structures. Stat. Comput. 21, 4 (2011), 537–553.

Digital Library

[13]

Dondapati Chowdary, Jessica Lathrop, Joanne Skelton, Kathleen Curtin, Thomas Briggs, Yi Zhang, Jack Yu, Yixin Wang, and Abhijit Mazumder. 2006. Prognostic gene expression signatures can be measured in tissues collected in RNAlater preservative. J. Molec. Diagnost. 8, 1 (2006), 31–39.

[14]

ENCODE Project Consortium et al. 2004. The ENCODE (ENCyclopedia of DNA elements) project. Science 306, 5696 (2004), 636–640.

[15]

Flaviu Cristian and Christof Fetzer. 1999. The timed asynchronous distributed system model. IEEE Trans. Parallel Distrib. Syst. 10, 6 (1999), 642–657.

Digital Library

[16]

Patrick Danaher, Pei Wang, and Daniela M. Witten. 2014. The joint graphical lasso for inverse covariance estimation across multiple classes. J. Roy. Stat. Soc.: Ser. B Stat. Methodol. 76, 2 (2014), 373–397.

[17]

Adriana Di Martino, Chao-Gan Yan, Qingyang Li, Erin Denio, Francisco X. Castellanos, Kaat Alaerts, Jeffrey S. Anderson, Michal Assaf, Susan Y. Bookheimer, Mirella Dapretto, et al. 2014. The autism brain imaging data exchange: Towards a large-scale evaluation of the intrinsic brain architecture in autism. Molec. Psychiatr. 19, 6 (2014), 659–667.

[18]

Adrian Dobra, Chris Hans, Beatrix Jones, Joseph R. Nevins, Guang Yao, and Mike West. 2004. Sparse graphical models for exploring gene expression data. J. Multivar. Anal. 90, 1 (2004), 196–212.

Digital Library

[19]

Jianqing Fan and Runze Li. 2001. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96, 456 (2001), 1348–1360.

[20]

Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2008. Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9, 3 (2008), 432–441.

[21]

Xu Gao, Weining Shen, Chee-Ming Ting, Steven C. Cramer, Ramesh Srinivasan, and Hernando Ombao. 2018. Modeling brain connectivity with graphical models on frequency domain. arXiv:1810.03279. Retrieved from https://arxiv.org/abs/1810.03279.

[22]

Patricia Gonzalez-Guerrero, Stephen G. Wilson, and Mircea R. Stan. 2019. Error-latency trade-off for asynchronous stochastic computing with streams for the IoT. In Proceedings of the 2019 32nd IEEE International System-on-Chip Conference (SOCC’19). IEEE, 97–102.

[23]

Jian Guo, Elizaveta Levina, George Michailidis, and Ji Zhu. 2011. Joint estimation of multiple graphical models. Biometrika 98, 1 (2011), 1–15.

[24]

Satoshi Hara and Takashi Washio. 2013. Learning a common substructure of multiple graphical gaussian models. Neur. Netw. 38 (2013), 23–38.

Digital Library

[25]

Holger Hoefling. 2010. A path algorithm for the fused lasso signal approximator. J. Comput. Graph. Stat. 19, 4 (2010), 984–1006.

[26]

Jean Honorio and Dimitris Samaras. 2010. Multi-task learning of Gaussian graphical models. In Proceedings of the 27th International Conference on Machine Learning (ICML'10). 447--454.

Digital Library

[27]

John E. Hopcroft and Jeffrey D. Ullman. 1973. Set merging algorithms. SIAM J. Comput. 2, 4 (1973), 294–303.

Digital Library

[28]

Wuyungerile Li, Bing Jia, Haotian Xu, Zhaopeng Zong, and Takashi Watanabe. 2019. A multi-task scheduling mechanism based on ACO for maximizing workers’ benefits in mobile crowdsensing service markets with the internet of things. IEEE Access 7 (2019), 41463–41469.

[29]

Xiaoyuan Liu, Hongwei Li, Guowen Xu, Sen Liu, Zhe Liu, and Rongxing Lu. 2020. PADL: Privacy-aware and asynchronous deep learning for IoT applications. IEEE IoT J. (2020).

[30]

Meng Ma, Ping Wang, and Chao-Hsien Chu. 2013. Data management for internet of things: Challenges, approaches and opportunities. In Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing. IEEE, 1144–1151.

Digital Library

[31]

Rahul Mazumder and Trevor Hastie. 2012. Exact covariance thresholding into connected components for large-scale graphical lasso. J. Mach. Learn. Res. 13 (Mar. 2012), 781–794.

Digital Library

[32]

Tom M. Mitchell, Svetlana V. Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L. Malave, Robert A. Mason, and Marcel Adam Just. 2008. Predicting human brain activity associated with the meanings of nouns. Science 320, 5880 (2008), 1191–1195.

[33]

Karthik Mohan, Maryam Fazel Palma London, Daniela Witten, and Su-In Lee. 2014. Node-based learning of multiple Gaussian graphical models. J. Mach. Learn. Res. 15, 1 (2014), 445.

Digital Library

[34]

Di Mu, Yunpeng Ge, Mo Sha, Steve Paul, Niranjan Ravichandra, and Souma Chowdhury. 2019. Robust optimal selection of radio type and transmission power for internet of things. ACM Trans. Sen. Netw. 15, 4, Article 39 (July 2019), 25 pages.

Digital Library

[35]

Michael A. Osborne, Stephen J. Roberts, Alex Rogers, and Nicholas R. Jennings. 2012. Real-time information processing of environmental sensor network data using bayesian gaussian processes. ACM Trans. Sen. Netw. 9, 1, Article 1 (Nov. 2012), 32 pages.

Digital Library

[36]

Rodrigo Perin, Martin Telefont, and Henry Markram. 2013. Computing the size and number of neuronal clusters in local circuits. Front. Neuroanat. 7 (2013), 1.

[37]

Russell A. Poldrack and Krzysztof J. Gorgolewski. 2017. OpenfMRI: Open sharing of task fMRI data. NeuroImage 144 (2017), 259–261.

[38]

Adam J. Rothman, Peter J. Bickel, Elizaveta Levina, Ji Zhu, et al. 2008. Sparse permutation invariant covariance estimation. Electr. J. Stat. 2 (2008), 494–515.

[39]

Adam J. Rothman, Elizaveta Levina, and Ji Zhu. 2009. Generalized thresholding of large covariance matrices. J. Am. Stat. Assoc. 104, 485 (2009), 177–186.

[40]

Mehrdad Salimitari, Shameek Bhattacharjee, Mainak Chatterjee, and Yaser P. Fallah. 2020. A prospect theoretic approach for trust management in IoT networks under manipulation attacks. ACM Trans. Sens. Netw. 16, 3, Article 26 (May 2020), 26 pages.

Digital Library

[41]

Paul T. Schultz and Robert A. Sartini. 2017. IoT communication utilizing secure asynchronous P2P communication and data exchange. US Patent 9,838,204.

[42]

Shihao Shen, Yiwen Han, Xiaofei Wang, and Yan Wang. 2019. Computation offloading with multiple agents in edge-computing–supported IoT. ACM Trans. Sens. Netw. 16, 1, Article 8 (Dec. 2019), 27 pages.

Digital Library

[43]

Noah Simon, Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2013. A sparse-group lasso. J. Comput. Graph. Stat. 22, 2 (2013), 231–245.

[44]

Zhiyong Sun, Junyong Ye, TongqingWang, Shijian Huang, and Jin Luo. 2020. Behavioral feature recognition of multi-task compressed sensing with fusion relevance in the Internet of Things environment. Comput. Commun, 157 (2020), 381--393.

[45]

Erming Tian, Fenghuang Zhan, Ronald Walker, Erik Rasmussen, Yupo Ma, Bart Barlogie, and John D Shaughnessy Jr. 2003. The role of the Wnt-signaling antagonist DKK1 in the development of osteolytic lesions in multiple myeloma. N. Engl. J. Med. 349, 26 (2003), 2483–2494.

[46]

Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B Methodol. 58, 1 (1996), 267–288.

[47]

Ivy F. Tso, Saige Rutherford, Yu Fang, Mike Angstadt, and Stephan F. Taylor. 2018. The “social brain” is highly sensitive to the mere presence of social information: An automated meta-analysis and an independent study. PLoS One 13, 5 (2018).

[48]

Beilun Wang, Ji Gao, and Yanjun Qi. 2017. A fast and scalable joint estimator for learning multiple related sparse gaussian graphical models. arXiv:1702.02715. Retrieved from https://arxiv.org/abs/1702.02715.

[49]

Beilun Wang, Ritambhara Singh, and Yanjun Qi. 2017. A constrained l1 minimization approach for estimating multiple sparse Gaussian or nonparanormal graphical models. Mach. Learn. 106, 9-10 (2017), 1381–1417.

Digital Library

[50]

Lizhe Wang and Rajiv Ranjan. 2015. Processing distributed internet of things data in clouds. IEEE Cloud Comput. 2, 1 (2015), 76–80.

[51]

Daniela M. Witten, Jerome H. Friedman, and Noah Simon. 2011. New insights and faster computations for the graphical lasso. J. Comput. Graph. Stat. 20, 4 (2011), 892–900.

[52]

Nuosi Wu, Jiang Huang, Xiao-Fei Zhang, Le Ou-Yang, Shan He, Zexuan Zhu, and Weixin Xie. 2019. Weighted fused pathway graphical lasso for joint estimation of multiple gene networks. Front. Genet. 10 (2019), 623.

[53]

Han Xiao, Changqiao Xu, Tengfei Cao, Lujie Zhong, and Gabriel-Miro Muntean. 2019. GTTC: A low-expenditure IoT multi-task coordinated distributed computing framework with fog computing. In Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM’19). IEEE, 1–6.

Digital Library

[54]

Eunho Yang, Aurélie C. Lozano, and Pradeep K. Ravikumar. 2014. Elementary estimators for graphical models. In Advances in Neural Information Processing Systems. 2159–2167.

Digital Library

[55]

Ming Yuan and Yi Lin. 2007. Model selection and estimation in the gaussian graphical model. Biometrika 94, 1 (2007), 19–35.

[56]

Bai Zhang and Yue Wang. 2012. Learning structural changes of Gaussian graphical models in controlled experiments. arXiv:1203.3532. Retrieved from https://arxiv.org/abs/1203.3532.

Digital Library

[57]

L. Zhang, S. Yu, X. Ding, and X. Wang. 2014. Research on IOT RESTful web service asynchronous composition based on BPEL. In Proceedings of the 2014 6th International Conference on Intelligent Human-Machine Systems and Cybernetics, Vol. 1. 62–65.

Digital Library

[58]

Yi Zhang and Jeff G. Schneider. 2010. Learning multiple tasks with a sparse matrix-normal penalty. In Advances in Neural Information Processing Systems. 2550–2558.

Digital Library

[59]

Yunzhang Zhu, Xiaotong Shen, and Wei Pan. 2014. Structural pursuit over multiple undirected graphs. J. Am. Stat. Assoc. 109, 508 (2014), 1683–1696.

[60]

Hui Zou. 2006. The adaptive lasso and its oracle properties. J. Am. Stat. Assoc. 101, 476 (2006), 1418–1429.

Cited By

Syu JLin JSrivastava G(2024)Distributed Learning Mechanisms for Anomaly Detection in Privacy-Aware Energy Grid Management SystemsACM Transactions on Sensor Networks10.1145/3640341Online publication date: 17-Jan-2024
https://dl.acm.org/doi/10.1145/3640341
Xia SXing TWu CLiu GYang JLi K(2024)AQMon: A Fine-grained Air Quality Monitoring System Based on UAV Images for Smart CitiesACM Transactions on Sensor Networks10.1145/363876620:2(1-20)Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1145/3638766
Su HLi JDu ZZhu LLu KShen H(2024)Cross-domain Recommendation via Dual Adversarial AdaptationACM Transactions on Information Systems10.1145/363252442:3(1-26)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3632524
Show More Cited By

Index Terms

Scalable Estimator for Multi-task Gaussian Graphical Models Based in an IoT Network
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
      1. Cloud computing
2. Computing methodologies
  1. Artificial intelligence
    1. Distributed artificial intelligence
  2. Machine learning
    1. Machine learning approaches
      1. Learning in probabilistic graphical models

Recommendations

IoT Mashups: From IoT Big Data to IoT Big Service
ICFNDS '17: Proceedings of the International Conference on Future Networks and Distributed Systems

Internet of Things (IoT) addresses the challenge to provide a transparent access to a huge number of IoT resources that can be either physical devices or just data resources. Moreover, because of the large number of resource-constrained devices and the ...
From IoT big data to IoT big services
SAC '17: Proceedings of the Symposium on Applied Computing

The large-scale deployments of Internet of Things (IoT) systems have introduced several new challenges in terms of processing their data. The massive amount of IoT-generated data requires design solutions to speed up data processing, scale up with the ...
External integrity verification for outsourced big data in cloud and IoT

As cloud computing is being widely adopted for big data processing, data security is becoming one of the major concerns of data owners. Data integrity is an important factor in almost any data and computation related context. It is not only one of the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 17, Issue 3

August 2021

333 pages

ISSN:1550-4859

EISSN:1550-4867

DOI:10.1145/3470624

Editor:
Yunhao Liu
Tsinghua University, China

Issue’s Table of Contents

Copyright © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 21 June 2021

Accepted: 01 October 2020

Revised: 01 September 2020

Received: 01 July 2020

Published in TOSN Volume 17, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China
Natural Science Foundation of Jiangsu Province
National Key R&D Program of China
National Natural Science Foundation of China
Fundamental Research Funds for the Central Universities, Jiangsu Provincial Key Laboratory of Network and Information Security
Key Laboratory of Computer Network and Information Integration of Ministry of Education of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
147
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Syu JLin JSrivastava G(2024)Distributed Learning Mechanisms for Anomaly Detection in Privacy-Aware Energy Grid Management SystemsACM Transactions on Sensor Networks10.1145/3640341Online publication date: 17-Jan-2024
https://dl.acm.org/doi/10.1145/3640341
Xia SXing TWu CLiu GYang JLi K(2024)AQMon: A Fine-grained Air Quality Monitoring System Based on UAV Images for Smart CitiesACM Transactions on Sensor Networks10.1145/363876620:2(1-20)Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1145/3638766
Su HLi JDu ZZhu LLu KShen H(2024)Cross-domain Recommendation via Dual Adversarial AdaptationACM Transactions on Information Systems10.1145/363252442:3(1-26)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3632524
Wang DLi FLiu KZhang X(2024)Real-time Cyber-Physical Security Solution Leveraging an Integrated Learning-Based ApproachACM Transactions on Sensor Networks10.1145/358200920:2(1-22)Online publication date: 9-Jan-2024
https://dl.acm.org/doi/10.1145/3582009
Long ZZhu CChen JLi ZRen YLiu Y(2024)Multi-View MERA Subspace ClusteringIEEE Transactions on Multimedia10.1109/TMM.2023.330723926(3102-3112)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3307239
Hui CZhang SCui WLiu SJiang FZhao D(2024)Rate-Adaptive Neural Network for Image Compressive SensingIEEE Transactions on Multimedia10.1109/TMM.2023.330121326(2515-2530)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3301213
Wang RWang FSu YSun JSun FLi H(2023)Attention-guided Multi-modality Interaction Network for RGB-D Salient Object DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/362474720:3(1-22)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3624747
Bölz FNurbakova DCalabretto SGerl ABrunie LKosch H(2023)HUMMUS: A Linked, Healthiness-Aware, User-centered and Argument-Enabling Recipe Data Set for RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3609491(1-11)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3609491
Gao LGuan L(2023)A Discriminant Information Theoretic Learning Framework for Multi-modal Feature RepresentationACM Transactions on Intelligent Systems and Technology10.1145/358725314:3(1-24)Online publication date: 13-Apr-2023
https://dl.acm.org/doi/10.1145/3587253
Pan TYe YCai HHuang SYang YWang GEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Multimodal Physiological Signals Fusion for Online Emotion RecognitionProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612555(5879-5888)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612555
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents