Personalized Federated Human Activity Recognition through Semi-supervised Learning and Enhanced Representation
Pages 463 - 468
Abstract
Owing to the widespread utilization of activity recognition, numerous deep learning models have been proposed to facilitate recognition in diverse contexts. Federated learning (FL) is an emerging learning paradigm that enables the collaborative learning of a shared model without compromising the data privacy of users. However, the assumption of FL is to rely on the annotated data on clients, which is difficult to acquire the annotations for human activity recognition (HAR) on all clients due to the lack of expertise or resource. Moreover, a general model is not suitable for each person due to data heterogeneity, resulting from the different physical characteristics and various contextual information. To this end, we propose a semi-supervised learning method for personalized federated HAR, in which clients have completely unlabeled data, while the server has a small amount of labeled data contributed by volunteers. Clients conduct unsupervised learning on autoencoders with locally unlabeled data to collaboratively learn a general representation model. The server conducts supervised learning on an activity classifier with labeled data stored on the server. After that, the shared model is personalized using individually pseudo-labeled data on each client side, wherein both confidence and uncertainty are taken into account concurrently, with the aim of achieving a balanced selection for assigning pseudo-labels to samples. We conduct extensive experiments with two different real-world HAR datasets, demonstrating the effectiveness of the proposed methods.
References
[1]
Muhammad Haseeb Arshad, Muhammad Bilal, and Abdullah Gani. 2022. Human Activity Recognition: Review, Taxonomy and Open Challenges. Sensors 22, 17 (January 2022). https://doi.org/10/gr8jwx
[2]
Yiqiang Chen, Xin Qin, Jindong Wang, Chaohui Yu, and Wen Gao. 2020. FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare. IEEE Intelligent Systems 35, 4 (July 2020), 83–93. https://doi.org/10/gmt6bf
[3]
Federico Cruciani, Anastasios Vafeiadis, Chris Nugent, Ian Cleland, Paul McCullagh, Konstantinos Votis, Dimitrios Giakoumis, Dimitrios Tzovaras, Liming Chen, and Raouf Hamzaoui. 2020. Feature Learning for Human Activity Recognition Using Convolutional Neural Networks. CCF Transactions on Pervasive Computing and Interaction 2, 1 (March 2020), 18–32. https://doi.org/10/gr8jww
[4]
Enmao Diao, Jie Ding, and Vahid Tarokh. 2022. SemiFL: Semi-supervised Federated Learning for Unlabeled Clients with Alternate Training. Advances in Neural Information Processing Systems 35 (October 2022), 17871–17884. https://doi.org/10.48550/arXiv.2106.01432.
[5]
Jie Feng, Can Rong, Funing Sun, Diansheng Guo, and Yong Li. 2020. PMF: A Privacy-Preserving Human Mobility Prediction Framework via Federated Learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (March 2020), 1–21. https://doi.org/10/gr8jwt
[6]
Chenglin Li, Di Niu, Bei Jiang, Xiao Zuo, and Jianming Yang. 2021. Meta-HAR: Federated Representation Learning for Human Activity Recognition. In Proceedings of the Web Conference 2021, Apr. 912–922. https://doi.org/10/gkzww4
[7]
Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. 2020. A Review of Applications in Federated Learning. Computers & Industrial Engineering 149 (November 2020), 106854. https://doi.org/10/ghk425
[8]
Haowen Lin, Jian Lou, Li Xiong, and Cyrus Shahabi. 2021. SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling. arXiv preprint arXiv:2108.09412 (August 2021). https://doi.org/10.48550/arXiv.2108.09412.
[9]
Zewei Long, Liwei Che, Yaqing Wang, Muchao Ye, Junyu Luo, Jinze Wu, Houping Xiao, and Fenglong Ma. 2020. FedSemi: An Adaptive Federated Semi-Supervised Learning Framework. arXiv preprint arXiv:2012.03292 24 (December 2020).
[10]
Fernando Moya Rueda, René Grzeszick, Gernot A Fink, Sascha Feldhorst, and Michael Ten Hompel. 2018. Convolutional Neural Networks for Human Activity Recognition Using Body-Worn Sensors. Informatics 5, 2 (June 2018), 26. https://doi.org/10/gm95b9
[11]
Ohoud Nafea, Wadood Abdul, Ghulam Muhammad, and Mansour Alsulaiman. 2021. Sensor-Based Human Activity Recognition with Spatio-Temporal Deep Learning. Sensors 21, 6 (January 2021), 2141. https://doi.org/10/gjkp95
[12]
Jaehoon Oh, Sangmook Kim, and Se-Young Yun. 2021. FedBABU: Towards Enhanced Representation for Federated Image Classification. arXiv preprint arXiv:2106.06042 (March 2021). https://doi.org/10.48550/arXiv.2106.06042.
[13]
Riccardo Presotto, Gabriele Civitarese, and Claudio Bettini. 2022. Semi-Supervised and Personalized Federated Activity Recognition Based on Active Learning and Label Propagation. Personal and Ubiquitous Computing 26, 5 (June 2022), 1281–1298. https://doi.org/10/gr8jwv
[14]
Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. 2021. In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-Label Selection Framework for Semi-Supervised Learning. arXiv preprint arXiv:2101.06329 (April 2021). https://doi.org/10.48550/arXiv.2101.06329.
[15]
Pekka Siirtola and Juha Röning. 2019. Incremental learning to personalize human activity recognition models: the importance of human AI collaboration. Sensors 19, 23 (2019), 5151.
[16]
Konstantin Sozinov, Vladimir Vlassov, and Sarunas Girdzijauskas. 2018. Human Activity Recognition Using Federated Learning. In 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications (ISPA/IUCC/BDCloud/SocialCom/SustainCom. IEEE, 1103–1111. https://doi.org/10.1109/BDCloud.2018.00164.
[17]
Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. 2022. Towards Personalized Federated Learning. IEEE Transactions on Neural Networks and Learning Systems (2022), 1–17. https://doi.org/10/gqj28w
[18]
Chi Ian Tang, Ignacio Perez-Pozuelo, Dimitris Spathis, Soren Brage, Nick Wareham, and Cecilia Mascolo. 2021. SelfHAR: Improving Human Activity Recognition through Self-Training with Unlabeled Data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (March 2021), 1–30. https://doi.org/10/f6tv
[19]
Dipanwita Thakur, Suparna Biswas, Edmond SL Ho, and Samiran Chattopadhyay. 2022. ConvAE-LSTM: Convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition. IEEE Access 10 (2022), 4137–4156. https://doi.org/10/gq8d2b
[20]
Bram van Berlo, Aaqib Saeed, and Tanir Ozcelebi. 2020. Towards Federated Unsupervised Representation Learning. In Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking, in EdgeSys ’20. Association for Computing Machinery, New York, NY, USA, 31–36. https://doi.org/10/gpn78c
[21]
Shaohua Wan, Lianyong Qi, Xiaolong Xu, Chao Tong, and Zonghua Gu. 2020. Deep Learning Models for Real-Time Human Activity Recognition with Smartphones. Mobile Networks and Applications 25, 2 (April 2020), 743–755. https://doi.org/10/gjd6g9
[22]
Lei Wang, Yangyang Xu, Jun Cheng, Haiying Xia, Jianqin Yin, and Jiaji Wu. 2018. Human Action Recognition by Learning Spatio-Temporal Features with Deep Neural Networks. IEEE Access 6 (2018), 17913–17922. https://doi.org/10/gq8dz9
[23]
Qiong Wu, Xu Chen, Zhi Zhou, and Junshan Zhang. 2022. FedHome: Cloud-edge Based Personalized Federated Learning for in-Home Health Monitoring. IEEE Transactions on Mobile Computing 21, 8 (August 2022), 2818–2832. https://doi.org/10/gqv5xb
[24]
Qiong Wu, Kaiwen He, and Xu Chen. 2020. Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework. IEEE Open Journal of the Computer Society 1 (2020), 35–44. https://doi.org/10/gk2c69
[25]
Kun Xia, Jianguang Huang, and Hanyu Wang. 2020. LSTM-CNN Architecture for Human Activity Recognition. IEEE Access 8 (2020), 56855–56866. https://doi.org/10/gm5mzn
[26]
Hongzheng Yu, Zekai Chen, Xiao Zhang, Xu Chen, Fuzhen Zhuang, Hui Xiong, and Xiuzhen Cheng. 2021. FedHAR: Semi-supervised Online Learning for Personalized Federated Human Activity Recognition. IEEE Transactions on Mobile Computing (2021), 1–1. https://doi.org/10/gqj86g
[27]
Yuchen Zhao, Hamed Haddadi, Severin Skillman, Shirin Enshaeifar, and Payam Barnaghi. 2020. Privacy-Preserving Activity and Health Monitoring on Databox. In Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking. Association for Computing Machinery, New York, NY, USA, 49–54.
[28]
Yuchen Zhao, Hanyang Liu, Honglin Li, Payam Barnaghi, and Hamed Haddadi. 2020. Semi-supervised federated learning for activity recognition. arXiv preprint arXiv:2011.00851 (2020). https://doi.org/10.48550/arXiv.2011.00851.
Index Terms
- Personalized Federated Human Activity Recognition through Semi-supervised Learning and Enhanced Representation
Recommendations
Federated Clustering and Semi-Supervised learning: A new partnership for personalized Human Activity Recognition
AbstractFederated Learning (FL) is currently studied by several research groups as a promising paradigm for sensor-based Human Activity Recognition (HAR) to mitigate the privacy and scalability issues of classic centralized approaches. However,...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningIn multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Federated Self-training for Semi-supervised Audio Recognition
Federated Learning is a distributed machine learning paradigm dealing with decentralized and personal datasets. Since data reside on devices such as smartphones and virtual assistants, labeling is entrusted to the clients or labels are extracted in an ...
Comments
Information & Contributors
Information
Published In
October 2023
822 pages
ISBN:9798400702006
DOI:10.1145/3594739
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Japan Society for the Promotion of Science KAKENHI
Conference
UbiComp/ISWC '23: The 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing
October 8 - 12, 2023
Cancun, Quintana Roo, Mexico
Acceptance Rates
Overall Acceptance Rate 764 of 2,912 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 175Total Downloads
- Downloads (Last 12 months)175
- Downloads (Last 6 weeks)20
Reflects downloads up to 16 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format