MetaFormer: Domain-Adaptive WiFi Sensing with Only One Labelled Target Sample
Article No.: 39, Pages 1 - 27
Abstract
WiFi based action recognition has attracted increasing attentions due to its convenience and universality in real-world applications, whereas the domain dependency leads to poor generalization ability towards new sensing environments or subjects. The majority of existing solutions fail to sufficiently extract action-related features from WiFi signals. Moreover, they are unable to make full use of the target data with only the labelled samples taken into consideration. To cope with these issues, we propose a WiFi-based sensing system, MetaFormer, which can effectively recognize actions from unseen domains with only one labelled target sample per category. Specifically, MetaFormer achieves this by firstly constructing a novel spatial-temporal transformer feature extraction structure with dense-sparse input named DS-STT to capture action primary and affiliated movements. It then designs Meta-teacher framework which meta-pre-trains source tasks and updates model parameters by dynamic pseudo label enhancement to bridge the relationship among the labelled and unlabelled target samples. In order to validate the performance of MetaFormer, we conduct comprehensive evaluations on SignFi, Widar and Wiar datasets and achieve superior performances under the one-shot case.
References
[1]
Xi Chen, Hang Li, Chenyi Zhou, Xue Liu, Di Wu, and Gregory Dudek. 2020. Fido: Ubiquitous fine-grained wifi-based localization for unlabelled users via domain adaptation. In Proceedings of The Web Conference 2020. 23--33.
[2]
Yinpeng Chen, Xiyang Dai, Dongdong Chen, Mengchen Liu, Xiaoyi Dong, Lu Yuan, and Zicheng Liu. 2022. Mobile-former: Bridging mobilenet and transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5270--5279.
[3]
Zhenghua Chen, Le Zhang, Chaoyang Jiang, Zhiguang Cao, and Wei Cui. 2018. WiFi CSI based passive human activity recognition using attention based BLSTM. IEEE Transactions on Mobile Computing 18, 11 (2018), 2714--2724.
[4]
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).
[5]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 248--255. https://doi.org/10.1109/CVPR.2009.5206848
[6]
Shuya Ding, Zhe Chen, Tianyue Zheng, and Jun Luo. 2020. RF-net: A unified meta-learning framework for RF-enabled one-shot human activity recognition. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems. 517--530.
[7]
Jeffrey Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Darrell. 2015. Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2625--2634.
[8]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
[9]
Xiaoyi Fan, Wei Gong, and Jiangchuan Liu. 2017. I2tag: RFID mobility and activity identification through intelligent profiling. ACM Transactions on Intelligent Systems and Technology (TIST) 9, 1 (2017), 1--21.
[10]
Abassin Sourou Fangbemi, Bin Liu, Neng Hai Yu, and Yanxiang Zhang. 2018. Efficient human action recognition interface for augmented and virtual reality applications based on binary descriptor. In International Conference on Augmented Reality, Virtual Reality and Computer Graphics. Springer, 252--260.
[11]
Chao Feng, Nan Wang, Yicheng Jiang, Xia Zheng, Kang Li, Zheng Wang, and Xiaojiang Chen. 2022. Wi-Learner: Towards One-shot Learning for Cross-Domain Wi-Fi based Gesture Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--27.
[12]
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning. PMLR, 1126--1135.
[13]
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17, 1 (2016), 2096--2030.
[14]
Ruiyang Gao, Mi Zhang, Jie Zhang, Yang Li, Enze Yi, Dan Wu, Leye Wang, and Daqing Zhang. 2021. Towards position-independent sensing for gesture recognition with Wi-Fi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--28.
[15]
Yu Guan and Thomas Plötz. 2017. Ensembles of deep lstm learners for activity recognition using wearables. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 1, 2 (2017), 1--28.
[16]
Linlin Guo, Lei Wang, Chuang Lin, Jialin Liu, Bingxian Lu, Jian Fang, Zhonghao Liu, Zeyang Shan, Jingwen Yang, and Silu Guo. 2019. Wiar: A public dataset for wifi-based activity recognition. IEEE Access 7 (2019), 154935--154945.
[17]
Pengli Hu, Chengpei Tang, Kang Yin, and Xie Zhang. 2021. WiGR: A Practical Wi-Fi-Based Gesture Recognition System with a Lightweight Few-Shot Network. Applied Sciences 11, 8 (2021), 3329.
[18]
Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shuochao Yao, Yaqing Wang, Ye Yuan, Hongfei Xue, Chen Song, Xin Ma, Dimitrios Koutsonikolas, et al. 2018. Towards environment independent device free human activity recognition. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. 289--304.
[19]
Hao Kong, Li Lu, Jiadi Yu, Yingying Chen, Xiangyu Xu, Feilong Tang, and Yi-Chao Chen. 2021. Multiauth: Enable multi-user authentication with single commodity wifi device. In Proceedings of the Twenty-second International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing. 31--40.
[20]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60, 6 (2017), 84--90.
[21]
Gierad Laput and Chris Harrison. 2019. Sensing fine-grained hand activity with smartwatches. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.
[22]
Bing Li, Wei Cui, Wei Wang, Le Zhang, Zhenghua Chen, and Min Wu. 2021. Two-stream convolution augmented transformer for human activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 286--293.
[23]
Chenning Li, Manni Liu, and Zhichao Cao. 2020. WiHF: Enable User Identified Gesture Recognition with WiFi. In IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.
[24]
Hong Li, Wei Yang, Jianxin Wang, Yang Xu, and Liusheng Huang. 2016. WiFinger: Talk to your smart devices with finger-grained gesture. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 250--261.
[25]
Xinyi Li, Liqiong Chang, Fangfang Song, Ju Wang, and Zheng Wang. 2021. CrossGR: Accurate and Low-cost Cross-target Gesture Recognition Using Wi-Fi. Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies 5, 1 (2021).
[26]
Haipeng Liu, Kening Cui, Kaiyuan Hu, Yuheng Wang, Anfu Zhou, Liang Liu, and Huadong Ma. 2022. MTransSee: Enabling environment-independent mmWave sensing based gesture recognition via transfer learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1--28.
[27]
Zijuan Liu, Xiulong Liu, and Keqiu Li. 2020. Deeper exercise monitoring for smart gym using fused rfid and cv data. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 11--19.
[28]
Yongsen Ma, Gang Zhou, Shuangquan Wang, Hongyang Zhao, and Woosub Jung. 2018. SignFi: Sign language recognition using WiFi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 1--21.
[29]
Sachin Mehta and Mohammad Rastegari. 2021. Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. arXiv preprint arXiv:2110.02178 (2021).
[30]
Panneer Selvam Santhalingam, Al Amin Hosain, Ding Zhang, Parth Pathak, Huzefa Rangwala, and Raja Kushalnagar. 2020. mmasl: Environment-independent asl gesture recognition using 60 ghz millimeter-wave signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--30.
[31]
Zhenguo Shi, J Andrew Zhang, Richard Yida Xu, and Qingqing Cheng. 2022. Environment-robust device-free human activity recognition with channel-state-information enhancement and one-shot learning. IEEE Transactions on Mobile Computing 21, 02 (2022), 540--554.
[32]
Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. Advances in neural information processing systems 30 (2017).
[33]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017).
[34]
Tran Huy Vu, Archan Misra, Quentin Roy, Kenny Choo Tsu Wei, and Youngki Lee. 2018. Smartwatch-based early gesture detection 8 trajectory tracking for interactive gesture-driven applications. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 1--27.
[35]
Dazhuo Wang, Jianfei Yang, Wei Cui, Lihua Xie, and Sumei Sun. 2022. Airfi: empowering wifi-based passive human gesture recognition to unseen environment via domain generalization. IEEE Transactions on Mobile Computing (2022).
[36]
Fangxin Wang, Jiangchuan Liu, and Wei Gong. 2019. WiCAR: wifi-based in-car activity recognition with multi-adversarial domain adaptation. In Proceedings of the International Symposium on Quality of Service. 1--10.
[37]
Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2015. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. 65--76.
[38]
Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking. 617--628.
[39]
Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. 2018. Feature generating networks for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5542--5551.
[40]
Chunjing Xiao, Daojun Han, Yongsen Ma, and Zhiguang Qin. 2019. CsiGAN: Robust Channel State Information-based Activity Recognition with GANs. IEEE Internet of Things Journal 6, 6 (2019), 10191--10204.
[41]
Rui Xiao, Jianwei Liu, Jinsong Han, and Kui Ren. 2021. OneFi: one-shot recognition for unseen gesture via COTS WiFi. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems. 206--219.
[42]
Yaxiong Xie, Zhenjiang Li, and Mo Li. 2015. Precise power delay profiling with commodity WiFi. In Proceedings of the 21st Annual international conference on Mobile Computing and Networking. 53--64.
[43]
Hongfei Xue, Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shiyang Wang, Ye Yuan, Shuochao Yao, Aidong Zhang, and Lu Su. 2020. DeepMV: Multi-view deep learning for device-free human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--26.
[44]
Diana Yacchirema, Jara Suárez de Puga, Carlos Palau, and Manuel Esteve. 2019. Fall detection system for elderly people using IoT and ensemble machine learning algorithm. Personal and Ubiquitous Computing 23, 5 (2019), 801--817.
[45]
Jianfei Yang, Han Zou, and Lihua Xie. 2022. Securesense: defending adversarial attack for secure device-free human activity recognition. IEEE Transactions on Mobile Computing (2022).
[46]
Shuochao Yao, Ailing Piao, Wenjun Jiang, Yiran Zhao, Huajie Shao, Shengzhong Liu, Dongxin Liu, Jinyang Li, Tianshi Wang, Shaohan Hu, et al. 2019. Stfnets: Learning sensing signals from the time-frequency perspective with short-time fourier neural networks. In The World Wide Web Conference. 2192--2202.
[47]
Guolin Yin, Junqing Zhang, Guanxiong Shen, and Yingying Chen. 2022. FewSense, towards a scalable and cross-domain Wi-Fi sensing system using few-shot learning. arXiv preprint arXiv:2203.02014 (2022).
[48]
Youwei Zeng, Dan Wu, Jie Xiong, Enze Yi, Ruiyang Gao, and Daqing Zhang. 2019. FarSense: Pushing the range limit of WiFi-based respiration sensing with CSI ratio of two antennas. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--26.
[49]
Daqing Zhang, Hao Wang, and Dan Wu. 2017. Toward centimeter-scale human activity sensing with Wi-Fi signals. Computer 50, 1 (2017), 48--57.
[50]
Jin Zhang, Zhuangzhuang Chen, Chengwen Luo, Bo Wei, Salil S Kanhere, and Jianqiang Li. 2022. MetaGanFi: Cross-Domain Unseen Individual Identification Using WiFi Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--21.
[51]
Jie Zhang, Zhanyong Tang, Meng Li, Dingyi Fang, Petteri Nurmi, and Zheng Wang. 2018. CrossSense: Towards cross-site and large-scale WiFi sensing. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. 305--320.
[52]
Yi Zhang, Zheng Yang, Guidong Zhang, Chenshu Wu, and Li Zhang. 2021. XGest: Enabling Cross-Label gesture recognition with RF signals. ACM Transactions on Sensor Networks (TOSN) 17, 4 (2021), 1--23.
[53]
Yue Zheng, Yi Zhang, Kun Qian, Guidong Zhang, Yunhao Liu, Chenshu Wu, and Zheng Yang. 2019. Zero-effort cross-domain gesture recognition with Wi-Fi. In Proceedings of the 17th annual international conference on mobile systems, applications, and services. 313--325.
[54]
Yue Zheng, Yi Zhang, Kun Qian, Guidong Zhang, and Zheng Yang. 2019. Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi. In Proceedings of the International Symposium on Quality of Service. 1--10.
[55]
Zhipeng Zhou, Feng Wang, Jihong Yu, Ju Ren, Zhi Wang, and Wei Gong. 2022. Target-oriented Semi-supervised Domain Adaptation for WiFi-based HAR. In IEEE INFOCOM 2022-IEEE Conference on Computer Communications. IEEE, 420--429.
Index Terms
- MetaFormer: Domain-Adaptive WiFi Sensing with Only One Labelled Target Sample
Recommendations
Improving Semi-Supervised Text Classification with Dual Meta-Learning
The goal of semi-supervised text classification (SSTC) is to train a model by exploring both a small number of labeled data and a large number of unlabeled data, such that the learned semi-supervised classifier performs better than the supervised ...
Meta Multi-Instance Multi-Label learning by heterogeneous network fusion
AbstractMulti-Instance Multi-Label Learning (MIML) models complex objects (bags), each of which is composed with a set of instances and associated with a set of labels. Current MIML solutions still focus on a single-type of bags and assume an independent ...
Highlights- We study how to achieve meta learning on interdependent MIML data.
- We propose a context learner for extracting structure information of linked objects.
- We introduce two task learners for acquiring meta knowledge across tasks.
- ...
OneFi: One-Shot Recognition for Unseen Gesture via COTS WiFi
SenSys '21: Proceedings of the 19th ACM Conference on Embedded Networked Sensor SystemsWiFi-based Human Gesture Recognition (HGR) becomes increasingly promising for device-free human-computer interaction. However, existing WiFi-based approaches have not been ready for real-world deployment due to the limited scalability, especially for ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 March 2024
Published in IMWUT Volume 8, Issue 1
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Funding Sources
- Key Program of the National Natural Science Foundation of China
- Natural Science Foundation for Excellent Young Scholars of Jiangsu Province
- National Science Fund for Distinguished Young Scholars of China
- National Natural Science Foundation of China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 269Total Downloads
- Downloads (Last 12 months)269
- Downloads (Last 6 weeks)28
Reflects downloads up to 10 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in