research-article

i-Sample: Augment Domain Adversarial Adaptation Models for WiFi-based HAR

Authors:

Wei GongAuthors Info & Claims

ACM Transactions on Sensor Networks, Volume 20, Issue 2

Article No.: 38, Pages 1 - 20

https://doi.org/10.1145/3616494

Published: 10 January 2024 Publication History

Abstract

Recently, using deep learning to achieve WiFi-based human activity recognition (HAR) has drawn significant attention. While capable of achieving accurate identification in a single domain (i.e., training and testing in the same consistent WiFi environment), it would become extremely tough when WiFi environments change significantly. As such, domain adversarial neural networks-based approaches have been proposed to handle such diversities across domains, yet often found to share the same limitation in practice: the imbalance between high-capacity of feature extractors and data insufficiency of source domains.

This article proposes i-Sample, an intermediate sample generation-based framework, striving to tackle this issue for WiFi-based HAR. i-Sample is mainly designed as two-stage training, where four data augmentation operations are proposed to train a coarse domain-invariant feature extractor in the first stage. In the second stage, we leverage the gradients of classification error to generate intermediate samples to refine the classifiers together with original samples, making i-Sample also capable to be integrated into most domain adversarial adaptation methods without neural network modification. We have implemented a prototype system to evaluate i-Sample, which shows that i-Sample can effectively augment the performance of nowadays mainstream domain adversarial adaptation models for WiFi-based HAR, especially when source domain data is insufficient.

References

[1]

Kamran Ali, Alex X. Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke recognition using wifi signals. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. 90–102.

Digital Library

[2]

Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. Adv. Neural Info. Process. Syst. 29 (2016).

[3]

Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. 2021. Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. ACM Comput. Surveys 54, 4 (2021), 1–40.

[4]

Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. 2019. AutoAugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 113–123.

[5]

Shuhao Cui, Xuan Jin, Shuhui Wang, Yuan He, and Qingming Huang. 2020. Heuristic domain adaptation. Adv. Neural Info. Process. Syst. 33 (2020), 7571–7583.

[6]

Shuya Ding, Zhe Chen, Tianyue Zheng, and Jun Luo. 2020. RF-net: A unified meta-learning framework for RF-enabled one-shot human activity recognition. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems. 517–530.

Digital Library

[7]

Xiaoyi Fan, Fangxin Wang, Feng Wang, Wei Gong, and Jiangchuan Liu. 2019. When RFID meets deep learning: Exploring cognitive intelligence for activity identification. IEEE Wireless Commun. 26, 3 (2019), 19–25.

Digital Library

[8]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 1 (2016), 2096–2030.

[9]

Taesik Gong, Yeonsu Kim, Jinwoo Shin, and Sung-Ju Lee. 2019. Metasense: Few-shot adaptation to untrained conditions in deep mobile sensing. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems. 110–123.

Digital Library

[10]

Wei Gong, Si Chen, and Jiangchuan Liu. 2017. Towards higher throughput rate adaptation for backscatter networks. In Proceedings of the IEEE 25th International Conference on Network Protocols (ICNP’17). IEEE, 1–10.

[11]

Wei Gong and Jiangchuan Liu. 2017. Robust indoor wireless localization using sparse recovery. In Proceedings of the IEEE 37th International Conference on Distributed Computing Systems (ICDCS’17). IEEE, 847–856.

[12]

Wei Gong, Jiangchuan Liu, and Zhe Yang. 2016. Fast and reliable unknown tag detection in large-scale RFID systems. In Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing. 141–150.

Digital Library

[13]

Wei Gong, Jiangchuan Liu, and Zhe Yang. 2017. Efficient unknown tag detection in large-scale RFID systems with unreliable channels. IEEE/ACM Trans. Netw. 25, 4 (2017), 2528–2539.

Digital Library

[14]

Wei Gong, Ivan Stojmenovic, Amiya Nayak, Kebin Liu, and Haoxiang Liu. 2015. Fast and scalable counterfeits estimation for large-scale RFID systems. IEEE/ACM Trans. Netw. 24, 2 (2015), 1052–1064.

Digital Library

[15]

Wei Gong, Longzhi Yuan, Qiwei Wang, and Jia Zhao. 2020. Multiprotocol backscatter for personal IoT sensors. In Proceedings of the 16th International Conference on Emerging Networking EXperiments and Technologies. 261–273.

Digital Library

[16]

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. Retrieved from https://arXiv:1412.6572

[17]

Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall. 2011. Tool release: Gathering 802.11n traces with channel state information. ACM SIGCOMM Comput. Commun. Rev. 41, 1 (2011), 53–53.

Digital Library

[18]

Andrey Ignatov. 2018. Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl. Soft Comput. 62 (2018), 915–922.

[19]

Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shuochao Yao, Yaqing Wang, Ye Yuan, Hongfei Xue, Chen Song, Xin Ma, Dimitrios Koutsonikolas et al. 2018. Towards environment independent device free human activity recognition. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. 289–304.

Digital Library

[20]

Bruno Korbar, Du Tran, and Lorenzo Torresani. 2019. Scsampler: Sampling salient clips from video for efficient action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 6232–6242.

[21]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60, 6 (2017), 84–90.

Digital Library

[22]

Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2018. Adversarial examples in the physical world. In Artificial Intelligence Safety and Security. Chapman and Hall/CRC, 99–112.

[23]

Haoxiang Liu, Wei Gong, Lei Chen, Wenbo He, Kebin Liu, and Yunhao Liu. 2014. Generic composite counting in RFID systems. In Proceedings of the IEEE 34th International Conference on Distributed Computing Systems. IEEE, 597–606.

Digital Library

[24]

Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan. 2019. Transferable adversarial training: A general approach to adapting deep classifiers. In Proceedings of the International Conference on Machine Learning. PMLR, 4013–4022.

[25]

Kebin Liu, Qiang Ma, Wei Gong, Xin Miao, and Yunhao Liu. 2014. Self-diagnosis for detecting system failures in large-scale wireless sensor networks. IEEE Trans. Wireless Commun. 13, 10 (2014), 5535–5545.

[26]

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. Retrieved from https://arXiv:1706.06083

[27]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2017. Universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1765–1773.

[28]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. Deepfool: A simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2574–2582.

[29]

Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 2016. The limitations of deep learning in adversarial settings. In Proceedings of the IEEE European Symposium on Security and Privacy (EuroS&P’16). IEEE, 372–387.

[30]

Ronald K. Pearson, Yrjö Neuvo, Jaakko Astola, and Moncef Gabbouj. 2016. Generalized hampel filters. EURASIP J. Adv. Signal Process. 2016 (2016), 1–18.

[31]

Cong Shi, Jian Liu, Hongbo Liu, and Yingying Chen. 2017. Smart user authentication through actuation of daily activities leveraging WiFi-enabled IoT. In Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing. 1–10.

Digital Library

[32]

Zhenguo Shi, J. Andrew Zhang, Richard Yida Xu, and Qingqing Cheng. 2020. Environment-robust device-free human activity recognition with channel-state-information enhancement and one-shot learning. IEEE Trans. Mobile Comput. 21, 2 (2020), 540–554.

Digital Library

[33]

Connor Shorten and Taghi M. Khoshgoftaar. 2019. A survey on image data augmentation for deep learning. J. Big Data 6, 1 (2019), 1–48.

[34]

Rui Shu, Hung H. Bui, Hirokazu Narui, and Stefano Ermon. 2018. A dirt-t approach to unsupervised domain adaptation. Retrieved from arXiv:1802.08735.

[35]

Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. 2019. One pixel attack for fooling deep neural networks. IEEE Trans. Evolution. Comput. 23, 5 (2019), 828–841.

[36]

Tao Sun, Cheng Lu, Tianshuo Zhang, and Haibin Ling. 2022. Safe self-refinement for transformer-based domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7191–7200.

[37]

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. Retrieved from https://arXiv:1312.6199

[38]

Youssef Tamaazousti, Hervé Le Borgne, Céline Hudelot, Mohamed Tamaazousti et al. 2019. Learning more universal representations for transfer-learning. IEEE Trans. Pattern Anal. Mach. Intell. 42, 9 (2019), 2212–2224.

Digital Library

[39]

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7167–7176.

[40]

Fangxin Wang, Jiangchuan Liu, and Wei Gong. 2019. WiCAR: WiFi-based in-car activity recognition with multi-adversarial domain adaptation. In Proceedings of the International Symposium on Quality of Service. 1–10.

Digital Library

[41]

Jindong Wang, Yiqiang Chen, Han Yu, Meiyu Huang, and Qiang Yang. 2019. Easy transfer learning by exploiting intra-domain structures. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME’19). IEEE, 1210–1215.

[42]

Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, and Philip S. Yu. 2018. Visual domain adaptation with manifold embedded distribution alignment. In Proceedings of the 26th ACM International Conference on Multimedia. 402–410.

Digital Library

[43]

Kailong Wang, Cong Shi, Jerry Cheng, Yan Wang, Minge Xie, and Yingying Chen. 2022. Solving the WiFi sensing dilemma in reality leveraging conformal prediction. In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems. 407–420.

Digital Library

[44]

Yichao Wang, Yili Ren, Yingying Chen, and Jie Yang. 2022. Wi-mesh: A WiFi vision-based approach for 3D human mesh construction. In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems. 362–376.

Digital Library

[45]

Yuxi Wang, Kaishun Wu, and Lionel M. Ni. 2016. Wifall: Device-free fall detection by wireless networks. IEEE Trans. Mobile Comput. 16, 2 (2016), 581–594.

Digital Library

[46]

Zhengyang Wang, Sheng Chen, Wei Yang, and Yang Xu. 2021. Environment-independent wi-fi human activity recognition with adversarial network. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’21). IEEE, 3330–3334.

[47]

Chao-Yuan Wu, Manzil Zaheer, Hexiang Hu, R. Manmatha, Alexander J. Smola, and Philipp Krähenbühl. 2018. Compressed video action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6026–6035.

[48]

Tongkun Xu, Weihua Chen, Pichao Wang, Fan Wang, Hao Li, and Rong Jin. 2021. CDTrans: Cross-domain transformer for unsupervised domain adaptation. Retrieved from https://arXiv:2109.06165

[49]

Jihong Yu, Jiangchuan Liu, Rongrong Zhang, Lin Chen, Wei Gong, and Shurong Zhang. 2019. Multi-seed group labeling in RFID systems. IEEE Trans. Mobile Comput. 19, 12 (2019), 2850–2862.

[50]

David Zakim and Matthias Schwab. 2015. Data collection as a barrier to personalized medicine. Trends Pharmacol. Sci. 36, 2 (2015), 68–71.

[51]

Maolin Zhang, Si Chen, Jia Zhao, and Wei Gong. 2021. Commodity-level BLE backscatter. In Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services. 402–414.

Digital Library

[52]

Maolin Zhang, Jia Zhao, Si Chen, and Wei Gong. 2020. Reliable backscatter with commodity ble. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM’20). IEEE, 1291–1299.

Digital Library

[53]

Pengyu Zhang, Mohammad Rostami, Pan Hu, and Deepak Ganesan. 2016. Enabling practical backscatter communication for on-body sensors. In Proceedings of the ACM SIGCOMM Conference. 370–383.

Digital Library

[54]

Ziyi Zhang, Weikai Chen, Hui Cheng, Zhen Li, Siyuan Li, Liang Lin, and Guanbin Li. 2022. Divide and contrast: Source-free domain adaptation via adaptive contrastive learning. In Advances in Neural Information Processing Systems.

[55]

Jia Zhao, Wei Gong, and Jiangchuan Liu. 2020. Towards scalable backscatter sensor mesh with decodable relay and distributed excitation. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. 67–79.

Digital Library

[56]

Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. 2020. Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 13001–13008.

[57]

Zhipeng Zhou, Lanqing Li, Peilin Zhao, Pheng-Ann Heng, and Wei Gong. 2023. Class-conditional sharpness-aware minimization for deep long-tailed recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3499–3509.

[58]

Zhipeng Zhou, Feng Wang, Jihong Yu, Ju Ren, Zhi Wang, and Wei Gong. 2022. Target-oriented semi-supervised domain adaptation for WiFi-based HAR. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM’22). IEEE, 420–429.

Digital Library

Cited By

Cui WWu KWu MLi XChen Z(2024)WiCAU: Comprehensive Partial Adaptation With Uncertainty-Aware for WiFi-Based Cross-Environment Activity RecognitionIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.339809473(1-10)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3398094
Li ZNing SLian BWang CWei Z(2024)WiCARPervasive and Mobile Computing10.1016/j.pmcj.2024.101963103:COnline publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.pmcj.2024.101963
Wang ZChen YZheng HLiu MHuang P(2023)Body RFID Skeleton-Based Human Activity Recognition Using Graph Convolution Neural NetworkIEEE Transactions on Mobile Computing10.1109/TMC.2023.333304323:6(7301-7317)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1109/TMC.2023.3333043

Index Terms

i-Sample: Augment Domain Adversarial Adaptation Models for WiFi-based HAR
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
2. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing theory, concepts and paradigms

Recommendations

Cross-domain Recommendation via Adversarial Adaptation
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Data scarcity, e.g., labeled data being either unavailable or too expensive, is a perpetual challenge of recommendation systems. Cross-domain recommendation leverages the label information in the source domain to facilitate the task in the target ...
Cross-domain Recommendation via Dual Adversarial Adaptation
Data scarcity is a perpetual challenge of recommendation systems, and researchers have proposed a variety of cross-domain recommendation methods to alleviate the problem of data scarcity in target domains. However, in many real-world cross-domain ...
Variational inference based adversarial domain adaptation
Abstract
Deep learning methods demand enormous amounts of labeled data. Although collecting labeled data is a challenge but it can be accomplished with difficulty. Nevertheless, domain shift or bias may occur due to some conditions. Recollecting data under ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 20, Issue 2

March 2024

572 pages

EISSN:1550-4867

DOI:10.1145/3618080

Editor:
Wen Hu
University of New South Wales, Australia

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 10 January 2024

Online AM: 18 August 2023

Accepted: 10 August 2023

Revised: 26 June 2023

Received: 10 February 2023

Published in TOSN Volume 20, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSFC

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
476
Total Downloads

Downloads (Last 12 months)367
Downloads (Last 6 weeks)21

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cui WWu KWu MLi XChen Z(2024)WiCAU: Comprehensive Partial Adaptation With Uncertainty-Aware for WiFi-Based Cross-Environment Activity RecognitionIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.339809473(1-10)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3398094
Li ZNing SLian BWang CWei Z(2024)WiCARPervasive and Mobile Computing10.1016/j.pmcj.2024.101963103:COnline publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.pmcj.2024.101963
Wang ZChen YZheng HLiu MHuang P(2023)Body RFID Skeleton-Based Human Activity Recognition Using Graph Convolution Neural NetworkIEEE Transactions on Mobile Computing10.1109/TMC.2023.333304323:6(7301-7317)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1109/TMC.2023.3333043

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents