research-article

Domain Adaptation with Representation Learning and Nonlinear Relation for Time Series

Authors:

Hazem HajjAuthors Info & Claims

ACM Transactions on Internet of Things, Volume 3, Issue 2

Article No.: 12, Pages 1 - 26

https://doi.org/10.1145/3502905

Published: 15 February 2022 Publication History

Abstract

In many real-world scenarios, machine learning models fall short in prediction performance due to data characteristics changing from training on one source domain to testing on a target domain. There has been extensive research to address this problem with Domain Adaptation (DA) for learning domain invariant features. However, when considering advances for time series, those methods remain limited to the use of hard parameter sharing (HPS) between source and target models, and the use of domain adaptation objective function. To address these challenges, we propose a soft parameter sharing (SPS) DA architecture with representation learning while modeling the relation as non-linear between parameters of source and target models and modeling the adaptation loss function as the squared Maximum Mean Discrepancy (MMD). The proposed architecture advances the state-of-the-art for time series in the context of activity recognition and in fields with other modalities, where SPS has been limited to a linear relation. An additional contribution of our work is to provide a study that demonstrates the strengths and limitations of HPS versus SPS. Experiment results showed the success of the method in three domain adaptation cases of multivariate time series activity recognition with different users and sensors.

References

[1]

Ali Akbari and Roozbeh Jafari. 2019. Transferring activity recognition models for new wearable sensors with deep generative domain adaptation. In Proceedings of the 18th International Conference on Information Processing in Sensor Networks. 85–96.

Digital Library

[2]

Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2007. Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems. 137–144.

Digital Library

[3]

Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. In Advances in Neural Information Processing Systems. 343–351.

Digital Library

[4]

Liangliang Cao, Zicheng Liu, and Thomas S. Huang. 2010. Cross-dataset action detection. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 1998–2005.

[5]

Rich Caruana. 1997. Multitask learning. Machine Learning 28, 1 (1997), 41–75.

Digital Library

[6]

Youngjae Chang, Akhil Mathur, Anton Isopoussu, Junehwa Song, and Fahim Kawsar. 2020. A systematic study of unsupervised domain adaptation for robust human-activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1–30.

Digital Library

[7]

Ricardo Chavarriaga, Hamidreza Bayati, and José Del Millán. 2013. Unsupervised adaptation for acceleration-based activity recognition: Robustness to sensor displacement and rotation. Personal and Ubiquitous Computing 17, 3 (2013), 479–490.

Digital Library

[8]

Ricardo Chavarriaga, Hamidreza Bayati, and José Del Millán. 2013. Unsupervised adaptation for acceleration-based activity recognition: Robustness to sensor displacement and rotation. Personal and Ubiquitous Computing 17, 3 (2013), 479–490.

Digital Library

[9]

Chenhui Chu and Rui Wang. 2018. A survey of domain adaptation for neural machine translation. arXiv preprint arXiv:1806.00258 (2018).

[10]

Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).

[11]

Razvan-Gabriel Cirstea, Darius-Valer Micu, Gabriel-Marcel Muresan, Chenjuan Guo, and Bin Yang. 2018. Correlated time series forecasting using multi-task deep neural networks. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. ACM, 1527–1530.

Digital Library

[12]

Diane Cook, Kyle D. Feuz, and Narayanan C. Krishnan. 2013. Transfer learning for activity recognition: A survey. Knowledge and Information Systems 36, 3 (2013), 537–556.

Digital Library

[13]

Jordana Dahmen, Alyssa La Fleur, Gina Sprint, Diane Cook, and Douglas L. Weeks. 2017. Using wrist-worn sensors to measure and compare physical activity changes for patients undergoing rehabilitation. (2017), 667–672.

[14]

Aras R. Dargazany, Paolo Stegagno, and Kunal Mankodiya. 2018. WearableDL: Wearable Internet-of-Things and deep learning for big data analytics–Concept, literature, and future. Mobile Information Systems 2018 (2018).

[15]

Renjie Ding, Xue Li, Lanshun Nie, Jiazhen Li, Xiandong Si, Dianhui Chu, Guozhong Liu, and Dechen Zhan. 2019. Empirical study and improvement on deep transfer learning for human activity recognition. Sensors 19, 1 (2019), 57.

[16]

David Eigen, Christian Puhrsch, and Rob Fergus. 2014. Depth map prediction from a single image using a multi-scale deep network. In Advances in Neural Information Processing Systems. 2366–2374.

Digital Library

[17]

Theodoros Evgeniou and Massimiliano Pontil. 2004. Regularized multi–task learning. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 109–117.

Digital Library

[18]

Thomas Fritz, Elaine M. Huang, Gail C. Murphy, and Thomas Zimmermann. 2014. Persuasive technology in the real world: A study of long-term use of activity sensing devices for fitness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 487–496.

Digital Library

[19]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17, 1 (2016), 2096–2030.

Digital Library

[20]

Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. Journal of Machine Learning Research 13, Mar (2012), 723–773.

Digital Library

[21]

Amir Hussein, Marc Djandji, Reem A. Mahmoud, Mohamad Dhaybi, and Hazem Hajj. 2020. Augmenting DL with adversarial training for robust prediction of epilepsy seizures. ACM Trans. Comput. Healthcare 1, 3, Article 18 (June 2020), 18 pages. DOI:https://doi.org/10.1145/3386580

Digital Library

[22]

Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. 2015. An empirical exploration of recurrent network architectures. In International Conference on Machine Learning. 2342–2350.

Digital Library

[23]

Md Abdullah Al Hafiz Khan and Nirmalya Roy. 2017. TransAct: Transfer learning enabled activity recognition. (2017), 545–550.

[24]

Md Abdullah Al Hafiz Khan, Nirmalya Roy, and Archan Misra. 2018. Scaling human activity recognition via deep learning-based domain adaptation. In 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1–9.

[25]

Jung Keun Lee, Stephen N. Robinovitch, and Edward J. Park. 2014. Inertial sensing-based pre-impact detection of falls involving near-fall scenarios. IEEE Transactions on Neural Systems and Rehabilitation Engineering 23, 2 (2014), 258–266.

[26]

Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101 (2016).

Digital Library

[27]

Shanhong Liu. 2019. Connected Wearable Devices Worldwide 2016-2022. Retrieved Jan 10, 2019 from https://www.statista.com/statistics/487291/global-connected-wearable-devices/.

[28]

Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. 2018. Transferable representation learning with deep adaptation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence (2018).

[29]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, Nov (2008), 2579–2605.

[30]

Takuya Maekawa, Daisuke Nakai, Kazuya Ohara, and Yasuo Namioka. 2016. Toward practical factory activity recognition: Unsupervised understanding of repetitive assembly work in a factory. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1088–1099.

Digital Library

[31]

Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. 2016. Cross-stitch networks for multi-task learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3994–4003.

[32]

Claude Nadeau and Yoshua Bengio. 2000. Inference for the generalization error. In Advances in Neural Information Processing Systems. 307–313.

Digital Library

[33]

Francisco Ordóñez and Daniel Roggen. 2016. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.

[34]

Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22, 10 (2010), 1345–1359.

Digital Library

[35]

Xin Qin, Yiqiang Chen, Jindong Wang, and Chaohui Yu. 2019. Cross-dataset activity recognition via adaptive spatial-temporal transfer learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 4 (2019), 1–25.

Digital Library

[36]

Sreenivasan Ramasamy Ramamurthy and Nirmalya Roy. 2018. Recent trends in machine learning for human activity recognition – A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery (2018), e1254.

[37]

Muhammad Rehman, Chee Liew, Teh Wah, Junaid Shuja, Babak Daghighi, et al. 2015. Mining personal data using smartphones and wearable devices: A survey. Sensors 15, 2 (2015), 4430–4469.

[38]

Bernardino Romera-Paredes, Andreas Argyriou, Nadia Berthouze, and Massimiliano Pontil. 2012. Exploiting unrelated tasks in multi-task learning. In International Conference on Artificial Intelligence and Statistics. 951–959.

[39]

A. Rozantsev, M. Salzmann, and P. Fua. 2019. Beyond sharing weights for deep domain adaptation. IEEE Annals of the History of Computing04 (2019), 801–814.

Digital Library

[40]

Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, and Anders Søgaard. 2017. Learning what to share between loosely related tasks. arXiv preprint arXiv:1705.08142 (2017).

[41]

Swami Sankaranarayanan, Yogesh Balaji, Carlos D. Castillo, and Rama Chellappa. 2018. Generate to adapt: Aligning domains using generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8503–8512.

[42]

Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjærgaard, Anind Dey, Tobias Sonne, and Mads Møller Jensen. 2015. Smart devices are different: Assessing and mitigating mobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. ACM, 127–140.

Digital Library

[43]

Masashi Sugiyama and Motoaki Kawanabe. 2012. Machine Learning in Non-stationary Environments: Introduction to Covariate Shift Adaptation. MIT press.

Digital Library

[44]

Xu Sun, Hisashi Kashima, and Naonori Ueda. 2013. Large-scale personalized human activity recognition using online multitask learning. IEEE Transactions on Knowledge and Data Engineering 25, 11 (2013), 2551–2563.

Digital Library

[45]

Melanie Swan. 2012. Sensor mania! the Internet of Things, wearable computing, objective metrics, and the quantified self 2.0. Journal of Sensor and Actuator Networks 1, 3 (2012), 217–253.

[46]

Timo Sztyler and Heiner Stuckenschmidt. 2016. On-body localization of wearable devices: An investigation of position-aware activity recognition. In 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1–9.

[47]

Dorra Trabelsi, Samer Mohammed, Faicel Chamroukhi, Latifa Oukhellou, and Yacine Amirat. 2013. An unsupervised approach for automatic activity recognition based on hidden Markov model regression. IEEE Transactions on Automation Science and Engineering 10, 3 (2013), 829–835.

[48]

Di Wang, Edwin Candinegara, Junhui Hou, Ah-Hwee Tan, and Chunyan Miao. 2017. Robust human activity recognition using lesser number of wearable sensors. In 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC). IEEE, 290–295.

[49]

Jindong Wang, Yiqiang Chen, Lisha Hu, Xiaohui Peng, and S. Yu Philip. 2018. Stratified transfer learning for cross-domain activity recognition. In IEEE International Conference on Pervasive Computing and Communications (PerCom’18). IEEE, 1–10.

[50]

Jindong Wang, Vincent W. Zheng, Yiqiang Chen, and Meiyu Huang. 2018. Deep transfer learning for cross-domain activity recognition. In Proceedings of the 3rd International Conference on Crowd Science and Engineering. ACM, 16.

Digital Library

[51]

Mei Wang and Weihong Deng. 2018. Deep visual domain adaptation: A survey. Neurocomputing 312 (2018), 135–153.

Digital Library

[52]

Jiahui Wen, Jadwiga Indulska, and Mingyang Zhong. 2016. Adaptive activity learning with dynamically available context. In IEEE International Conference on Pervasive Computing and Communications (PerCom’16). IEEE, 1–11.

[53]

Garrett Wilson and Diane J. Cook. 2020. A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST) 11, 5 (2020), 1–46.

Digital Library

[54]

Qian Xu, Sinno Jialin Pan, Hannah Hong Xue, and Qiang Yang. 2011. Multitask learning for protein subcellular location prediction. IEEE/ACM Transactions on Computational Biology and Bioinformatics 8, 3 (2011), 748–759.

Digital Library

[55]

Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor, and Rob Fergus. 2010. Deconvolutional networks. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 2528–2535.

Cited By

He YBian YDing XWu BGuan JZhang JZhou S(2024)Variate Associated Domain Adaptation for Unsupervised Multivariate Time Series Anomaly DetectionACM Transactions on Knowledge Discovery from Data10.1145/366357318:8(1-24)Online publication date: 3-May-2024
https://dl.acm.org/doi/10.1145/3663573
Ren LCheng X(2024)Single/Multi-Source Black-Box Domain Adaption for Sensor Time Series DataIEEE Transactions on Cybernetics10.1109/TCYB.2023.330083254:8(4712-4723)Online publication date: Aug-2024
https://doi.org/10.1109/TCYB.2023.3300832
Liu YLi DWang JLi BHang B(2024)MDLRInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10363861:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2023.103638
Show More Cited By

Index Terms

Domain Adaptation with Representation Learning and Nonlinear Relation for Time Series

Recommendations

Domain Adaptation for Face Recognition: Targetize Source Domain Bridged by Common Subspace

In many applications, a face recognition model learned on a source domain but applied to a novel target domain degenerates even significantly due to the mismatch between the two domains. Aiming at learning a better face recognition model for the target ...
Meta-learning for efficient unsupervised domain adaptation
Abstract
The standard machine learning assumption that training and test data are drawn from the same probability distribution does not hold in many real-world applications due to the inability to reproduce testing conditions at training time. Existing ...
Adversarial domain adaptation using contrastive learning
Abstract
In recent research, problems with biased datasets or domain shift have presented challenges to the practical applications of deep learning methods. In this paper, we propose a simple method using adversarial learning combined with contrastive ...
Highlights
- We investigated an unsupervised domain adaptation task.
- Combining adversarial and contrastive learning can address the domain-shift issue.
- The proposed method is practical and implementation is very simple.

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet of Things

ACM Transactions on Internet of Things Volume 3, Issue 2

May 2022

214 pages

EISSN:2577-6207

DOI:10.1145/3505220

Editors:
Schahram Dustdar
TU Wien, Austria
,
Gian Pietro Picco
University of Trento, Italy

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 15 February 2022

Accepted: 01 November 2021

Revised: 01 June 2021

Received: 01 September 2020

Published in TIOT Volume 3, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
1,276
Total Downloads

Downloads (Last 12 months)247
Downloads (Last 6 weeks)11

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

He YBian YDing XWu BGuan JZhang JZhou S(2024)Variate Associated Domain Adaptation for Unsupervised Multivariate Time Series Anomaly DetectionACM Transactions on Knowledge Discovery from Data10.1145/366357318:8(1-24)Online publication date: 3-May-2024
https://dl.acm.org/doi/10.1145/3663573
Ren LCheng X(2024)Single/Multi-Source Black-Box Domain Adaption for Sensor Time Series DataIEEE Transactions on Cybernetics10.1109/TCYB.2023.330083254:8(4712-4723)Online publication date: Aug-2024
https://doi.org/10.1109/TCYB.2023.3300832
Liu YLi DWang JLi BHang B(2024)MDLRInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10363861:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2023.103638
Wilson GDoppa JCook D(2023)CALDA: Improving Multi-Source Time Series Domain Adaptation With Contrastive Adversarial LearningIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.329834645:12(14208-14221)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3298346
Singhal PWalambe RRamanna SKotecha K(2023)Domain Adaptation: Challenges, Methods, Datasets, and ApplicationsIEEE Access10.1109/ACCESS.2023.323702511(6973-7020)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3237025
Shi YYing XYang J(2022)Deep Unsupervised Domain Adaptation with Time Series Sensor Data: A SurveySensors10.3390/s2215550722:15(5507)Online publication date: 23-Jul-2022
https://doi.org/10.3390/s22155507
He QSiu SSi Y(2022)Attentive recurrent adversarial domain adaptation with Top-k pseudo-labeling for time series classificationApplied Intelligence10.1007/s10489-022-04176-x53:11(13110-13129)Online publication date: 6-Oct-2022
https://dl.acm.org/doi/10.1007/s10489-022-04176-x

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents