TS-HCL: Hierarchical Layer-Wise Contrastive Learning for Unsupervised Domain Adaptation on Time-Series
Pages 31 - 45
Abstract
Time series data is increasingly prevalent in diverse sectors such as finance, IoT, and healthcare, with notable applications in neuroscience. Although neural networks exhibit proficiency in handling time series data, domain shift often impedes their effectiveness. To address this issue, we propose an innovative approach called Hierarchical Layer-wise Contrastive Learning for Unsupervised Domain Adaptation on Time-Series (TS-HCL). TS-HCL addresses three key aspects: cross-domain sample similarity, interference from noisy domain labels, and conditional distribution shifts. Firstly, commonalities are established across domains by treating domain feature representations at corresponding layers as positive pairs through domain-level contrastive learning. Secondly, Environment Label Smoothing (ELS) is introduced, encouraging the marginal discriminator to estimate soft probabilities, thereby alleviating the impact of domain label noise. Lastly, a conditional domain discriminator is designed to provide enhanced context and align conditional distributions. The proposed TS-HCL method exhibits performance in cross-domain scenarios, as demonstrated by its effectiveness across both public and private datasets, with particular excellence in medical applications.
References
[1]
Anguita, D., Ghio, A., Oneto, L., Parra, X., Reyes-Ortiz, J.L., et al.: A public domain dataset for human activity recognition using smartphones. In: Esann, vol. 3, p. 3 (2013)
[2]
Arsenault C, Gage A, Kim MK, Kapoor NR, Akweongo P, Amponsah F, Aryal A, Asai D, Awoonor-Williams JK, Ayele W, et al. Covid-19 and resilience of healthcare systems in ten countries Nat. Med. 2022 28 6 1314-1324
[3]
Cai, R., Chen, J., Li, Z., Chen, W., Zhang, K., Ye, J., Li, Z., Yang, X., Zhang, Z.: Time series domain adaptation via sparse associative structure alignment. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 6859–6867 (2021)
[4]
Chen, C., et al.: Homm: higher-order moment matching for unsupervised domain adaptation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 3422–3429 (2020)
[5]
Cheng D, Yang F, Xiang S, and Liu J Financial time series forecasting with multi-modality graph neural network Pattern Recogn. 2022 121
[6]
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C.K., Li, X.: Cotmix: contrastive domain adaptation for time-series via temporal mixup. arXiv preprint arXiv:2212.01555 (2022)
[7]
Ganin Y et al. Domain-adversarial training of neural networks J. Mach. Learn. Res. 2016 17 59 1-35
[8]
He, H., Queen, O., Koker, T., Cuevas, C., Tsiligkaridis, T., Zitnik, M.: Domain adaptation for time series under feature and label shifts. arXiv preprint arXiv:2302.03133 (2023)
[9]
Kwapisz JR, Weiss GM, and Moore SA Activity recognition using cell phone accelerometers ACM SIGKDD Explorations Newsl 2011 12 2 74-82
[10]
Liu, Q., Xue, H.: Adversarial spectral kernel matching for unsupervised time series domain adaptation. In: IJCAI, pp. 2744–2750 (2021)
[11]
Long, M., Cao, Z., Wang, J., Jordan, M.I.: Conditional adversarial domain adaptation. Advances in neural information processing systems 31 (2018)
[12]
Ozyurt, Y., Feuerriegel, S., Zhang, C.: Contrastive learning for unsupervised domain adaptation of time series. arXiv preprint arXiv:2206.06243 (2022)
[13]
Qian, Q., Qin, Y., Luo, J., Xiao, D.: Cross-machine transfer fault diagnosis by ensemble weighting subdomain adaptation network. IEEE Trans. Ind. Electron. (2023)
[14]
Ragab, M., Eldele, E., Chen, Z., Wu, M., Kwoh, C.K., Li, X.: Self-supervised autoregressive domain adaptation for time series data. IEEE Trans. Neural Networks Learn. Syst. (2022)
[15]
Ragab M et al. Adatime: a benchmarking suite for domain adaptation on time series data ACM Trans. Knowl. Discov. Data 2023 17 8 1-18
[16]
Rahman, M.M., Fookes, C., Baktashmotlagh, M., Sridharan, S.: On minimum discrepancy estimation for deep domain adaptation. Domain Adaptation for Visual Understanding, pp. 81–94 (2020)
[17]
Shu, R., Bui, H.H., Narui, H., Ermon, S.: A dirt-t approach to unsupervised domain adaptation. arXiv preprint arXiv:1802.08735 (2018)
[18]
Stisen, A., et al.: Smart devices are different: assessing and mitigatingmobile sensing heterogeneities for activity recognition. In: Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, pp. 127–140 (2015)
[19]
Sun, B., Feng, J., Saenko, K.: Correlation alignment for unsupervised domain adaptation. Domain adaptation in computer vision applications, pp. 153–171 (2017)
[20]
Tang Y et al. A survey on machine learning models for financial time series forecasting Neurocomputing 2022 512 363-380
[21]
Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014)
[22]
Wang, Y., Han, Y., Wang, H., Zhang, X.: Contrast everything: a hierarchical contrastive framework for medical time-series. arXiv preprint arXiv:2310.14017 (2023)
[23]
Wang, Y., Han, Y., Wang, H., Zhang, X.: Contrast everything: a hierarchical contrastive framework for medical time-series. Advances in Neural Information Processing Systems 36 (2024)
[24]
Wang, Y., et al.: Sensor alignment for multivariate time-series unsupervised domain adaptation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 10253–10261 (2023)
[25]
Yu, H., et al.: Characterizing brain signals for epileptic pre-ictal signal classification. In: AMIA Annual Symposium Proceedings vol. 2021, p. 1215. American Medical Informatics Association (2021)
[26]
Zeiler MD and Fergus R Fleet D, Pajdla T, Schiele B, and Tuytelaars T Visualizing and understanding convolutional networks Computer Vision – ECCV 2014 2014 Cham Springer 818-833
[27]
Zhang, X., Yao, L., Wang, X., Monaghan, J., Mcalpine, D., Zhang, Y.: A survey on deep learning based brain computer interface: Recent advances and new frontiers. arXiv preprint arXiv:1905.0414966 (2019)
[28]
Zhang, Y., et al.: Free lunch for domain adversarial training: environment label smoothing. arXiv preprint arXiv:2302.00194 (2023)
[29]
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Object detectors emerge in deep scene cnns. arXiv preprint arXiv:1412.6856 (2014)
[30]
Zhu Y et al. Deep subdomain adaptation network for image classification IEEE Trans. Neural Networks Learn. Syst. 2020 32 4 1713-1722
Index Terms
- TS-HCL: Hierarchical Layer-Wise Contrastive Learning for Unsupervised Domain Adaptation on Time-Series
Index terms have been assigned to the content through auto-classification.
Recommendations
Domain consensual contrastive learning for few-shot universal domain adaptation
AbstractTraditional unsupervised domain adaptation (UDA) aims to transfer the learned knowledge from a fully labeled source domain to another unlabeled target domain on the same label set. The strong assumptions of full annotations on the source domain ...
Prototype-Augmented Contrastive Learning for Few-Shot Unsupervised Domain Adaptation
Knowledge Science, Engineering and ManagementAbstractUnsupervised domain adaptation aims to learn a classification model from the source domain with much-supervised information, which is applied to the utterly unsupervised target domain. However, collecting enough labeled source samples is difficult ...
Confidence-diffusion instance contrastive learning for unsupervised domain adaptation
AbstractUnsupervised domain adaptation (UDA) aims to utilize knowledge from a related but inconsistently distributed source domain for target model training. The challenge of significant domain divergence, known as domain drift, can impact model ...
Comments
Information & Contributors
Information
Published In
Aug 2024
524 pages
ISBN:978-981-97-7237-7
DOI:10.1007/978-981-97-7238-4
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 31 August 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025