research-article

Imbalanced Source-free Domain Adaptation

Authors:

Zi HuangAuthors Info & Claims

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Pages 3330 - 3339

https://doi.org/10.1145/3474085.3475487

Published: 17 October 2021 Publication History

Abstract

Conventional Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a well-labeled source domain to an unlabeled target domain only when data from both domains is simultaneously accessible, which is challenged by the recent Source-free Domain Adaptation (SFDA). However, we notice that the performance of existing SFDA methods would be dramatically degraded by intra-domain class imbalance and inter-domain label shift. Unfortunately, class-imbalance is a common phenomenon in real-world domain adaptation applications. To address this issue, we present Imbalanced Source-free Domain Adaptation (ISFDA) in this paper. Specifically, we first train a uniformed model from the source domain, and then propose secondary label correction, curriculum sampling, plus intra-class tightening and inter-class separation to overcome the joint presence of covariate shift and label shift. Extensive experiments on three imbalanced benchmarks verify that ISFDA could perform favorably against existing UDA and SFDA methods under various conditions of class-imbalance, and outperform existing SFDA methods by over 15% in terms of per-class average accuracy on a large-scale long-tailed imbalanced dataset.

References

[1]

Kamyar Azizzadenesheli, Anqi Liu, Fanny Yang, and Animashree Anandkumar. 2019. Regularized learning for domain adaptation under label shifts. arXiv preprint arXiv:1903.09734 (2019).

[2]

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning. 41--48.

Digital Library

[3]

Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. arXiv preprint arXiv:1608.06019 (2016).

Digital Library

[4]

Zhangjie Cao, Lijia Ma, Mingsheng Long, and Jianmin Wang. 2018. Partial adversarial domain adaptation. In Proceedings of the European Conference on Computer Vision (ECCV). 135--150.

[5]

Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. 2019. Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. In International conference on machine learning. PMLR, 1081--1090.

[6]

Qi Dong, Shaogang Gong, and Xiatian Zhu. 2018. Imbalanced deep learning by minority class incremental rectification. IEEE transactions on pattern analysis and machine intelligence, Vol. 41, 6 (2018), 1367--1381.

[7]

Fangxiang Feng, Xiaojie Wang, and Ruifan Li. 2014. Cross-modal retrieval with correspondence autoencoder. In Proceedings of the 22nd ACM international conference on Multimedia. 7--16.

Digital Library

[8]

Thomas Forgione, Axel Carlier, Géraldine Morin, Wei Tsang Ooi, Vincent Charvillat, and Praveen Kumar Yadav. 2018. An implementation of a dash client for browsing networked virtual environment. In Proceedings of the 26th ACM international conference on Multimedia. 1263--1264.

Digital Library

[9]

Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. PMLR, 1180--1189.

Digital Library

[10]

Lianli Gao, Jingkuan Song, Xingyi Liu, Junming Shao, Jiajun Liu, and Jie Shao. 2017. Learning in high-dimensional multimedia data: the state of the art. Multimedia Systems, Vol. 23, 3 (2017), 303--313.

Digital Library

[11]

Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, David Balduzzi, and Wen Li. 2016. Deep reconstruction-classification networks for unsupervised domain adaptation. In European Conference on Computer Vision. Springer, 597--613.

[12]

Ryan Gomes, Andreas Krause, and Pietro Perona. 2010. Discriminative clustering by regularized information maximization. (2010).

Digital Library

[13]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[14]

Yunzhong Hou and Liang Zheng. 2021. Visualizing Adapted Knowledge in Domain Transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13824--13833.

[15]

Weihua Hu, Takeru Miyato, Seiya Tokui, Eiichi Matsumoto, and Masashi Sugiyama. 2017. Learning discrete representations via information maximizing self-augmented training. In International Conference on Machine Learning. PMLR, 1558--1567.

Digital Library

[16]

Xiang Jiang, Qicheng Lao, Stan Matwin, and Mohammad Havaei. 2020. Implicit class-conditioned domain alignment for unsupervised domain adaptation. In International Conference on Machine Learning. PMLR, 4816--4827.

[17]

Mingsheng Long Junguang Jiang, Bo Fu. 2020. Transfer-Learning-library. https://github.com/thuml/Transfer-Learning-Library.

[18]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, Vol. 25 (2012), 1097--1105.

Digital Library

[19]

Vinod K Kurmi, Venkatesh K Subramanian, and Vinay P Namboodiri. 2021. Domain Impression: A Source Data Free Domain Adaptation Method. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 615--625.

[20]

Jingjing Li, Erpeng Chen, Zhengming Ding, Lei Zhu, Ke Lu, and Zi Huang. 2019 a. Cycle-consistent conditional adversarial transfer networks. In Proceedings of the 27th ACM International Conference on Multimedia. 747--755.

Digital Library

[21]

Jingjing Li, Erpeng Chen, Zhengming Ding, Lei Zhu, Ke Lu, and Heng Tao Shen. 2020. Maximum density divergence for domain adaptation. IEEE transactions on pattern analysis and machine intelligence (2020).

[22]

Jingjing Li, Mengmeng Jing, Ke Lu, Lei Zhu, and Heng Tao Shen. 2019 b. Locality preserving joint transfer for domain adaptation. IEEE Transactions on Image Processing, Vol. 28, 12 (2019), 6103--6115.

Digital Library

[23]

Jingjing Li, Mengmeng Jing, Hongzu Su, Ke Lu, Lei Zhu, and Heng Tao Shen. 2021. Faster domain adaptation networks. IEEE Transactions on Knowledge and Data Engineering (2021).

[24]

Jingjing Li, Ke Lu, Zi Huang, Lei Zhu, and Heng Tao Shen. 2018a. Heterogeneous domain adaptation through progressive alignment. IEEE transactions on neural networks and learning systems, Vol. 30, 5 (2018), 1381--1391.

[25]

Jingjing Li, Ke Lu, Zi Huang, Lei Zhu, and Heng Tao Shen. 2018b. Transfer independently together: A generalized framework for domain adaptation. IEEE transactions on cybernetics, Vol. 49, 6 (2018), 2144--2155.

[26]

Shuang Li, Chi Harold Liu, Binhui Xie, Limin Su, Zhengming Ding, and Gao Huang. 2019 c. Joint adversarial domain adaptation. In Proceedings of the 27th ACM International Conference on Multimedia. 729--737.

Digital Library

[27]

Jian Liang, Dapeng Hu, and Jiashi Feng. 2020 a. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning. PMLR, 6028--6039.

[28]

Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, and Jiashi Feng. 2020 b. Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. arXiv preprint arXiv:2012.07297 (2020).

[29]

Omri Lifshitz and Lior Wolf. 2020. A sample selection approach for universal domain adaptation. arXiv preprint arXiv:2001.05071 (2020).

[30]

Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. 2018. Detecting and correcting for label shift with black box predictors. In International conference on machine learning. PMLR, 3122--3130.

[31]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. In International conference on machine learning. PMLR, 97--105.

Digital Library

[32]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2016. Unsupervised domain adaptation with residual transfer networks. arXiv preprint arXiv:1602.04433 (2016).

Digital Library

[33]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2017. Deep transfer learning with joint adaptation networks. In International conference on machine learning. PMLR, 2208--2217.

Digital Library

[34]

Ajinkya More. 2016. Survey of resampling techniques for improving classification performance in unbalanced datasets. arXiv preprint arXiv:1608.06048 (2016).

[35]

Rafael Müller, Simon Kornblith, and Geoffrey Hinton. 2019. When does label smoothing help? arXiv preprint arXiv:1906.02629 (2019).

Digital Library

[36]

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. 2019. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1406--1415.

[37]

Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. 2017. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924 (2017).

[38]

Mohammad Rostami and Aram Galstyan. 2020. Sequential unsupervised domain adaptation through prototypical distributions. arXiv preprint arXiv:2007.00197 (2020).

[39]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International journal of computer vision, Vol. 115, 3 (2015), 211--252.

Digital Library

[40]

Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3723--3732.

[41]

Yuan Shi and Fei Sha. 2012. Information-theoretical learning of discriminative clusters for unsupervised domain adaptation. arXiv preprint arXiv:1206.6438 (2012).

Digital Library

[42]

Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. 2018. A dirt-t approach to unsupervised domain adaptation. arXiv preprint arXiv:1802.08735 (2018).

[43]

Shuhan Tan, Xingchao Peng, and Kate Saenko. 2020. Class-Imbalanced Domain Adaptation: An Empirical Odyssey. In European Conference on Computer Vision. Springer, 585--602.

[44]

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. 2017. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5018--5027.

[45]

Vedran Vukoti?, Christian Raymond, and Guillaume Gravier. 2016. Multimodal and crossmodal representation learning from textual and visual features with bidirectional deep neural networks for video hyperlinking. In Proceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia Fusion. 37--44.

Digital Library

[46]

Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Heng Tao Shen. 2017. Adversarial cross-modal retrieval. In Proceedings of the 25th ACM international conference on Multimedia. 154--162.

Digital Library

[47]

Mei Wang and Weihong Deng. 2018. Deep visual domain adaptation: A survey. Neurocomputing, Vol. 312 (2018), 135--153.

Digital Library

[48]

Peter Wlodarczak, Jeffrey Soar, and Mustafa Ally. 2015. Multimedia data mining using deep learning. In 2015 Fifth International Conference on Digital Information Processing and Communications (ICDIPC). IEEE, 190--196.

[49]

Jun Yang, Rong Yan, and Alexander G Hauptmann. 2007. Cross-domain video concept detection using adaptive svms. In Proceedings of the 15th ACM international conference on Multimedia. 188--197.

Digital Library

[50]

Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. 2020. Unsupervised domain adaptation without source data by casting a bait. arXiv preprint arXiv:2010.12427 (2020).

[51]

Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. 2013. Domain adaptation under target and conditional shift. In International Conference on Machine Learning. PMLR, 819--827.

Digital Library

[52]

Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. 2019. Bridging theory and algorithm for domain adaptation. In International Conference on Machine Learning. PMLR, 7404--7413.

[53]

Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. 2019. On learning invariant representations for domain adaptation. In International Conference on Machine Learning. PMLR, 7523--7532.

Cited By

Xu ZChai ZXu CYuan CYang H(2024)Towards Effective Collaborative Learning in Long-Tailed RecognitionIEEE Transactions on Multimedia10.1109/TMM.2023.331498026(3754-3764)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3314980
Ru JTian JXiao CLi JShen H(2024)Imbalanced Open Set Domain Adaptation via Moving-Threshold Estimation and Gradual AlignmentIEEE Transactions on Multimedia10.1109/TMM.2023.329776826(2504-2514)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3297768
Jing MLi JLu KZhu LShen H(2024)Visually Source-Free Domain Adaptation via Adversarial Style MatchingIEEE Transactions on Image Processing10.1109/TIP.2024.335353933(1032-1044)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3353539
Show More Cited By

Index Terms

Imbalanced Source-free Domain Adaptation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Learning paradigms
      1. Multi-task learning
        Transfer learning

Recommendations

Mutual Domain Adaptation
Highlights
- We tackle a realistic problem setting of domain adaptation, where most domains are label-deficient and need to be helped and recent data become more sparsely labeled which makes the learning even more difficult.
- To tackle this problem, ...
Abstract
To solve the label sparsity problem, domain adaptation has been well-established, suggesting various methods such as finding a common feature space of different domains using projection matrices or neural networks. Despite recent advances, domain ...
Feature-level domain adaptation

Domain adaptation is the supervised learning setting in which the training and test data are sampled from different distributions: training data is sampled from a source domain, whilst test data is sampled from a target domain. This paper proposes and ...
Pairwise Adversarial Training for Unsupervised Class-imbalanced Domain Adaptation
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Unsupervised domain adaptation (UDA) has become an appealing approach for knowledge transfer from a labeled source domain to an unlabeled target domain. However, when the classes in source and target domains are imbalanced, most existing UDA methods ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

October 2021

5796 pages

ISBN:9781450386517

DOI:10.1145/3474085

General Chairs:
Heng Tao Shen
University of Electronic Science&Technology of China, China
,
Yueting Zhuang
Zhejiang University, China
,
John R. Smith
IBM, USA
,
Program Chairs:
Yang Yang
University of Electronic Science and Technology of China, China
,
Pablo Cesar
CWI&TU Delft, The Netherlands
,
Florian Metze
FACEBOOK, Inc., USA
,
Balakrishnan Prabhakaran
University of Texas at Dallas, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Sichuan Science and Technology Program
National Natural Science Foundation of China

Conference

MM '21

Sponsor:

SIGMM

MM '21: ACM Multimedia Conference

October 20 - 24, 2021

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
950
Total Downloads

Downloads (Last 12 months)240
Downloads (Last 6 weeks)31

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu ZChai ZXu CYuan CYang H(2024)Towards Effective Collaborative Learning in Long-Tailed RecognitionIEEE Transactions on Multimedia10.1109/TMM.2023.331498026(3754-3764)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3314980
Ru JTian JXiao CLi JShen H(2024)Imbalanced Open Set Domain Adaptation via Moving-Threshold Estimation and Gradual AlignmentIEEE Transactions on Multimedia10.1109/TMM.2023.329776826(2504-2514)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3297768
Jing MLi JLu KZhu LShen H(2024)Visually Source-Free Domain Adaptation via Adversarial Style MatchingIEEE Transactions on Image Processing10.1109/TIP.2024.335353933(1032-1044)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3353539
Li SWang ZLuo HDing LWu D(2024)T-TIME: Test-Time Information Maximization Ensemble for Plug-and-Play BCIsIEEE Transactions on Biomedical Engineering10.1109/TBME.2023.330328971:2(423-432)Online publication date: Feb-2024
https://doi.org/10.1109/TBME.2023.3303289
Fan CWang ZGuo DWang M(2024)Data-Free Quantization via Pseudo-label Filtering2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00534(5589-5598)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00534
Li ZLiu XZhang KLi CChen J(2024)Intelligent fault diagnosis under imbalanced multivariate working conditions leveraging dynamic unsupervised domain adaptation with sample and margin regularizationMeasurement Science and Technology10.1088/1361-6501/ad3fd435:7(076128)Online publication date: 26-Apr-2024
https://doi.org/10.1088/1361-6501/ad3fd4
Zhang NLu JLi KFang ZZhang G(2024)Source-Free Unsupervised Domain AdaptationNeurocomputing10.1016/j.neucom.2023.126921564:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.neucom.2023.126921
Zhang BZhang XHuang FMiao D(2024)Cross-domain knowledge collaboration for blending-target domain adaptationInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10373061:4Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103730
Shu XYan SLu ZWang XXie Y(2024)AdaTriplet-RA: Domain matching via adaptive triplet and reinforced attention for unsupervised domain adaptationSignal Processing: Image Communication10.1016/j.image.2023.117024120(117024)Online publication date: Jan-2024
https://doi.org/10.1016/j.image.2023.117024
Lü SLi ZZhang XLi J(2024)Consistency regularization-based mutual alignment for source-free domain adaptationExpert Systems with Applications10.1016/j.eswa.2023.122577241(122577)Online publication date: May-2024
https://doi.org/10.1016/j.eswa.2023.122577
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents