research-article

InterBN: Channel Fusion for Adversarial Unsupervised Domain Adaptation

Authors:

Zhigang LuoAuthors Info & Claims

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Pages 3691 - 3700

https://doi.org/10.1145/3474085.3475481

Published: 17 October 2021 Publication History

Abstract

A classifier trained on one dataset rarely works on other datasets obtained under different conditions because of domain shifting. Such a problem is usually solved by domain adaptation methods. In this paper, we propose a novel unsupervised domain adaptation (UDA) method based on Interchangeable Batch Normalization (InterBN) to fuse different channels in deep neural networks for adversarial domain adaptation.Specifically, we first observe that the channels with small batch normalization scaling factor have less influence on the whole domain adaption, followed by a theoretical proof that the scaling factors for some channels will definitely come close to zero when imposing a sparsity regularization. Then, we replace the channels that have smaller scaling factors in the source domain with the mean of the channels which have larger scaling factors in the target domain or vice versa. Such a simple but effective channel fusion scheme can drastically increase the domain adaption ability.Extensive experimental results show that our InterBN significantly outperforms the current adversarial domain adaptation methods by a large margin on four visual benchmarks. In particular, InterBN achieves a remarkable improvement of 7.7% over the conditional adversarial adaptation networks (CDAN) on VisDA-2017 benchmark.

References

[1]

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).

[2]

Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine learning (2010), 151--175.

Digital Library

[3]

Woong-Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han. 2019. Domain-specific batch normalization for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7354--7362.

[4]

Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. 2019. Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. In International conference on machine learning. 1081--1090.

[5]

Tai-Te Chu, Chia-Chun Chang, An-Zi Yen, Hen-Hsen Huang, and Hsin-Hsi Chen. 2020. Multimodal Retrieval through Relations between Subjects and Objects in Lifelog Images. In Proceedings of the Third Annual Workshop on Lifelog Search Challenge. 51--55.

Digital Library

[6]

Zhijie Deng, Yucen Luo, and Jun Zhu. 2019. Cluster alignment with a teacher for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9944--9953.

[7]

Thomas Forgione, Axel Carlier, Géraldine Morin, Wei Tsang Ooi, Vincent Charvillat, and Praveen Kumar Yadav. 2018. An implementation of a dash client for browsing networked virtual environment. In Proceedings of the 26th ACM international conference on Multimedia. 1263--1264.

Digital Library

[8]

Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. 1180--1189.

Digital Library

[9]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The journal of machine learning research (2016), 2096--2030.

Digital Library

[10]

Yixiao Ge, Dapeng Chen, and Hongsheng Li. 2020. Mutual mean-teaching: Pseudo label refinery for unsupervised domain adaptation on person re-identification. International Conference on Learning Representations (2020).

[11]

Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. The Journal of Machine Learning Research (2012), 723--773.

Digital Library

[12]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[13]

Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. 2018. Cycada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning. 1989--1998.

[14]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. 448--456.

Digital Library

[15]

Minsoo Kang and Bohyung Han. 2020. Operation-Aware Soft Channel Pruning using Differentiable Masks. In International Conference on Machine Learning. 5122--5131.

[16]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE (1998), 2278--2324.

[17]

Jingjing Li, Erpeng Chen, Zhengming Ding, Lei Zhu, Ke Lu, and Zi Huang. 2019 a. Cycle-consistent conditional adversarial transfer networks. In Proceedings of the 27th ACM International Conference on Multimedia. 747--755.

Digital Library

[18]

Jingjing Li, Erpeng Chen, Zhengming Ding, Lei Zhu, Ke Lu, and Heng Tao Shen. 2020. Maximum density divergence for domain adaptation. IEEE transactions on pattern analysis and machine intelligence (2020).

[19]

Shuang Li, Chi Harold Liu, Binhui Xie, Limin Su, Zhengming Ding, and Gao Huang. 2019 b. Joint adversarial domain adaptation. In Proceedings of the 27th ACM International Conference on Multimedia. 729--737.

Digital Library

[20]

Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. 2016. Revisiting batch normalization for practical domain adaptation. International Conference on Learning Representations (2016).

[21]

Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. 2017. Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE International Conference on Computer Vision. 2736--2744.

[22]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. In International conference on machine learning. 97--105.

Digital Library

[23]

Mingsheng Long, Yue Cao, Jianmin Wang, and Philip S Yu. 2016a. Composite correlation quantization for efficient multimodal retrieval. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 579--588.

Digital Library

[24]

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2017a. Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems (2017).

Digital Library

[25]

Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip S Yu. 2014. Transfer joint matching for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1410--1417.

Digital Library

[26]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2016b. Unsupervised domain adaptation with residual transfer networks. International Conference on Neural Information Processing Systems (2016), 136--144.

Digital Library

[27]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2017b. Deep transfer learning with joint adaptation networks. In International conference on machine learning. 2208--2217.

Digital Library

[28]

Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, and Samuel Rota Bulo. 2017. Autodial: Automatic domain alignment layers. In Proceedings of the IEEE International Conference on Computer Vision. 5067--5075.

[29]

Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. 2018. Methods for interpreting and understanding deep neural networks. Digital Signal Processing (2018), 1--15.

[30]

Hideki Nakasone, Mats Remberger, Lu Tian, Petter Brodin, Bita Sahaf, Fang Wu, Jonas Mattsson, Robert Lowsky, Robert Negrin, David B Miklos, et al. 2015. Risks and benefits of sex-mismatched hematopoietic cell transplantation differ according to conditioning strategy. Haematologica (2015), 1477.

[31]

Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. 2017. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924 (2017).

[32]

Pedro O Pinheiro. 2018. Unsupervised domain adaptation with similarity learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8004--8013.

[33]

Murray Rosenblatt. 1956. A central limit theorem and a strong mixing condition. Proceedings of the National Academy of Sciences of the United States of America (1956), 43.

[34]

Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. 2010. Adapting visual category models to new domains. In European conference on computer vision. 213--226.

Digital Library

[35]

Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3723--3732.

[36]

Mark Schmidt, Glenn Fung, and Rmer Rosales. 2007. Fast optimization methods for l1 regularization: A comparative study and two new approaches. In European Conference on Machine Learning. 286--297.

Digital Library

[37]

Zhiqiang Tang, Yunhe Gao, Yi Zhu, Zhi Zhang, Mu Li, and Dimitris Metaxas. 2021. SelfNorm and CrossNorm for Out-of-Distribution Robustness. arXiv preprint arXiv:2102.02811 (2021).

[38]

Marco Toldo, Umberto Michieli, and Pietro Zanuttigh. 2021. Unsupervised Domain Adaptation in Semantic Segmentation via Orthogonal and Clustered Embeddings. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1358--1368.

[39]

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7167--7176.

[40]

Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014).

[41]

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016).

[42]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research (2008).

[43]

Vedran Vukotić, Christian Raymond, and Guillaume Gravier. 2016. Multimodal and crossmodal representation learning from textual and visual features with bidirectional deep neural networks for video hyperlinking. In Proceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia Fusion. 37--44.

Digital Library

[44]

Haotian Wang, Wenjing Yang, Ji Wang, Ruxin Wang, Long Lan, and Mingyang Geng. 2020 b. Pairwise Similarity Regularization for Adversarial Domain Adaptation. In Proceedings of the 28th ACM International Conference on Multimedia. 2409--2418.

Digital Library

[45]

Ximei Wang, Ying Jin, Mingsheng Long, Jianmin Wang, and Michael Jordan. 2019. Transferable normalization: Towards improving transferability of deep neural networks. (2019).

Digital Library

[46]

Yikai Wang, Wenbing Huang, Fuchun Sun, Tingyang Xu, Yu Rong, and Junzhou Huang. 2020 a. Deep multimodal fusion by channel exchanging. arXiv preprint arXiv:2011.05005 (2020).

[47]

Zhonghao Wang, Mo Yu, Yunchao Wei, Rogerio Feris, Jinjun Xiong, Wen-mei Hwu, Thomas S Huang, and Honghui Shi. 2020 c. Differential treatment for stuff and things: A simple unsupervised domain adaptation method for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12635--12644.

[48]

Yuxin Wu and Kaiming He. 2018. Group normalization. In Proceedings of the European conference on computer vision (ECCV). 3--19.

Digital Library

[49]

Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin. 2019. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1426--1435.

[50]

Xun Yang, Jianfeng Dong, Yixin Cao, Xun Wang, Meng Wang, and Tat-Seng Chua. 2020. Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval. In SIGIR. 1339--1348.

Digital Library

[51]

Xun Yang, Fuli Feng, Wei Ji, Meng Wang, and Tat-Seng Chua. 2021. Deconfounded Video Moment Retrieval with Causal Intervention. In SIGIR.

Digital Library

[52]

Xun Yang, Xiangnan He, Xiang Wang, Yunshan Ma, Fuli Feng, Meng Wang, and Tat-Seng Chua. 2019. Interpretable fashion matching with rich attributes. In SIGIR. 775--784.

Digital Library

[53]

Xun Yang, Meng Wang, Richang Hong, Qi Tian, and Yong Rui. 2017. Enhancing person re-identification in a self-trained subspace. TOMM (2017), 1--23.

Digital Library

[54]

Xun Yang, Peicheng Zhou, and Meng Wang. 2018. Person reidentification via structural deep metric learning. IEEE transactions on neural networks and learning systems, Vol. 30, 10 (2018), 2987--2998.

[55]

Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschl"ager, and Susanne Saminger-Platz. 2017. Central moment discrepancy (cmd) for domain-invariant representation learning. International Conference on Learning Representations (2017).

[56]

Weichen Zhang, Wanli Ouyang, Wen Li, and Dong Xu. 2018. Collaborative and adversarial network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3801--3809.

[57]

Yongchun Zhu, Fuzhen Zhuang, Jindong Wang, Guolin Ke, Jingwu Chen, Jiang Bian, Hui Xiong, and Qing He. 2020. Deep subdomain adaptation network for image classification. IEEE transactions on neural networks and learning systems (2020).

[58]

Yang Zou, Zhiding Yu, BVK Kumar, and Jinsong Wang. 2018. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European conference on computer vision (ECCV). 289--305.

Digital Library

Cited By

Wang YWu XLiu XChu FLiu HHan Z(2025)Label smoothing regularization-based no hyperparameter domain generalizationKnowledge-Based Systems10.1016/j.knosys.2024.112877309(112877)Online publication date: Jan-2025
https://doi.org/10.1016/j.knosys.2024.112877
Ying XPeng HXie J(2024)Big data analysis for Covid-19 in hospital information systemsPLOS ONE10.1371/journal.pone.029448119:5(e0294481)Online publication date: 22-May-2024
https://doi.org/10.1371/journal.pone.0294481
Wang MChen JWang YWang Sli LSu HGong ZWu KChen Z(2024)Joint Adversarial Domain Adaptation With Structural Graph AlignmentIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.330257411:1(604-612)Online publication date: Jan-2024
https://doi.org/10.1109/TNSE.2023.3302574
Show More Cited By

Index Terms

InterBN: Channel Fusion for Adversarial Unsupervised Domain Adaptation
1. Networks
  1. Network architectures
    1. Network design principles

Recommendations

CLDA: an adversarial unsupervised domain adaptation method with classifier-level adaptation
Abstract
Domain adaptation is an active and important research field in transfer learning. Unsupervised domain adaptation, which is better in line with real-world scenarios than supervised and semi-supervised domain adaptation, has attracted much attention ...
Semi-supervised adversarial discriminative domain adaptation
Abstract
Domain adaptation is a potential method to train a powerful deep neural network across various datasets. More precisely, domain adaptation methods train the model on training data and test that model on a completely separate dataset. The ...
Unsupervised domain adaptation with adversarial distribution adaptation network
Abstract
Adversarial domain adaptation is a powerful approach to transfer the knowledge of the label-rich source domain to the label-scarce target domain by mitigating domain shifts across distributions. Existing domain adaptation methods align either the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

October 2021

5796 pages

ISBN:9781450386517

DOI:10.1145/3474085

General Chairs:
Heng Tao Shen
University of Electronic Science&Technology of China, China
,
Yueting Zhuang
Zhejiang University, China
,
John R. Smith
IBM, USA
,
Program Chairs:
Yang Yang
University of Electronic Science and Technology of China, China
,
Pablo Cesar
CWI&TU Delft, The Netherlands
,
Florian Metze
FACEBOOK, Inc., USA
,
Balakrishnan Prabhakaran
University of Texas at Dallas, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation

Conference

MM '21

Sponsor:

SIGMM

MM '21: ACM Multimedia Conference

October 20 - 24, 2021

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
358
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)5

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang YWu XLiu XChu FLiu HHan Z(2025)Label smoothing regularization-based no hyperparameter domain generalizationKnowledge-Based Systems10.1016/j.knosys.2024.112877309(112877)Online publication date: Jan-2025
https://doi.org/10.1016/j.knosys.2024.112877
Ying XPeng HXie J(2024)Big data analysis for Covid-19 in hospital information systemsPLOS ONE10.1371/journal.pone.029448119:5(e0294481)Online publication date: 22-May-2024
https://doi.org/10.1371/journal.pone.0294481
Wang MChen JWang YWang Sli LSu HGong ZWu KChen Z(2024)Joint Adversarial Domain Adaptation With Structural Graph AlignmentIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.330257411:1(604-612)Online publication date: Jan-2024
https://doi.org/10.1109/TNSE.2023.3302574
Wang MLiu YYuan JWang SWang ZWang W(2024)Inter-Class and Inter-Domain Semantic Augmentation for Domain GeneralizationIEEE Transactions on Image Processing10.1109/TIP.2024.335442033(1338-1347)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3354420
Xu RLiu ZLuo YHu HShen LDu BKuang KYang J(2024)SGDA: Towards 3-D Universal Pulmonary Nodule Detection via Slice Grouped Domain AttentionIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2023.325371321:4(1093-1105)Online publication date: Jul-2024
https://doi.org/10.1109/TCBB.2023.3253713
Xu HShi CFan WChen Z(2024)Improving diversity and discriminability based implicit contrastive learning for unsupervised domain adaptationApplied Intelligence10.1007/s10489-024-05351-y54:20(10007-10017)Online publication date: 1-Aug-2024
https://doi.org/10.1007/s10489-024-05351-y
Li XGuan YFu BLuo Z(2024)Anomaly-aware symmetric non-negative matrix factorization for short text clusteringKnowledge and Information Systems10.1007/s10115-024-02226-zOnline publication date: 4-Nov-2024
https://doi.org/10.1007/s10115-024-02226-z
Zhang XTan ZLu FYan RLiu J(2024)Adaptive semi-supervised learning from stronger augmentation transformations of discrete text informationKnowledge and Information Systems10.1007/s10115-024-02100-y66:8(4609-4629)Online publication date: 11-Apr-2024
https://doi.org/10.1007/s10115-024-02100-y
Kong XJiang WJia JShi YXu RLiu SEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative PerceptionProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3611948(1943-1954)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3611948
Zhuo JZhao XCui SHuang QWang SEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Adaptive Feature Swapping for Unsupervised Domain AdaptationProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3611896(7017-7028)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3611896
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents