research-article

Dynamic Weighted Adversarial Learning for Semi-Supervised Classification under Intersectional Class Mismatch

Authors:

Chen GongAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications and Applications, Volume 20, Issue 4

Article No.: 115, Pages 1 - 24

https://doi.org/10.1145/3635310

Published: 11 January 2024 Publication History

Abstract

Nowadays, class-mismatch problem has drawn intensive attention in Semi-Supervised Learning (SSL), where the classes of labeled data are assumed to be only a subset of the classes of unlabeled data. However, in a more realistic scenario, the labeled data and unlabeled data often share some common classes while they also have their individual classes, which leads to an “intersectional class-mismatch” problem. As a result, existing SSL methods are often confused by these individual classes and suffer from performance degradation. To address this problem, we propose a novel Dynamic Weighted Adversarial Learning (DWAL) framework to properly utilize unlabeled data for boosting the SSL performance. Specifically, to handle the influence of the individual classes in unlabeled data (i.e., Out-Of-Distribution classes), we propose an enhanced adversarial domain adaptation to dynamically assign weight for each unlabeled example from the perspectives of domain adaptation and a class-wise weighting mechanism, which consists of transferability score and prediction confidence value. Besides, to handle the influence of the individual classes in labeled data (i.e., private classes), we propose a dissimilarity maximization strategy to suppress the inaccurate correlations caused by the examples of individual classes within labeled data. Therefore, our DWAL can properly make use of unlabeled data to acquire an accurate SSL classifier under intersectional class-mismatch setting, and extensive experimental results on five public datasets demonstrate the effectiveness of the proposed model over other state-of-the-art SSL methods.

Supplementary Material

3635310-app (3635310-app.pdf)

Supplementary material

Download
102.41 KB

References

[1]

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the International Conference on Machine Learning. 41–48.

Digital Library

[2]

David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. 2019. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. In Proceedings of the International Conference on Learning Representations.

[3]

Kaidi Cao, Maria Brbic, and Jure Leskovec. 2022. Open-world semi-supervised learning. In Proceedings of the International Conference on Learning Representations.

[4]

Zhangjie Cao, Lijia Ma, Mingsheng Long, and Jianmin Wang. 2018. Partial adversarial domain adaptation. In Proceedings of the European Conference on Computer Vision. 135–150.

Digital Library

[5]

Zhangjie Cao, Kaichao You, Mingsheng Long, Jianmin Wang, and Qiang Yang. 2019. Learning to transfer examples for partial domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2985–2994.

[6]

Paola Cascante-Bonilla, Fuwen Tan, Yanjun Qi, and Vicente Ordonez. 2021. Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35, 6912–6920.

[7]

Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. 2009. Semi-supervised learning (Chapelle, O. et al., eds.; 2006)[book reviews]. IEEE Transactions on Neural Networks and Learning Systems 20, 3 (2009), 542–542.

Digital Library

[8]

Yanbei Chen, Xiatian Zhu, Wei Li, and Shaogang Gong. 2020. Semi-supervised learning under class distribution mismatch. In Proceedings of the International Conference on Machine Learning. Vol. 34, 3569–3576.

[9]

Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter. 2017. A downsampled variant of ImageNet as an alternative to the CIFAR datasets. CoRR abs/1707.08819 (2017).

[10]

Shuhao Cui, Xuan Jin, Shuhui Wang, Yuan He, and Qingming Huang. 2020. Heuristic domain adaptation. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 33, 7571–7583.

[11]

Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, and Qi Tian. 2020. Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3941–3950.

[12]

Zihang Dai, Zhilin Yang, Fan Yang, William W. Cohen, and Russ R. Salakhutdinov. 2017. Good semi-supervised learning that requires a bad GAN. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 30.

[13]

Jinhao Dong and Tong Lin. 2019. MarginGAN: Adversarial training in semi-supervised learning. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 32.

[14]

Sichao Fu, Weifeng Liu, Weili Guan, Yicong Zhou, Dapeng Tao, and Changsheng Xu. 2021. Dynamic graph learning convolutional networks for semi-supervised classification. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1s (2021), 1–13.

Digital Library

[15]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17, 1 (2016), 2096–2030.

[16]

Chen Gong, Dacheng Tao, Keren Fu, and Jie Yang. 2014. Fick’s law assisted propagation for semisupervised learning. IEEE Transactions on Neural Networks and Learning Systems 26, 9 (2014), 2148–2162.

[17]

Chen Gong, Dacheng Tao, Stephen J. Maybank, Wei Liu, Guoliang Kang, and Jie Yang. 2016. Multi-modal curriculum learning for semi-supervised image classification. IEEE Transactions on Image Process 25, 7 (2016), 3249–3260.

Digital Library

[18]

Chengyue Gong, Dilin Wang, and Qiang Liu. 2021. AlphaMatch: Improving consistency for semi-supervised learning with alpha-divergence. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 13683–13692.

[19]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems. Vol. 27.

[20]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Communications of the ACM 63, 11 (2020), 139–144.

Digital Library

[21]

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. In International Conference on Learning Representations.

[22]

Yves Grandvalet and Yoshua Bengio. 2005. Semi-supervised learning by entropy minimization. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 367, 281–296.

[23]

Lan-Zhe Guo, Yi-Ge Zhang, Zhi-Fan Wu, Jie-Jing Shao, and Yu-Feng Li. 2022. Robust semi-supervised learning when not all classes have labels. In Advances in Neural Information Processing Systems.

[24]

Lan-Zhe Guo, Zhen-Yu Zhang, Yuan Jiang, Yu-Feng Li, and Zhi-Hua Zhou. 2020. Safe deep semi-supervised learning for unseen-class unlabeled data. In Proceedings of the International Conference on Machine Learning. 3897–3906.

[25]

Junkai Huang, Chaowei Fang, Weikai Chen, Zhenhua Chai, Xiaolin Wei, Pengxu Wei, Liang Lin, and Guanbin Li. 2021. Trash to treasure: Harvesting OOD data with cross-modal matching for open-set semi-supervised learning. In Proceedings of the IEEE International Conference on Computer Vision. 8310–8319.

[26]

Zhuo Huang, Li Shen, Jun Yu, Bo Han, and Tongliang Liu. 2023. FlatMatch: Bridging labeled data and unlabeled data with cross-sharpness for semi-supervised learning. In Proceedings of the Conference on Neural Information Processing Systems.

[27]

Zhuo Huang, Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu. 2023. Harnessing out-of-distribution examples via augmenting content and style. In Proceedings of the International Conference on Learning Representations.

[28]

Zhuo Huang, Jian Yang, and Chen Gong. 2022. They are not completely useless: Towards recycling transferable unlabeled data for class-mismatched semi-supervised learning. IEEE Transactions on Multimedia 25 (2022), 1844–1857.

[29]

T. Joachims. 1999. Transductive inference for text classification using support vector machines. In Proceedings of the International Conference on Machine Learning. 200–209.

Digital Library

[30]

T. Joachims. 2003. Transductive learning via spectral graph partitioning. In Proceedings of the International Conference on Machine Learning. 290–297.

[31]

Zhanghan Ke, Daoye Wang, Qiong Yan, Jimmy Ren, and Rynson W. H. Lau. 2019. Dual student: Breaking the limits of the teacher in semi-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 6728–6736.

[32]

Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In Proceedings of the Conference and Workshop on Neural Information Processing Systems. Vol. 33, 18661–18673.

[33]

Vladimir Koltchinskii and Dmitry Panchenko. 2002. Empirical margin distributions and bounding the generalization error of combined classifiers. Annals of Statistics 30, 1 (2002), 1–50.

[34]

Abhishek Kumar, Prasanna Sattigeri, and Tom Fletcher. 2017. Semi-supervised learning with GANs: Manifold invariance with improved inference. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 30.

[35]

Samuli Laine and Timo Aila. 2016. Temporal ensembling for semi-supervised learning. In Proceedings of the International Conference on Learning Representations.

[36]

Dong-Hyun Lee. 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Proceedings of the International Conference on Machine Learning. Vol. 3, 896.

[37]

Chongxuan Li, Taufik Xu, Jun Zhu, and Bo Zhang. 2017. Triple generative adversarial nets. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 30.

Digital Library

[38]

Yang Li, Zhiqun Zhao, Hao Sun, Yigang Cen, and Zhihai He. 2020. Snowball: Iterative model evolution and confident sample discovery for semi-supervised learning on very small labeled datasets. IEEE Transactions on Multimedia 23 (2020), 1354–1366.

[39]

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. 2018. Conditional adversarial domain adaptation. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 31.

[40]

Takeru Miyato, Shin-Ichi Maeda, Masanori Koyama, and Shin Ishii. 2018. Virtual adversarial training: A regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 8 (2018), 1979–1993.

[41]

Bernice D. Mowery. 2011. The paired t-test. Pediatric Nursing 37, 6 (2011), 320–322.

[42]

Islam Nassar, Samitha Herath, Ehsan Abbasnejad, Wray Buntine, and Gholamreza Haffari. 2021. All labels are not created equal: Enhancing semi-supervision via label grouping and co-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7241–7250.

[43]

Youngtaek Oh, Dong-Jin Kim, and In So Kweon. 2022. DASO: Distribution-aware semantics-oriented pseudo-label for imbalanced semi-supervised learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9786–9796.

[44]

Avital Oliver, Augustus Odena, Colin Raffel, Ekin D. Cubuk, and Ian J. Goodfellow. 2018. Realistic evaluation of deep semi-supervised learning algorithms. In Proceedings of the International Conference on Neural Information Processing Systems. 3239–3250.

[45]

Sungrae Park, JunKeon Park, Su-Jin Shin, and Il-Chul Moon. 2018. Adversarial dropout for supervised and semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelligence.

[46]

Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S. Rawat, and Mubarak Shah. 2021. In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. In Proceedings of the International Conference on Learning Representations.

[47]

Kuniaki Saito, Donghyun Kim, and Kate Saenko. 2021. OpenMatch: Open-set consistency regularization for semi-supervised learning with outliers. In Proceedings of the International Conference on Neural Information Processing Systems. 25956–25967.

[48]

Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. 2016. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 29, 1163–1171.

[49]

Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. 2020. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In Proceedings of the International Conference on Neural Information Processing Systems. 596–608.

[50]

Teppei Suzuki and Ikuro Sato. 2020. Adversarial transformations for semi-supervised learning. In Proceedings of the International Conference on Machine Learning. Vol. 34, 5916–5923.

[51]

Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the International Conference on Learning Representations.

[52]

Wenjing Wang, Lilang Lin, Zejia Fan, and Jiaying Liu. 2023. Semi-supervised learning for Mars imagery classification and segmentation. ACM Transactions on Multimedia Computing, Communications and Applications 19, 4 (2023), 1–23.

Digital Library

[53]

Yunyun Wang, Songcan Chen, and Zhi-Hua Zhou. 2012. New semi-supervised classification method based on modified cluster assumption. IEEE Transactions on Neural Networks and Learning Systems 23, 5 (2012), 689–702.

[54]

Yu Wang, Pengchong Qiao, Chang Liu, Guoli Song, Xiawu Zheng, and Jie Chen. 2023. Out-of-distributed semantic pruning for robust semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 23849–23858.

[55]

Yingda Xia, Fengze Liu, Dong Yang, Jinzheng Cai, Lequan Yu, Zhuotun Zhu, Daguang Xu, Alan Yuille, and Holger Roth. 2020. 3d semi-supervised learning with uncertainty-aware multi-view co-training. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 3646–3655.

[56]

Jiayu Xiao, Liang Li, Chaofei Wang, Zheng-Jun Zha, and Qingming Huang. 2022. Few shot generative model adaption via relaxed spatial structural alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11204–11213.

[57]

Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V. Le. 2020. Self-training with noisy student improves ImageNet classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10687–10698.

[58]

Fan Yang, Kai Wu, Shuyi Zhang, Guannan Jiang, Yong Liu, Feng Zheng, Wei Zhang, Chengjie Wang, and Long Zeng. 2022. Class-aware contrastive semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14421–14430.

[59]

Yueming Yin, Zhen Yang, Haifeng Hu, and XiaofuWu. 2020. Universal multi-source domain adaptation. Pattern Recognition 121 (2020), 108238.

[60]

Kaichao You, Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. 2019. Universal domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2720–2729.

[61]

Qing Yu, Daiki Ikami, Go Irie, and Kiyoharu Aizawa. 2020. Multi-task curriculum framework for open-set semi-supervised learning. In Proceedings of the European Conference on Computer Vision. Springer, 438–454.

Digital Library

[62]

Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. 2021. FlexMatch: Boosting semi-supervised learning with curriculum pseudo labeling. In Proceedings of the International Conference on Neural Information Processing Systems. Vol. 34, 18408–18419.

[63]

Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2017. mixup: Beyond empirical risk minimization. In Proceedings of the International Conference on Learning Representations.

[64]

Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. 2019. Bridging theory and algorithm for domain adaptation. In Proceedings of the International Conference on Machine Learning. 7404–7413.

[65]

Yuhang Zhang, Xiaopeng Zhang, Jie Li, Robert Qiu, Haohang Xu, and Qi Tian. 2022. Semi-supervised contrastive learning with similarity co-calibration. IEEE Transactions on Multimedia 25 (2022), 1749–1759.

[66]

Jian Zhao, Xianhui Liu, and Weidong Zhao. 2022. Balanced and accurate pseudo-labels for semi-supervised image classification. ACM Transactions on Multimedia Computing, Communications and Applications 18, 3s (2022), 1–18.

Digital Library

[67]

Zhi-Hua Zhou and Ming Li. 2005. Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on Knowledge and Data Engineering 17, 11 (2005), 1529–1541.

Digital Library

[68]

Bowei Zhu and Yong Liu. 2021. General approximate cross validation for model selection: Supervised, semi-supervised and pairwise learning. In Proceedings of the ACM International Conference on Multimedia. 5281–5289.

Digital Library

[69]

Hui Zhu, Yongchun Lu, Hongbin Wang, Xunyi Zhou, Qin Ma, Yanhong Liu, Ning Jiang, Xin Wei, Linchengxi Zeng, and Xiaofang Zhao. 2022. Enhancing semi-supervised learning with cross-modal knowledge. In Proceedings of the ACM International Conference on Multimedia. 4456–4465.

Digital Library

Cited By

Wen HSong XChen XWei YNie LChua THui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657727(229-239)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657727
Huang ZLi MShen LYu JGong CHan BLiu T(2024)Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution GeneralizationInternational Journal of Computer Vision10.1007/s11263-024-02075-xOnline publication date: 1-Aug-2024
https://doi.org/10.1007/s11263-024-02075-x

Index Terms

Dynamic Weighted Adversarial Learning for Semi-Supervised Classification under Intersectional Class Mismatch
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations
  2. Machine learning
    1. Learning settings
      1. Semi-supervised learning settings
    2. Machine learning approaches
      1. Neural networks

Recommendations

Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

In multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Class-Wise Contrastive Prototype Learning for Semi-Supervised Classification Under Intersectional Class Mismatch
Traditional Semi-Supervised Learning (SSL) classification methods focus on leveraging unlabeled data to improve the model performance under the setting where labeled set and unlabeled set share the same classes. Nevertheless, the above-mentioned setting ...
Semi-supervised multi-label classification using incomplete label information
Highlights
- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
Abstract
Classifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 20, Issue 4

April 2024

676 pages

EISSN:1551-6865

DOI:10.1145/3613617

Editor:
Abdulmotaleb El Saddik
Mohamed Bin Zayed University of Artificial Intelligence, UAE and University of Ottawa, Canada

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 January 2024

Online AM: 02 December 2023

Accepted: 26 November 2023

Revised: 06 August 2023

Received: 05 May 2023

Published in TOMM Volume 20, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF of China
NSF of Jiangsu Province
NSF for Distinguished Young Scholar of Jiangsu Province
Fundamental Research Funds for the Central Universities
CAAI-Huawei MindSpore Open Fund, and “111” Program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
265
Total Downloads

Downloads (Last 12 months)265
Downloads (Last 6 weeks)33

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wen HSong XChen XWei YNie LChua THui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657727(229-239)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657727
Huang ZLi MShen LYu JGong CHan BLiu T(2024)Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution GeneralizationInternational Journal of Computer Vision10.1007/s11263-024-02075-xOnline publication date: 1-Aug-2024
https://doi.org/10.1007/s11263-024-02075-x

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents