Uncertainty-Aware Pseudo-Labeling and Dual Graph Driven Network for Incomplete Multi-View Multi-Label Classification
Authors: 
Wulin Xie, Xiaohuan Lu, Yadong Liu, Jiang Long, Bob Zhang, Shuping Zhao, Jie WenAuthors Info & Claims
Wulin Xie, Xiaohuan Lu, Yadong Liu, Jiang Long, Bob Zhang, Shuping Zhao, Jie WenAuthors Info & ClaimsPages 6656 - 6665
Abstract
Multi-view multi-label classification has recently received extensive attention due to its wide-ranging applications across various fields, such as medical imaging and bioinformatics. However, views and labels are usually incomplete in practical scenarios, attributed to the uncertainties in data collection and manual labeling. To cope with this issue, we propose an uncertainty-aware pseudo-labeling and dual graph driven network (UPDGD-Net), which can fully leverage the supervised information of the available labels and feature information of available views. Different from the existing works, we leverage the label matrix to impose dual graph constraints on the embedded features of both view-level and label-level, which enables the method to maintain the inherent structure of the real data during the feature extraction stage. Furthermore, our network incorporates an uncertainty-aware pseudo-labeling strategy to fill the missing labels, which not only addresses the learning issue of incomplete multi-labels but also enables the method to explore more reliable supervised information to guide the network training. Extensive experiments on five datasets demonstrate that our method outperforms other state-of-the-art methods.
References
[1]
P. Duygulu, Kobus Barnard, J. F. G. de Freitas, and David A. Forsyth. 2002. Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In Proceedings of the 7th European Conference on Computer Vision-Part IV (ECCV '02). Springer-Verlag, Berlin, Heidelberg, 97--112.
[2]
Jihad El-Sana and Neta Sokolovsky. 2003. View-Dependent Rendering for Large Polygonal Models over Networks. 3, 2 (April 2003), 265--.
[3]
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2010. The Pascal Visual Object Classes (VOC) Challenge. International Journal of Computer Vision 88, 2 (June 2010), 303--338.
[4]
N. Fan and Cheng Jin. 2012. Geometric Invariants Construction for Semantic Scene Understanding from Multiple Views Inspired by the Human Visual System. 12, 2 (April 2012), 1250012. https://doi.org/10.1142/S021946781250012X
[5]
Xiang Fang, Yuchong Hu, Pan Zhou, and Dapeng Wu. 2021. Animc: A soft approach for autoweighted noisy and incomplete multiview clustering. IEEE Transactions on Artificial Intelligence 3, 2 (2021), 192--206.
[6]
Xiang Fang, Yuchong Hu, Pan Zhou, and Dapeng Oliver Wu. 2020. V3H: View variation and view heredity for incomplete multiview clustering. IEEE Transactions on Artificial Intelligence 1, 3 (2020), 233--247.
[7]
Xiang Fang, Yuchong Hu, Pan Zhou, and Dapeng OliverWu. 2021. Unbalanced incomplete multi-view clustering via the scheme of view evolution:Weak views are meat; strong views do eat. IEEE Transactions on Emerging Topics in Computational Intelligence 6, 4 (2021), 913--927.
[8]
Zihan Fang, Shide Du, Zhiling Cai, Shiyang Lan, Chunming Wu, Yanchao Tan, and Shiping Wang. 2024. Representation Learning Meets Optimization-Derived Networks: From Single-View to Multi-View. IEEE Transactions on Multimedia (2024), 1--13. https://doi.org/10.1109/TMM.2024.3383295
[9]
Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (New York, NY, USA) (ICML'16). JMLR.org, 1050--1059.
[10]
Michael Grubinger, Paul Clough, Henning Müller, and Thomas Deselaers. 2006. The IAPR TC12 Benchmark: A New Evaluation Resource for Visual Information Systems. Intl.conf.on Language Resources Evaluation (2006).
[11]
Jun-Yi Hang and Min-Ling Zhang. 2022. Collaborative Learning of Label Semantics and Deep Label-Specific Features for Multi-Label Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 12 (2022), 9860--9871. https://doi.org/10.1109/TPAMI.2021.3136592
[12]
Mark J. Huiskes and Michael S. Lew. 2008. The MIR Flickr Retrieval Evaluation. Association for Computing Machinery, New York, NY, USA.
[13]
A. Iwata, K. Kato, and K. Yamamoto. 2002. The Detection of Obstacles by the Horizon View Camera. 2, 2 (April 2002), 331--.
[14]
Jack Lanchantin, Tianlu Wang, Vicente Ordonez, and Yanjun Qi. 2021. General Multi-label Image Classification with Transformers. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16473--16483. https://doi. org/10.1109/CVPR46437.2021.01621
[15]
Changhee Lee and Mihaela van der Schaar. 2021. A Variational Information Bottleneck Approach to Multi-Omics Data Integration. In International Conference on Artificial Intelligence and Statistics.
[16]
Xiang Li and Songcan Chen. 2022. A Concise Yet Effective Model for Non-Aligned Incomplete Multi-View and Missing Multi-Label Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 10 (2022), 5918--5932. https: //doi.org/10.1109/TPAMI.2021.3086895
[17]
Xingfeng Li, Yinghui Sun, Quansen Sun, Zhenwen Ren, and Yuan Sun. 2023. Cross-view Graph Matching Guided Anchor Alignment for Incomplete Multiview Clustering. Inf. Fusion 100 (2023), 101941. https://api.semanticscholar.org/ CorpusID:260215512
[18]
Zhenghong Lin, Yanchao Tan, Yunfei Zhan, Weiming Liu, Fan Wang, Chaochao Chen, Shiping Wang, and Carl Yang. 2023. Contrastive Intra- and Inter-Modality Generation for Enhancing Incomplete Multimedia Recommendation. In Proceedings of the 31st ACM International Conference on Multimedia (Ottawa ON, Canada) (MM '23). Association for Computing Machinery, New York, NY, USA, 6234--6242. https://doi.org/10.1145/3581783.3612362
[19]
Bo Liu, Weibin Li, Yanshan Xiao, Xiaodong Chen, Laiwang Liu, Changdong Liu, Kai Wang, and Peng Sun. 2023. Multi-view multi-label learning with high-order label correlation. Inf. Sci. 624, C (may 2023), 165--184. https://doi.org/10.1016/j. ins.2022.12.072
[20]
Chengliang Liu, Jie Wen, Xiaoling Luo, Chao Huang, Zhihao Wu, and Yong Xu. 2023. DICNet: Deep Instance-Level Contrastive Network for Double Incomplete Multi-View Multi-Label Classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8807--8815.
[21]
Chengliang Liu, Jie Wen, Xiaoling Luo, and Yong Xu. 2023. Incomplete Multi- View Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8816--8824.
[22]
Xinwang Liu, Sihang Zhou, Yueqing Wang, Miaomiao Li, Yong Dou, En Zhu, and Jianping Yin. 2017. Optimal Neighborhood Kernel Clustering with Multiple Kernels. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (San Francisco, California, USA) (AAAI'17). AAAI Press, 2266--2272.
[23]
Hang Lv, Zehai Chen, Yacong Yang, Guofang Ma, Tan Yanchao, and Carl Yang. 2024. BoxCare: A Box Embedding Model for Disease Representation and Diagnosis Prediction in Healthcare Data. In Companion Proceedings of the ACM on Web Conference 2024 (Singapore, Singapore) (WWW '24). Association for Computing Machinery, New York, NY, USA, 1130--1133. https://doi.org/10.1145/3589335. 3651448
[24]
Fan Ma, Deyu Meng, Qi Xie, Zina Li, and Xuanyi Dong. 2017. Self-Paced Cotraining. In International Conference on Machine Learning. 2275--2284.
[25]
Fan Ma, Deyu Meng, Qi Xie, Zina Li, and Xuanyi Dong. 2017. Self-Paced Cotraining. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 2275--2284.
[26]
Jianghong Ma, Huiyue Sun, and Tong Zhu. 2023. Multiview Multilabel Classification with Group-based Feature and Label Selection. IEEE Transactions on Consumer Electronics (2023), 1--1. https://doi.org/10.1109/TCE.2023.3278457
[27]
Mamshad Nayeem Rizve, Kevin Duarte, Yogesh Singh Rawat, and Mubarak Shah. 2021. In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning. ArXiv abs/2101.06329 (2021). https://api.semanticscholar.org/CorpusID:231632854
[28]
Shiliang Sun and Daoming Zong. 2020. LCBM: A Multi-View Probabilistic Model for Multi-Label Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (2020), 2682--2696.
[29]
Yuan Sun, Yang Qin, Yongxiang Li, Dezhong Peng, Xi Peng, and Peng Hu. 2024. Robust Multi-View Clustering with Noisy Correspondence. IEEE Transactions on Knowledge and Data Engineering (2024), 1--14. https://doi.org/10.1109/TKDE. 2024.3423307
[30]
Yuan Sun, Zhenwen Ren, Peng Hu, Dezhong Peng, and Xu Wang. 2024. Hierarchical Consensus Hashing for Cross-Modal Retrieval. IEEE Transactions on Multimedia 26 (2024), 824--836. https://doi.org/10.1109/TMM.2023.3272169
[31]
Qiaoyu Tan, Guoxian Yu, Carlotta Domeniconi, Jun Wang, and Zili Zhang. 2018. Incomplete Multi-View Weak-Label Learning. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (Stockholm, Sweden) (IJCAI'18). AAAI Press, 2703--2709.
[32]
Qiaoyu Tan, Guoxian Yu, JunWang, Carlotta Domeniconi, and Xiangliang Zhang. 2021. Individuality- and Commonality-Based Multiview Multilabel Learning. IEEE Transactions on Cybernetics 51, 3 (2021), 1716--1727. https://doi.org/10.1109/ TCYB.2019.2950560
[33]
Yanchao Tan, Carl Yang, Xiangyu Wei, Yun Ma, and Xiaolin Zheng. 2021. Multi- Facet Recommender Networks with Spherical Optimization. 2021 IEEE 37th International Conference on Data Engineering (ICDE) (2021), 1524--1535. https: //api.semanticscholar.org/CorpusID:226244847
[34]
Chuan Tang, Kun Sun, Chang Tang, Xiao Zheng, Xinwang Liu, Jun-Jie Huang, andWei Zhang. 2023. Multi-view subspace clustering via adaptive graph learning and late fusion alignment. Neural networks : the official journal of the International Neural Network Society 165 (2023), 333--343. https://api.semanticscholar.org/ CorpusID:259067294
[35]
Luis von Ahn and Laura Dabbish. 2004. Labeling Images with a Computer Game. Association for Computing Machinery, New York, NY, USA.
[36]
Jiao Wang, Bin Wu, Zhenwen Ren, Hongying Zhang, and Yunhui Zhou. 2023. Multi-scale deep multi-view subspace clustering with self-weighting fusion and structure preserving. Expert Syst. Appl. 213, PB (mar 2023), 11 pages. https: //doi.org/10.1016/j.eswa.2022.119031
[37]
Jie Wen, Chengliang Liu, Shijie Deng, Yicheng Liu, Lunke Fei, Ke Yan, and Yong Xu. 2023. Deep Double Incomplete Multi-View Multi-Label Learning With Incomplete Labels and Missing Views. IEEE Transactions on Neural Networks and Learning Systems (2023), 1--13. https://doi.org/10.1109/TNNLS.2023.3260349
[38]
JieWen, Zheng Zhang, Lunke Fei, Bob Zhang, Yong Xu, Zhao Zhang, and Jinxing Li. 2023. A Survey on Incomplete Multiview Clustering. IEEE Transactions on Systems, Man, and Cybernetics: Systems 53, 2 (2023), 1136--1149. https://doi.org/10.1109/TSMC.2022.3192635
[39]
BaoyuanWu, Zhilei Liu, ShangfeiWang, Bao-Gang Hu, and Qiang Ji. 2014. Multilabel Learning with Missing Labels. In 2014 22nd International Conference on Pattern Recognition. 1964--1968. https://doi.org/10.1109/ICPR.2014.343
[40]
Wenqi Yang, Yansu Wang, Chang Tang, Hengjian Tong, Ao Wei, and Xia Wu. 2022. One step multi-view spectral clustering via joint adaptive graph learning and matrix factorization. Neurocomputing 524 (2022), 95--105. https: //api.semanticscholar.org/CorpusID:254781832
[41]
Kun Zhan, Changqing Zhang, Junpeng Guan, and Junsheng Wang. 2018. Graph Learning for Multiview Clustering. IEEE Transactions on Cybernetics 48, 10 (2018), 2887--2895. https://doi.org/10.1109/TCYB.2017.2751646
[42]
Changqing Zhang, Ziwei Yu, Qinghua Hu, Pengfei Zhu, Xinwang Liu, and Xiaobo Wang. 2018. Latent Semantic Aware Multi-View Multi-Label Classification. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence (New Orleans, Louisiana, USA) (AAAI'18/IAAI'18/EAAI'18). AAAI Press, Article 541, 8 pages.
[43]
Qingyang Zhang, Yake Wei, Zongbo Han, Huazhu Fu, Xi Peng, Cheng Deng, Qinghua Hu, Cai Xu, Jie Wen, Di Hu, and Changqing Zhang. 2024. Multimodal Fusion on Low-quality Data: A Comprehensive Survey. ArXiv abs/2404.18947 (2024). https://api.semanticscholar.org/CorpusID:269457475
[44]
Dawei Zhao, Qingwei Gao, Yixiang Lu, Dong Sun, and Yusheng Cheng. 2021. Consistency and diversity neural network multi-view multi-label learning. Knowl. Based Syst. 218 (2021), 106841.
[45]
Dawei Zhao, Qingwei Gao, Yixiang Lu, Dong Sun, and Yusheng Cheng. 2021. Consistency and Diversity Neural Network Multi-View Multi-Label Learning. Know.-Based Syst. 218, C (apr 2021), 11 pages. https://doi.org/10.1016/j.knosys.2021.106841
[46]
Changming Zhu, Yanchen Liu, Duoqian Miao, Yilin Dong, and Witold Pedrycz. 2023. Within- cross- consensus-view representation-based multi-view multi-label learning with incomplete data. Neurocomputing 557 (2023), 126729.
Index Terms
- Uncertainty-Aware Pseudo-Labeling and Dual Graph Driven Network for Incomplete Multi-View Multi-Label Classification
Recommendations
Deep Incomplete Multi-View Network Semi-Supervised Multi-Label Learning with Unbiased Loss
MM '24: Proceedings of the 32nd ACM International Conference on MultimediaDue to the explosive growth in data sources and label categories, multi-view multi-label learning has garnered widespread attention. However, multi-view multi-label data often exhibits incomplete features and a huge number of unlabeled instances, due to ...
Informative pseudo-labeling for graph neural networks with few labels
AbstractGraph neural networks (GNNs) have achieved state-of-the-art results for semi-supervised node classification on graphs. Nevertheless, the challenge of how to effectively learn GNNs with very few labels is still under-explored. As one of the ...
Semi-supervised multi-label classification using incomplete label information
Highlights- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
AbstractClassifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China under Grant
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 89Total Downloads
- Downloads (Last 12 months)89
- Downloads (Last 6 weeks)30
Reflects downloads up to 31 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in