research-article

Multilabel relationship learning

Authors:

Dit-Yan YeungAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data (TKDD), Volume 7, Issue 2

Article No.: 7, Pages 1 - 30

https://doi.org/10.1145/2499907.2499910

Published: 02 August 2013 Publication History

Abstract

Multilabel learning problems are commonly found in many applications. A characteristic shared by many multilabel learning problems is that some labels have significant correlations between them. In this article, we propose a novel multilabel learning method, called MultiLabel Relationship Learning (MLRL), which extends the conventional support vector machine by explicitly learning and utilizing the relationships between labels. Specifically, we model the label relationships using a label covariance matrix and use it to define a new regularization term for the optimization problem. MLRL learns the model parameters and the label covariance matrix simultaneously based on a unified convex formulation. To solve the convex optimization problem, we use an alternating method in which each subproblem can be solved efficiently. The relationship between MLRL and two widely used maximum margin methods for multilabel learning is investigated. Moreover, we also propose a semisupervised extension of MLRL, called SSMLRL, to demonstrate how to make use of unlabeled data to help learn the label covariance matrix. Through experiments conducted on some multilabel applications, we find that MLRL not only gives higher classification accuracy but also has better interpretability as revealed by the label covariance matrix.

References

[1]

Argyriou, A., Evgeniou, T., and Pontil, M. 2008a. Convex multi-task feature learning. Mach. Learn. 73, 3, 243--272.

Digital Library

[2]

Argyriou, A., Micchelli, C. A., Pontil, M., and Ying, Y. 2008b. A spectral regularization framework for multi-task structure learning. Adv. Neural Inf. Process. Syst. 20, 25--32.

[3]

Boutell, M. R., Luo, J., Shen, X., and Brown, C. M. 2004. Learning multi-label scene classification. Pattern Recogn. 37, 9, 1757--1771.

[4]

Boyd, S. and Vandenberghe, L. 2004. Convex Optimization. Cambridge University Press, Cambridge, UK.

Digital Library

[5]

Bucak, S. S., Mallapragada, P. K., Jin, R., and Jain, A. K. 2009. Efficient multi-label ranking for multiclass learning: Application to object recognition. In Proceedings of the 12^th IEEE International Conference on Computer Vision.

[6]

Cesa-Bianchi, N., Gentile, C., and Zaniboni, L. 2006. Hierarchical classification: Combining bayes with svm. In Proceedings of the 23^rd International Conference on Machine Learning. 177--184.

Digital Library

[7]

Chang, C.-C. and Lin, C.-J. 2001. LIBSVM: A library for support vector machine. http://www.csie.ntu.edu.tw/&sim;cjlin/libsvm.

[8]

Chapelle, O., Zien, A., and Scholkopf, B., Eds. 2006. Semi-Supervised Learning. MIT Press.

[9]

Chen, G., Song, Y., Wang, F., and Zhang, C. 2008. Semi-supervised multi-label learning by solving a sylvester equation. In Proceedings of the SIAM International Conference on Data Mining. 410--419.

[10]

Clare, A. and King, R. D. 2001. Knowledge discovery in multi-label phenotype data. In Proceedings of the 5^th European Conference on Principles of Data Mining and Knowledge Discovery. 42--53.

Digital Library

[11]

Dinuzzo, F. and Fukumizu, K. 2011. Learning low-rank output kernels. In Proceedings of the 3^rd Asian Conference on Machine Learning. 181--196.

[12]

Dinuzzo, F., Ong, C. S., Gehler, P. V., and Pillonetto, G. 2011. Learning output kernels with block coordinate descent. In Proceedings of the 28^th International Conference on Machine Learning. 49--56.

[13]

Elisseeff, A. and Weston, J. 2001. A kernel method for multi-labelled classification. Adv. Neural Inf. Process. Syst. 14, 681--687.

[14]

Fan, R.-E., Chen, P.-H., and Lin, C.-J. 2005. Working set selection using second order information for training support vector machines. J. Mach. Learn. Res. 6, 1889--1918.

Digital Library

[15]

Furnkranz, J., Hullermeier, E., Mencia, E. L., and Brinker, K. 2008. Multilabel classification via calibrated label ranking. Mach. Learn. 73, 2, 133--153.

Digital Library

[16]

Gupta, A. K. and Nagar, D. K. 2000. Matrix Variate Distributions. Chapman and Hall.

[17]

Hariharan, B., Zelnik-Manor, L., Vishwanathan, S. V. N., and Varma, M. 2010. Large scale max-margin multi-label classification with priors. In Proceedings of the 27^th International Conference on Machine Learning. 423--430.

[18]

Ji, S., Tang, L., Yu, S., and Ye, J. 2010. A shared-subspace learning framework for multi-label classification. ACM Trans. Knowl. Discov. Data 4, 2, 1--29.

Digital Library

[19]

Joachims, T. 1998. Making large-scale svm learning practical. In Advances in Kernel Methods- Support Vector Learning, B. Scholkopf, C. J. C. Burges, and A. J. Smola, Eds., MIT Press, Cambridge, MA, 169--184.

Digital Library

[20]

Kwok, J. 1999. Moderating the outputs of support vector machine classifiers. IEEE Trans. Neural Netw. 10, 5, 1018--1031.

Digital Library

[21]

Liu, Y., Jin, R., and Yang, L. 2006. Semi-supervised multi-label learning by constrained non-negative matrix factorization. In Proceedings of the 21^st National Conference on Artificial Intelligence.

Digital Library

[22]

Read, J., Pfahringer, B., Holmes, G., and Frank, E. 2009. Classifier chains for multi-label classification. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases. 254--269.

Digital Library

[23]

Rousu, J., Saunders, C., Szedmak, S., and Shawe-Taylor, J. 2006. Kernel-based learning of hierarchical multilabel classification models. J. Mach. Learn. Res. 7, 1601--1626.

Digital Library

[24]

Schapire, R. E. and Singer, Y. 2000. Boostexter: A boosting-based system for text categorization. Mach. Learn. 39, 2--3, 135--168.

Digital Library

[25]

Seber, G. A. F. 2007. A Matrix Handbook for Statisticians. Wiley-Interscience.

Digital Library

[26]

Srebro, N., Rennie, J. D. M., and Jaakkola, T. 2004. Maximum-margin matrix factorization. Adv. Neural Inf. Process. Syst. 17, 1329--1336.

[27]

Tsoumakas, G. and Katakis, I. 2007. Multi-label classification: An overview. Int. J. Data Warehous. Min. 3, 3, 1--13.

[28]

Tsoumakas, G., Katakis, I., and Vlahavas, I. 2010. Mining multi-label data. In Data Mining and Knowledge Discovery Handbook, O. Maimon and L. Rokach, Eds., Springer, 667--685.

[29]

Vens, C., Struyf, J., Schietgat, L., Dzeroski, S., and Blockeel, H. 2008. Decision trees for hierarchical multi-label classification. Mach. Learn. 73, 2, 185--214.

Digital Library

[30]

Zha, Z.-J., Mei, T., Wang, J., Wang, Z., and Hua, X.-S. 2009. Graph-based semi-supervised learning with multiple labels. J. Vis. Comm. Image Represent. 20, 2, 97--103.

Digital Library

[31]

Zhang, M.-L. and Zhang, K. 2010. Multi-label learning by exploiting label dependency. In Proceedings of the 16^th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 999--1008.

Digital Library

[32]

Zhang, M.-L. and Zhou, Z.-H. 2006. Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans. Knowl. Data Engin. 18, 10, 1338--1351.

Digital Library

[33]

Zhang, M.-L. and Zhou, Z.-H. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recogn. 40, 7, 2038--2048.

Digital Library

[34]

Zhang, Y. and Yeung, D.-Y. 2010. A convex formulation for learning task relationships in multi-task learning. In Proceedings of the 26^th Conference on Uncertainty in Artificial Intelligence. 733--742.

[35]

Zhou, D., Bousquet, O., Lal, T. N., Weston, J., and Scholkopf, B. 2003. Learning with local and global consistency. Adv. Neural Inf. Process. Syst. 16, 321--328.

[36]

Zhu, X., Ghahramani, Z., and Lafferty, J. D. 2003. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20^th International Conference on Machine Learning. 912--919.

Cited By

Li YFu ZHe XHuang J(2023) An optimized multi-label TSK fuzzy system for emotion recognition of multimodal physiological signals * 2022 IEEE International Conference on Cyborg and Bionic Systems (CBS)10.1109/CBS55922.2023.10115349(362-367)Online publication date: 24-Mar-2023
https://doi.org/10.1109/CBS55922.2023.10115349
He ZZhang CLiu BLi B(2023)Label recovery and label correlation co-learning for multi-view multi-label classification with incomplete labelsApplied Intelligence10.1007/s10489-022-03945-y53:8(9444-9462)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s10489-022-03945-y
Qian KMin XCheng YSong GMin F(2023)Self-dependence multi-label learning with double k for missing labelsArtificial Intelligence Review10.1007/s10462-022-10279-156:6(5057-5094)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1007/s10462-022-10279-1
Show More Cited By

Index Terms

Multilabel relationship learning
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

Transductive Multilabel Learning via Label Set Propagation

The problem of multilabel classification has attracted great interest in the last decade, where each instance can be assigned with a set of multiple class labels simultaneously. It has a wide variety of real-world applications, e.g., automatic image ...
Exploiting Multilabel Information for Noise-Resilient Feature Selection
Research Survey and Regular Papers

In a conventional supervised learning paradigm, each data instance is associated with one single class label. Multilabel learning differs in the way that data instances may belong to multiple concepts simultaneously, which naturally appear in a variety ...
A Bayesian network correlation-based classifier chain algorithm for multilabel learning

This article introduces a new multilabel learning method, the Bayesian network correlation-based CC (BNCC) algorithm, to decrease the uncertainty in the label order from the classifier chain (CC) algorithm. It uses a neural network constructed in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 7, Issue 2

July 2013

107 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/2499907

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 August 2013

Accepted: 01 December 2012

Revised: 01 July 2012

Received: 01 April 2011

Published in TKDD Volume 7, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Research Grants Council, University Grants Committee, Hong Kong

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
770
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)1

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li YFu ZHe XHuang J(2023) An optimized multi-label TSK fuzzy system for emotion recognition of multimodal physiological signals * 2022 IEEE International Conference on Cyborg and Bionic Systems (CBS)10.1109/CBS55922.2023.10115349(362-367)Online publication date: 24-Mar-2023
https://doi.org/10.1109/CBS55922.2023.10115349
He ZZhang CLiu BLi B(2023)Label recovery and label correlation co-learning for multi-view multi-label classification with incomplete labelsApplied Intelligence10.1007/s10489-022-03945-y53:8(9444-9462)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s10489-022-03945-y
Qian KMin XCheng YSong GMin F(2023)Self-dependence multi-label learning with double k for missing labelsArtificial Intelligence Review10.1007/s10462-022-10279-156:6(5057-5094)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1007/s10462-022-10279-1
Zhang YYang Q(2022)A Survey on Multi-Task LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.307020334:12(5586-5609)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TKDE.2021.3070203
Lou QDeng ZXiao ZChoi KWang S(2022)Multilabel Takagi-Sugeno-Kang Fuzzy SystemIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2021.311596730:9(3410-3425)Online publication date: Sep-2022
https://doi.org/10.1109/TFUZZ.2021.3115967
Lughofer E(2022)Evolving multi-label fuzzy classifier with advanced robustness respecting human uncertaintyKnowledge-Based Systems10.1016/j.knosys.2022.109717255(109717)Online publication date: Nov-2022
https://doi.org/10.1016/j.knosys.2022.109717
Lughofer E(2022)Evolving multi-label fuzzy classifierInformation Sciences10.1016/j.ins.2022.03.045597(1-23)Online publication date: Jun-2022
https://doi.org/10.1016/j.ins.2022.03.045
Wang LDing ZFu Y(2021)Generic Multi-label Annotation via Adaptive Graph and Marginalized AugmentationACM Transactions on Knowledge Discovery from Data10.1145/345188416:1(1-20)Online publication date: 20-Jul-2021
https://dl.acm.org/doi/10.1145/3451884
Zhao DGao QLu YSun DCheng Y(2021)Consistency and diversity neural network multi-view multi-label learningKnowledge-Based Systems10.1016/j.knosys.2021.106841(106841)Online publication date: Feb-2021
https://doi.org/10.1016/j.knosys.2021.106841
Tian YZhang Y(2021)A comprehensive survey on regularization strategies in machine learningInformation Fusion10.1016/j.inffus.2021.11.005Online publication date: Nov-2021
https://doi.org/10.1016/j.inffus.2021.11.005
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents