Cited By
View all- Wu WZhang YXing F(2023)Active Selection Transfer Learning AlgorithmNeural Processing Letters10.1007/s11063-023-11240-155:7(10093-10116)Online publication date: 1-Dec-2023
Active learning support vector machines with low-rank transformation
Authors: Liang Gong, 
Wenbo Guo, Yupu YangAuthors Info & Claims
Wenbo Guo, Yupu YangAuthors Info & ClaimsAbstract
Active learning has proven to be quite effective in a vast array of machine learning tasks. Despite the lower labeling cost of active learning, it has been shown that active learning still can not reach state-of-the-art performance on several classification tasks and is sensitive to initial state. In this work, we propose a novel algorithm to improve the performance of active learning and it’s robustness to initial state. More specifically, we integrate low-rank transformation (LRT) with active learning. In each iteration, LRT is applied to project original high dimensional data to a feature space where data are easier to be classified and then support vector machines classifier is updated in this feature space. As iteration goes on, active learning’s propriety of labeling data improves the performance of LRT, which further promotes the accuracy of SVM classifier. Experiment on several benchmark binary classification datasets results showed the proposed algorithm outperforms other active learning methods in accuracy and robustness.
References
[1]
A. Asuncion and D. Newman, Uci machine learning repository, 2007.
[2]
G. Baudat and F. Anouar, Kernel-based methods and function approximation, In Neural Networks, 2001. Proceedings. IJCNN’01. International Joint Conference on, volume 2, IEEE, 2001, pages 1244–1249.
[3]
Z. Bodó, Z. Minier and L. Csató, Active learning with clustering, In Active Learning and Experimental Design@ AISTATS, 2011, pages 127–139.
[4]
A. Bordes, S. Ertekin, J. Weston and L. Bottou, Fast kernel classifiers with online and active learning, Journal of Machine Learning Research 6(Sep) (2005), 1579–1619.
[5]
C.J. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery 2(2) (1998), 121–167.
[6]
E.J. Candès, X. Li, Y. Ma and J. Wright, Robust principal component analysis? Journal of the ACM (JACM) 58(3) (2011), 11.
[7]
C.-C. Chang and C.-J. Lin, Libsvm: a library for support vector machines, ACM Transactions on Intelligent Systems and Technology (TIST) 2(3) (2011), 27.
[8]
R.R. Curtin, J.R. Cline, N.P. Slagle, W.B. March, P. Ram, N.A. Mehta and A.G. Gray, Mlpack: A scalable c++ machine learning library, Journal of Machine Learning Research 14(Mar) (2013), 801–805.
[9]
S. Dasgupta and D. Hsu, Hierarchical sampling for active learning, In Proceedings of the 25th International Conference on Machine Learning, ACM, 2008, pages 208–215.
[10]
E. Elhamifar and R. Vidal, Sparse subspace clustering: Algorithm, theory and applications, IEEE Transactions on Pattern Analysis and Machine Intelligence 35(11) (2013), 2765–2781.
[11]
C. Fu, L. Gong and Y. Yang, An improved active learning method based on feature selection, 2015.
[12]
C.-J. Fu and Y.-P. Yang, A batch-mode active learning svm method based on semi-supervised clustering, Intelligent Data Analysis 19(2) (2015), 345–358.
[13]
W. Guo, C. Zhong and Y. Yang, Spectral clustering based active learning with applications to text classification, In MATEC Web of Conferences, volume 56. EDP Sciences, 2016.
[14]
S.C. Hoi, R. Jin, J. Zhu and M.R. Lyu, Semisupervised svm batch mode active learning with applications to image retrieval, ACM Transactions on Information Systems (TOIS) 27(3) (2009), 16.
[15]
C.-W. Hsu and C.-J. Lin, A comparison of methods for multiclass support vector machines, IEEE transactions on Neural Networks 13(2) (2002), 415–425.
[16]
W. Hu, W. Hu, N. Xie and S. Maybank, Unsupervised active learning based on hierarchical graph-theoretic clustering, IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) 39(5) (2009), 1147–1161.
[17]
H. Janssen, Monte-carlo based uncertainty analysis: Sampling efficiency and sampling convergence, Reliability Engineering & System Safety 109 (2013), 123–132.
[18]
A.J. Joshi, F. Porikli and N. Papanikolopoulos, Multi-class active learning for image classification, In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, IEEE, 2009, pages 2372–2379.
[19]
C.-P. Lee and C.-J. Lin, A study on l2-loss (squared hinge-loss) multiclass svm, Neural Computation 25(5) (2013), 1302–1323.
[20]
N. Nissim, R. Moskovitch, L. Rokach and Y. Elovici, Novel active learning methods for enhanced pc malware detection in windows os, Expert Systems with Applications 41(13) (2014), 5843–5857.
[21]
J. Platt et al., Sequential minimal optimization: A fast algorithm for training support vector machines, 1998.
[22]
R.B. Prudencio, C. Soares and T.B. Ludermir, Uncertainty sampling methods for selecting datasets in active meta-learning, In Neural Networks (IJCNN), The 2011 International Joint Conference on, IEEE, 2011, pages 1082–1089.
[23]
Q. Qiu and G. Sapiro, Learning transformations for clustering and classification, Journal of Machine Learning Research 16 (2015), 187–225.
[24]
B. Recht, M. Fazel and P.A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Review 52(3) (2010), 471–501.
[25]
N. Roy and A. McCallum, Toward optimal active learning through monte carlo estimation of error reduction, ICML, Williamstown, 2001, pp. 441–448.
[26]
C. Sanderson, Armadillo: An open source c++ linear algebra library for fast prototyping and computationally intensive experiments, 2010.
[27]
B. Settles, M. Craven and S. Ray, Multiple-instance active learning, In Advances in Neural Information Processing Systems, 2008, pp. 1289–1296.
[28]
H. Shao, B. Tong and E. Suzuki, Query by committee in a heterogeneous environment, In International Conference on Advanced Data Mining and Applications, Springer, 2012, pp. 186–198.
[29]
S. Sivaraman and M.M. Trivedi, A general active-learning framework for on-road vehicle recognition and tracking, IEEE Transactions on Intelligent Transportation Systems 11(2) (2010), 267–276.
[30]
B.K. Sriperumbudur and G.R. Lanckriet, A proof of convergence of the concave-convex procedure using zangwill’s theory, Neural Computation 24(6) (2012), 1391–1407.
[31]
Q. Wang, W. Guo, K. Zhang, I. Ororbia, G. Alexander, X. Xing, C.L. Giles and X. Liu, Learning adversary-resistant deep neural networks, arXiv preprint arXiv:1612.01401, 2016.
[32]
Q. Wang, W. Guo, K. Zhang, X. Xing, C.L. Giles and X. Liu, Random feature nullification for adversary resistant deep architecture, arXiv preprint arXiv:1610.01239, 2016.
[33]
G.A. Watson, Characterization of the subdifferential of some matrix norms, Linear Algebra and Its Applications 170 (1992), 33–45.
[34]
P. Wolfe, A duality theorem for non-linear programming, Quarterly of applied mathematics, 1961, pp. 239–244.
[35]
J. Yang et al., Automatically labeling video data using multi-class active learning, In Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on, IEEE, 2003, pp. 516–523.
[36]
A.L. Yuille and A. Rangarajan, The concave-convex procedure, Neural Computation 15(4) (2003), 915–936.
[37]
Z. Zhang, J. Wang and H. Zha, Adaptive manifold learning, IEEE Transactions on Pattern Analysis and Machine Intelligence 34(2) (2012), 253–265.
Index Terms
- Active learning support vector machines with low-rank transformation
Index terms have been assigned to the content through auto-classification.
Recommendations
Active Learning with Support Vector Machines for Tornado Prediction
ICCS '07: Proceedings of the 7th international conference on Computational Science, Part I: ICCS 2007In this paper, active learning with support vector machines (SVMs) is applied to the problem of tornado prediction. This method is used to predict which storm-scale circulations yield tornadoes based on the radar derived Mesocyclone Detection Algorithm (...
Support vector machine active learning with applications to text classification
Support vector machines have met with significant success in numerous real-world learning tasks. However, like most machine learning algorithms, they are generally applied using a randomly selected training set classified in advance. In many settings, ...
Sparse multikernel support vector regression machines trained by active learning
A method for the sparse multikernel support vector regression machines is presented. The proposed method achieves a high accuracy versus complexity ratio and allows the user to adjust the complexity of the resulting models. The sparse representation is ...
Comments
Information & Contributors
Information
Published In
© 2018 – IOS Press and the authors. All rights reserved.
Publisher
IOS Press
Netherlands
Publication History
Published: 01 January 2018
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
Cited By
View all- Wu WZhang YXing F(2023)Active Selection Transfer Learning AlgorithmNeural Processing Letters10.1007/s11063-023-11240-155:7(10093-10116)Online publication date: 1-Dec-2023