Computer Science and Information Systems 2024 Volume 21, Issue 1, Pages: 143-166
https://doi.org/10.2298/CSIS230124001D
Full text (
927 KB)
MK-MSMCR: An efficient multiple kernel approach to multi-class classification
Dong Zijie (School of Mathematics and Statistics, Bigdata Modeling and Intelligent Computing research institute, Hubei University of Education, Second Gaoxin Road, Wuhan, China + Hubei Key Laboratory of Applied Mathematics, Faculty of Mathematics and Statistics, Hubei University, Wuhan, China), zjdong07@163.com
Chen Fen (School of Finance, Hubei University of Economics, Wuhan, China), fenfen_chen@163.com
Yu Zhang (School of Mathematics and Statistics, Hubei University of Education, Wuhan, China), romeozyu@163.com
This paper introduces a novel multi-class support vector classification and regression (MSVCR) algorithm with multiple kernel learning (MK-MSVCR). We present a new MK-MSVCR algorithm based on two-stage learning (MK-MSVCRTSL). The two-stage learning aims to make classification algorithms better when dealing with complex data by using the first stage of learning to generate ”representative” or ”important” samples. We first establish the fast learning rate of MKMSVCR algorithm for multi-class classification with independent and identically distributed (i.i.d.) samples amd uniformly ergodic Markov chain (u.e.M.c.) smaples, and prove that MK-MSVCR algorithm is consistent. We show the numerical investigation on the learning performance of MK-MSVCR-TSL algorithm. The experimental studies indicate that the proposed MK-MSVCR-TSL algorithm has better learning performance in terms of prediction accuracy, sampling and training total time than other multi-class classification algorithms.
Keywords: multi-class classification, multiple kernel learning, learning rate, support vector classification and regression
Show references
Angulo, C., Parra, X., Català, A.: K-svcr: A support vector machine for multi-class classification. Neurocomputing 55(1¨C2), 57-77 (2003)
Bennett, K., Mangasarian, O.L.: Combining support vector and mathematical programming methods for induction. Advances in Kernel Methods-SV Learning pp. 307-326 (1999)
Bottou, L., Cortes, C., Denker, J.S., Drucker, H., Guyon, I., Jackel, L.D., LeCun, Y., Muller, U.A., Sackinger, E., Simard, P.: Comparison of classifier methods: A case study in handwritten digit recognition. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition. pp. 77-82 (1994)
Chao, Z.: Machine learning-based intelligent weather modification forecast in smart city potential area. Computer Science and Information Systems (20), 631-656 (2023)
Chavaltada, C., Pasupa, K., Hardoon, D.R.: Combining multiple features for product categorisation by multiple kernel learning. International Conference on Computing and Information Technology pp. 3-12 (2018)
Cucker, F., Smale, S.: Best choices for regularization parameters in learning theory: On the bias-variance problem. Foundations of Computational Mathematics 2(4), 413-428 (2002)
Dong, Z., Gong, J., Zou, B., Wang, Y., Xu, J.: Generalization and learning rate of multi-class support vector classification and regression. International Journal of Wavelets, Multiresolution and Information Processing (20), 2250017 (2022)
Dong, Z., Qin, Y., Zou, B., Xu, J., Tang, Y.Y.: Lmsvcr: Novel effective method of semisupervised multi-classification. Neural Computing and Application (34), 3857-3873 (2022)
Dong, Z., Xu, C., Xu, J., Zou, B., Zeng, J., Tang, Y.Y.: Generalization capacity of multi-class svm based on markovian resampling. Pattern Recognition (142), 109720 (2023)
Duan, Y., Zou, B., Xu, J., Chen, F.,Wei, J., Tang, Y.Y.: Oaa-svm-ms: A fast and efficient multiclass classification algorithm. Neurocomputing (454), 448-460 (2021)
Duğan, U., Glasmachers, T., Igel, C.: A unified view on multi-class support vector classification. Journal of Machine Learning Research 17(45), 1-32 (2016)
Feng, Y., Yang, Y., Zhao, Y., Lv, S., Suykens, J.A.: Learning with Kernelized Elastic Net Regularization. KU Leuven, Leuven, Belgium (2014)
Gönen, M., Alpaydin, E.: Multiple kernel learning algorithms. Journal of Machine Learning Research (12), 2211-2268 (2011)
Huang, C.L., Dun, J.F.: A distributed pso¨csvm hybrid system with feature selection and parameter optimization. Applied Soft Computing 8(4), 1381-1391 (2008)
Krebel, U.H.G.: Pairwise classification and support vector machines. Advances in kernel methods: support vector learning pp. 255-268 (1999)
Lanckriet, G.R.G., Cristianini, N., Bartlett, P.L., Ghaoui, L.E., Jordan, M.I.: Learning the kernel matrix with semi-definite programming. Journal of Machine Learning Research (5), 27-72 (2004)
Lauriola, I., Gallicchio, C., Aiolli, F.: Enhancing deep neural networks via multiple kernel learning. Pattern Recognition (101), 107194 (2020)
Lee, Y., Lin, Y., Wahba, G.: Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data. Journal of the American Statistical Association 99(465), 67-81 (2004)
Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J.: Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications 35(4), 1817-1824 (2008)
Luo, J., Orabona, F., Fornoni, M., Caputo, B., Cesa-Bianchi, N.: Om-2: An online multi-class multi-kernel learning algorithm. IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops pp. 43-50 (2010)
Lv, S.G., Zhou, F.Y.: Optimal learning rates of lp-type multiple kernel learning under general conditions. Information Sciences (294), 255-268 (2015)
Lv, S.G., Zhu, J.D.: Error bounds for lp-norm multiple kernel learning with least square loss. Abstract and Applied Analysis pp. 1-18 (2012)
Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability. Springer Science & Business Media (2012)
Qian, M., Nie, F., Zhang, C.: Efficient multi-class unlabeled constrained semi-supervised svm. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management. pp. 1665-1668 (2009)
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Wang, T., Su, H., Li, J.: Dws-mkl: Depth-width-scaling multiple kernel learning for data classification. Neurocomputing (411), 455-467 (2020)
Weston, J.,Watkins, C.: Support vector machines for multi-class pattern recognition. Esann pp. 219-224 (1999)
Wilson, C.M., Li, K.Q., Yu, X.Q., Kuan, P.F.,Wang, X.F.: Multiple-kernel learning for genomic data mining and prediction. BMC Bioinformatics 20(1), 1-7 (2019)
Wu, Q., Ying, Y., Zhou, D.X.: Learning rates of least-square regularized regression. Foundations of Computational Mathematics 6(2), 171-192 (2006)
Wu, Q., Zhou, D.X.: Svm soft margin classifiers: Linear programming versus quadratic programming. Neural Computation 17(5), 1160-1187 (2005)
Xu, J., Tang, Y.Y., Zou, B., Xu, Z., Li, L., Lu, Y., Zhang, B.: The generalization ability of svm classification based on markov sampling. IEEE Transactions on Cybernetics 45(6), 1169-1179 (2015)
Yang, Y., Guo, Y., Chang, X.: Angle-based cost-sensitive multicategory classification. Computational Statistics & Data Analysis (156), 107107 (2021)
Yao, B., Liu, S., Wang, L.: Using machine learning approach to construct the people flow tracking system for smart cities. Computer Science and Information Systems (20), 679-700 (2023)
Yi, Z.H., Etemadi, A.H.: Line-to-line fault detection for photovoltaic arrays based on multiresolution signal decomposition and two-stage support vector machine. IEEE Transactions on Industrial Electronics 64(11), 8546-8556 (2017)
Zou, B., Li, L., Xu, Z.: The generalization performance of erm algorithm with strongly mixing observations. Machine Learning 75(3), 275-295 (2009)