Rotation forest of random subspace models
Pages 315 - 324
Abstract
During the last decade, a variety of ensembles methods has been developed. All known and widely used methods of this category produce and combine different learners utilizing the same algorithm as the basic classifiers. In the present study, we use two well-known approaches, namely, Rotation Forest and Random Subspace, in order to increase the effectiveness of a single learning algorithm. We have conducted experiments with other well-known ensemble methods, with 25 sub-classifiers, in order to test the proposed model. The experimental study that we have conducted is based on 35 various datasets. According to the Friedman test, the Rotation Forest of Random Subspace C4.5 (RFRS C4.5) and the PART (RFRS PART) algorithms exhibit the best scores in our resulting ranking. Our results have shown that the proposed method exhibits competitive performance and better accuracy in most of the cases.
References
[1]
Wolpert DH, Macready WG. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation. 1997; 1(1): 67-82.
[2]
Adam SP, Alexandropoulos SAN, Pardalos PM, Vrahatis MN. No free lunch theorem: A review. Approximation and Optimization. 2019; pp. 57-82.
[3]
Pardalos PM, Rasskazova V, Vrahatis MN, editors. Black Box Optimization, Machine Learning, and No-Free Lunch Theorems. vol. 170 Springer Optimization and Its Applications-Mathematics. Springer, 2021.
[4]
Dietterich TG. Ensemble methods in machine learning. In: International Workshop on Multiple Classifier Systems. Springer, 2000; pp. 1-15.
[5]
Opitz D, Maclin R. Popular ensemble methods: An empirical study. Journal of Artificial Intelligence Research. 1999; 11: 169-198.
[6]
Aličković E, Subasi A. Breast cancer diagnosis using GA feature selection and Rotation Forest. Neural Computing and Applications. 2017; 28(4): 753-763.
[7]
Zhu HJ, You ZH, Zhu ZX, Shi WL, Chen X, Cheng L. DroidDet: effective and robust detection of android malware using static analysis along with rotation forest model. Neurocomputing. 2018; 272: 638-646.
[8]
Rodriguez JJ, Kuncheva LI, Alonso CJ. Rotation forest: A new classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2006; 28(10): 1619-1630.
[9]
Ho TK. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998; 20(8): 832-844.
[10]
Breiman L. Bagging predictors. Machine Learning. 1996; 24(2): 123-140.
[11]
Freund Y, Schapire RE. Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning. vol. 96. Morgan Kaufmann; 1996. pp. 148-156.
[12]
Melville P, Mooney RJ. Constructing diverse classifier ensembles using artificial training examples. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence. vol. 3. Morgan Kaufmann; 2003. pp. 505-510.
[13]
Zhou ZH. Ensemble methods: foundations and algorithms. CRC press, 2012.
[14]
Rokach L, Maimon O. Decision trees. In: Data Mining and Knowledge Discovery Handbook. Springer; 2005; pp. 165-192.
[15]
Cohen WW, Singer Y. A simple, fast, and effective rule learner. AAAI/IAAI. 1999; 99(335-342): 3.
[16]
Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C. Safe-level-smote: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer; 2009; pp. 475-482.
[17]
Ros F, Guillaume S. Sampling Techniques for Supervised or Unsupervised Tasks. Springer, 2020.
[18]
Yariyan P, Janizadeh S, Van Phong T, Nguyen HD, Costache R, Van Le H, et al. Improvement of best first decision trees using bagging and dagging ensembles for flood probability mapping. Water Resources Management. 2020; 34(9): 3037-3053.
[19]
Schapire RE, Singer Y. Improved boosting algorithms using confidence-rated predictions. Machine Learning. 1999; 37(3): 297-336.
[20]
Webb GI. Multiboosting: A technique for combining boosting and wagging. Machine Learning. 2000; 40(2): 159-196.
[21]
Kuncheva LI, Skurichina M, Duin RP. An experimental study on diversity for bagging and boosting with linear classifiers. Information Fusion. 2002; 3(4): 245-258.
[22]
Breiman L. Random forests. Machine Learning. 2001; 45(1): 5-32.
[23]
Vidal R, Ma Y, Sastry SS. Generalized Principal Component Analysis. Springer; 2016.
[24]
Pham BT, Bui DT, Prakash I, Dholakia M. Rotation forest fuzzy rule-based classifier ensemble for spatial prediction of landslides using GIS. Natural Hazards. 2016; 83(1): 97-127.
[25]
Burrough PA, McDonnell R, McDonnell RA, Lloyd CD. Principles of Geographical Information Systems. Oxford University Press, 2015.
[26]
Blaser R, Fryzlewicz P. Random rotation ensembles. The Journal of Machine Learning Research. 2016; 17(1): 126-151.
[27]
Dua D, Graff C. UCI Machine Learning Repository, University of California, School of Information and Computer Science, Irvine, CA, 2019; 2019.
[28]
Witten I, Frank E, Hall M, Pal C. Data mining fourth edition: Practical machine learning tools and techniques. San Francisco: Morgan Kaufmann Publishers Inc; 2016.
[29]
Kotsiantis SB. Decision trees: a recent overview. Artificial Intelligence Review. 2013; 39(4): 261-283.
[30]
Quinlan JR. Induction of decision trees. Machine Learning. 1986; 1(1): 81-106.
[31]
Roelofs R, Shankar V, Recht B, Fridovich-Keil S, Hardt M, Miller J, et al. A meta-analysis of overfitting in machine learning. Advances in Neural Information Processing Systems. 2019; 32: 9179-9189.
[32]
Alexandropoulos SAN, Kotsiantis SB, Vrahatis MN. Data preprocessing in predictive data mining. The Knowledge Engineering Review. 2019; 34.
[33]
Ruggieri S. Efficient C4.5 [classification algorithm]. IEEE Transactions on Knowledge and Data Engineering. 2002; 14(2): 438-444.
[34]
Hodges J, Lehmann EL. Rank methods for combination of independent experiments in analysis of variance. In: Selected Works of EL Lehmann. Springer; 2012; pp. 403-418.
[35]
Finner H. On a monotonicity problem in step-down multiple test procedures. Journal of the American Statistical Association. 1993; 88(423): 920-923.
[36]
Weiss SM, Indurkhya N. Rule-based machine learning methods for functional prediction. Journal of Artificial Intelligence Research. 1995; 3: 383-403.
[37]
Frank E, Witten IH. Generating accurate rule sets without global optimization. 1998.
[38]
Džeroski S, Ženko B. Is combining classifiers with stacking better than selecting the best one? Machine Learning. 2004; 54(3): 255-273.
[39]
Murty MN, Devi VS. Combination of classifiers. In: Pattern Recognition. Springer; 2011; pp. 188-206.
Index Terms
- Rotation forest of random subspace models
Index terms have been assigned to the content through auto-classification.
Recommendations
Combining bagging, boosting, rotation forest and random subspace methods
Bagging, boosting, rotation forest and random subspace methods are well known re-sampling ensemble methods that generate and combine a diversity of learners using the same learning algorithm for the base-classifiers. Boosting and rotation forest ...
Supervised subspace projections for constructing ensembles of classifiers
We present a method for constructing ensembles of classifiers using supervised projections of random subspaces. The method combines the philosophy of boosting, focusing on difficult instances, with the improved accuracy achieved by supervised projection ...
Cancer classification using Rotation Forest
We address the microarray dataset based cancer classification using a newly proposed multiple classifier system (MCS), referred to as Rotation Forest. To the best of our knowledge, it is the first time that Rotation Forest has been applied to the ...
Comments
Information & Contributors
Information
Published In
© 2022 – IOS Press. All rights reserved.
Publisher
IOS Press
Netherlands
Publication History
Published: 01 January 2022
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in