Abstract
Random Forest (RF) is composed of decision trees as base classifiers. In general, a decision tree recursively partitions the feature space into two disjoint subspaces using a single feature as axis-parallel splits for each internal node. The oblique decision tree uses a linear combination of features (to form a hyperplane) to partition the feature space in two subspaces. The later approach is an NP-hard problem to compute the best-suited hyperplane. In this work, we propose to use multiple features at a node for splitting the data as in axis parallel method. Each feature independently divides into two subspaces and this process is done by multiple features at one node. Hence, the given space is divided into multiple subspaces simultaneously, and in turn, to construct the M-ary trees. Hence, the forest formed is named as M-ary Random Forest (MaRF). To measure the performance of the task in MaRF, we have extended the notion of tree strength of the regression tree. We empirically prove that the performance of the MaRF improves due to the improvement in the strength of the M-ary trees. We have shown the performance to wide range of datasets ranging from UCI datasets, Hyperspectral dataset, MNIST dataset, Caltech 101 and Caltech 256 datasets. The efficiency of the MaRF approach is found satisfactory as compared to state-of-the-art methods.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Referred to as conventional random forest throughout the text
References
Akash PS, Kadir ME, Ali AA, Tawhid MNA, Shoyaib M (2019) Introducing confidence as a weight in random forest. In: 2019 International conference on robotics, electrical and signal processing techniques (ICREST). IEEE, pp 611–616
Amaratunga D, Cabrera J, Lee YS (2008) Enriched random forests. Bioinformatics 24(18):2010–2014
Asuncion A, Newman D (2007) Uci machine learning repository
Biau G, Devroye L, Lugosi G (2008) Consistency of random forests and other averaging classifiers. J Mach Learn Res 9(Sep):2015–2033
Biau G (2012) Analysis of a random forests model. J Mach Learn Res 13:1063–1095
Bosch A, Zisserman A, Munoz X (2007) Image classification using random forests and ferns. In: 2007 IEEE 11Th international conference on computer vision. IEEE, pp 1–8
Breiman L (1994) Bagging predictors, Technical Report. UC Berkeley
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Denil M, Matheson D, De Freitas N (2014) Narrowing the gap: Random forests in theory and in practice. In: International conference on machine learning, pp 665–673
Fei-Fei L, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: 2004 Conference on computer vision and pattern recognition workshop. IEEE, pp 178–178
Geurts P, Ernst D, Wehenkel L (2006) Extremely randomized trees. Mach Learn 63(1):3–42
Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset
He Z, Liu L, Deng R, Shen Y (2016) Low-rank group inspired dictionary learning for hyperspectral image classification. Signal Process 120:209–221
Hyperspectral dataset. http://lesun.weebly.com/hyperspectral-data-set.html. Online Accessed: 25-September-2019
Hyperspectral benchmark dataset on soil moisture. https://zenodo.org/record/1227837#.XmYNw_fhVuQ.html. Online Accessed: 20-January-2020
Ishwaran H (2015) The effect of splitting on random forests. Mach Learn 99(1):75–118
Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37
Jain V, Phophalia A (2019) M-ary random forest. In: International conference on pattern recognition and machine intelligence. Springer, pp 161–169
Jain V, Sharma J, Singhal K, Phophalia A (2019) Exponentially weighted random forest. In: International conference on pattern recognition and machine intelligence. Springer, pp 170–178
Ji R, Gao Y, Hong R, Liu Q, Tao D, Li X (2014) Spectral-spatial constraint hyperspectral image classification. IEEE Trans Geosci Remote Sens 52(3):1811–1824
Landgrebe DA (2005) Signal theory methods in multispectral remote sensing, vol 29. Wiley, New York
LeCun Y, Bottou L, Bengio Y, Haffner P, et al. (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Louppe G (2014) Understanding random forests: From theory to practice. arXiv:1407.7502
Menze BH, Kelm BM, Splitthoff DN, Koethe U, Hamprecht FA (2011) On oblique random forests. In: Joint european conference on machine learning and knowledge discovery in databases. Springer, pp 453–469
Mishina Y, Murata R, Yamauchi Y, Yamashita T, Fujiyoshi H (2015) Boosted random forest. IEICE Trans Inf Syst 98(9):1630–1636
Murthy SK, Kasif S, Salzberg S (1994) A system for induction of oblique decision trees. J Artif Intell Res 2:1–32
Oshiro TM, Perez PS, Baranauskas JA (2012) How many trees in a random forest?. In: International workshop on machine learning and data mining in pattern recognition. Springer, pp 154–168
Paul A, Mukherjee DP (2014) Enhanced random forest for mitosis detection. In: Proceedings of the 2014 Indian conference on computer vision graphics and image processing. ACM, pp 85
Paul A, Mukherjee DP (2016) Reinforced random forest. In: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing. ACM, pp 1
Paul A, Mukherjee DP (2019) Reinforced quasi-random forest. Pattern Recogn 94:13–24
Paul A, Mukherjee DP, Das P, Gangopadhyay A, Chintha AR, Kundu S (2018) Improved random forest for classification. IEEE Transactions on Image Processing
Wang Y, Xia ST, Tang Q, Wu J, Zhu X (2018) A novel consistent random forest framework: Bernoulli random forests IEEE transactions on neural networks and learning systems
Wang L, Zhao C (2016) Hyperspectral image processing. Springer, Berlin
Wickramarachchi D, Robertson B, Reale M, Price C, Brown J (2016) Hhcart: an oblique decision tree. Comput Stat Data Anal 96:12–23
Winham SJ, Freimuth RR, Biernacka JM (2013) A weighted random forests approach to improve predictive performance. Stat Anal Data Min ASA Data Sci J 6(6):496–505
Yang J, Jiang YG, Hauptmann AG, Ngo CW (2007) Evaluating bag-of-visual-words representations in scene classification. In: Proceedings of the international workshop on Workshop on multimedia information retrieval. ACM, pp 197–206
Zhang L, Suganthan PN (2014) Oblique decision tree ensemble via multisurface proximal support vector machine. IEEE Trans Cybern 45(10):2165–2176
Zhang Y, Cao G, Li X, Wang B (2018) Cascaded random forest for hyperspectral image classification. IEEE J Sel Top Appl Earth Observ Remote Sens 11(4):1082–1094
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jain, V., Phophalia, A. M-ary Random Forest - A new multidimensional partitioning approach to Random Forest. Multimed Tools Appl 80, 35217–35238 (2021). https://doi.org/10.1007/s11042-020-10047-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-10047-9