Abstract
Accuracy is a critical factor in predictive modeling. A predictive model such as a decision tree must be accurate to draw conclusions about the system being modeled. This research aims at analyzing and improving the performance of classification and regression trees (CART), a decision tree algorithm, by evaluating and deriving a new methodology based on the performance of real-world data sets that were studied. This paper introduces a new approach to tree induction to improve the efficiency of the CART algorithm by combining the existing functionality of CART with the addition of artificial neural networks (ANNs). Trained ANNs are utilized by the tree induction algorithm by generating new, synthetic data, which have been shown to improve the overall accuracy of the decision tree model when actual training samples are limited. In this paper, traditional decision trees developed by the standard CART methodology are compared with the enhanced decision trees that utilize the ANN’s synthetic data generation, or CART+. This research demonstrates the improved accuracies that can be obtained with CART+, which can ultimately improve the knowledge that can be extracted by researchers about a system being modeled.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Chih-Hung H, Mao-Jiun JW (2005) Using decision tree-based data mining to establish a sizing system for the manufacture of garments. Int J Adv Manuf Technol 26(5):669–674
Witten IH, Frank E (2011) Data mining: practical machine learning tools and techniques 3rd edn. Morgan Kaufmann Series in Data Management Systems, Morgan Kaufmann, Burlington, MA
Kriegel H-P, Borgwardt KM, Kröger P, Pryakhin A, Schubert M, Zimek A (2007) Future trends in data mining. Data Min Knowl Discov 15(1):87–97
Lavrac N (2004) Introduction: lesson learned from data mining applications and collaborative problem solving. Mach Learn 57:13–34
Kusiak A, Smith M (2007) Data mining in design of products and production systems. Ann Rev Control 31:147–156
Brand E, Gerritsen R (1998) Decision trees, DBMS, data mining solutions supplément, http://www.dbmsmag.com/9807m05.html
Apte C, Weiss S (1997) Data mining with decision trees and decision rules. Futur Gener Comput Syst 13:197–210
Li Y (2006) Predicting materials properties and behavior using classification and regression trees. Mater Sci Eng A 433:261–268
Classification and Regression Trees (C&RT) available at http://www.statsoft.com/textbook/stcart.html
Wilkinson L (1992) Tree structured data analysis: AID, CHAID and CART, Department of Statistics, Northwestern University, Evanston, IL 60201
Shih Y-S (2005) QUEST classification tree available at http://www.stat.wisc.edu/~loh/quest.html
Feldman D, Gross S (2005) Mortgage default: classification trees analysis. J Real Estate Fin Econ 30(4):369–396
Questier F, Put R, Coomans D, Walczak B, Vander Heydan Y (2005) The user of CART and multivariate regression trees for supervised and unsupervised feature selection. Chemom Intell Lab Syst 76:45–54
Caetanoa S, Aires-de-Sousab J, Daszykowskia M, Vander Heydena Y (2005) Prediction of enantioselectivity using chirality codes and classification and regression trees. Anal Chim Acta 544:315–326
Andryashin A (2005) Financial applications of classification and regression trees, MS, Thesis, Center of Applied Statistics and Economics, Humboldt University, Berlin
Waheed T, Bonnell RB, Prasher SO, Paulet E (2006) Measuring performance in precision agriculture: CART—a decision tree approach. Agric Water Manage 84:173–185
Provost F, Danyluk A (1999) Problem definition, data cleaning, and evaluation: a classifier learning case study. Informatica 23:123–136
Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. Wadsworth, Inc., Monterey, CA
Hsu C-H, Wang M-JJ (2005) Using decision tree-based data mining to establish a sizing system for the manufacture of garments. Int J Adv Manuf Technol 26:669–674
Evans B, Fisher D (1994) Overcoming process delays with decision tree induction. IEEE Expert 9:60–66
Wu M-C, Lin S-Y, Lin C-H (2006) An effective application of decision tree to stock trading. Expert Syst Appl 31:270–274
Yang B-S, Lin DS, Tan ACC (2005) VIBEX: an expert system for vibration fault diagnosis of rotating machinery using decision tree and decision table. Expert Syst Appl 28:735–742
McSherry D (1999) Strategic induction of decision trees. Knowl Based Syst 12:269–275
Changnon D, Ritsche M, Elyea K, Shelton S, Schramm K (2000) Integrating climate forecasts and natural gas supply information Into a natural gas purchasing decision. Meterol Appl 7:211–216
Moon TH, Sohn SY (2005) Intelligent approach for effective management of governmental funds for small and medium enterprises. Expert Syst Appl 29(566):572
Hsia T-C, Shie A-J, Chen L-C (2008) Course planning of extension education to meet market demand by using data mining techniques—an example of Chinkuo technology university in Taiwan. Expert Syst Appl 34:596–602
Rangwala MH (2006) Empirical investigation of decision tree extraction from neural networks, MS, Thesis, Department of Manufacturing and Systems Engineering, Ohio University, Athens, OH
Michie D (1989) Problems of computer-aided concept formation. In: Quinlan JR (ed) Applications of expert systems volume 2. Addison Wesley, Wokingham, UK, pp 310–333
Gardner MW, Dorling SR (1998) Artificial neural networks (the multilayer Perceptron)—a review of applications in the atmospheric sciences. Atmos Environ 32:2627–2636
Aakerlund L, Hemmingsen R (1998) Neural networks as models of psychopathology. Biol Psychiatr 43(7):471–482
Yang CC, Prasher SO, Landry JA, Ramaswamy HS, Tommasi D (2000) Application of artificial neural networks in image recognition and classification of crop and weeds. Can Agric 13 Eng 42:147–152
Laskaria EC, Meletiou GC, Tasoulis DK, Vrahatis MN (2006) Studying the performance of artificial neural networks on problems related to cryptography. Nonlinear Anal Real World Appl 7:937–942
Madani K, Braz J et al (eds) Industrial and real world applications of artificial neural networks, illusion or reality? Informatics in control. Autom Robot I:11–26
Allex CF, Shavlik JW, Blattner FR (1999) Neural network input representations that produce accurate consensus sequences from DNA fragment assemblies. Bioinformatics 15(9):723–728
An Introduction to Neural Networks, Prof Leslie Smith, Centre for Cognitive and Computational Neuroscience available at http://www.cs.stir.ac.uk/~lss/NNIntro/InvSlides.html
Krawiec K, Slowinski R, Szczesniak I (1998) Pedagogical method for extraction of symbolic knowledge from neural networks. In: Polkowski L, Skowron A (eds) Proceedings of the first international conference on rough sets and current trends in computing (RSCTC '98). Springer, London, UK
Bhagat PM (2005) Pattern recognition in industry. Elsevier. ISBN 0-08-044538-1
Craven MW (1996) Extracting comprehensible models from trained neural networks, PhD Thesis, Computer Science Department, University of Wisconsin, Madison, WI
Dwyer K, Holte R Decision tree instability and active learning. Department of Computing Science, University of Alberta, Edmonton AB, Canada
Neural Networks by Christos Stergiou and Dimitrios Siganos at http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html
Artificial Neural Networks available at www.makhfi.com/tutorial/introduction.htm
Nicholls JG, Martin AR, Wallace BG, Fuchs PA (2001) From neuron to brain, 4th ed. Sinauer Associates, Sunderland, MA. ISBN 0878934391
Artificial Neural Networks at https://www.dacs.dtic.mil/techs/neural/neural2.php
Chen F (2004) Learning accurate and understandable rules from SVM classifiers, MS, Thesis, School of Computer Science, Simon Fraser University
Luin AD (1988) Queries and concept learning. Mach Learn 2:319–342
Murphy PM (1995) UCI repository of machine learning databases—a machinereadable data repository, maintained at the Department of Information and Computer Science, University of California, Irvine., anonymous FTP from ics.uci.edu in the directory pub/machinelearningdatabases
Heinz G, Peterson LJ, Johnson RW, Kerk CJ (2003) Exploring relationships in body dimensions. J Stat Educ 11(2). American Statistical Association Website at http://www.amstat.org/publications/jse
Millie DF, Weckman GR, Pigg RJ, Teste PA, Dyble J, Litaker RW, Carrick HJ, Fahnestiel GL (2006) Modeling phytoplankton abundance in Saginaw bay, Lake Huron: using artificial neural networks to discern functional influence of environmental variables and relevance to a great lakes observing system. J Phycol 42:336–349
Hernandez S, Nesic S, Weckman G, Ghai V (2005) Use of artificial neural networks for predicting crude oil effect on CO2 corrosion of carbon steels. Corrosion. Submitted and accepted for publication, April 2006
Rastogi P (2005) Assessing wireless network dependability using neural networks, MS, Thesis., School of Communication, Ohio University, Athens, OH
Weckman G, Snow A, Rastogi P, Rangwala M (2008) Assessing wireless network dependability through knowledge extraction via decision trees. IARIA: ICONS 2008
CART Pro 6.0 available at http://www.salford-systems.com
Classification and Regression trees available at http://www.statsoft.com/textbook/stathome.html
Maindonald J, Braun J (2003) Data analysis and graphics using R: an example-based approach. Cambridge University Press, Cambridge, UK
NeuroDimensions, NeuroSolutions v5.07, available at http://www.nd.com
Rangwala M (2006) Empirical investigation of decision tree extraction from neural networks, MS, Thesis, Russ College of Engineering and Technology, Department of Industrial and Systems Enginnering, Ohio Univerity, Athens, OH
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Young, W.A., Weckman, G.R., Hari, V. et al. Using artificial neural networks to enhance CART. Neural Comput & Applic 21, 1477–1489 (2012). https://doi.org/10.1007/s00521-012-0887-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-012-0887-4