Abstract
In this paper we introduce a new method for decision tree pruning, based on the minimisation of the expected classification error method by Niblett and Bratko. The original Niblett-Bratko pruning algorithm uses Laplace probability estimates. Here we introduce a new, more general Bayesian approach to estimating probabilities which we call m-probability-estimation. By varying a parameter m in this method, tree pruning can be adjusted to particular properties of the learning domain, such as level of noise. The resulting pruning method improves on the original Niblett-Bratko pruning in the following respects: apriori probabilities can be incorporated into error estimation, several trees pruned to various degrees can be generated, and the degree of pruning is not affected by the number of classes. These improvements are supported by experimental findings. m-probability-estimation also enables the combination of learning data obtained from various sources.
Chapter PDF
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Berger, J.O. (1985), Statistical Decision Theory and Bayesian Analysis, Springer-Verlag, New York.
Bratko, I., Kononenko, I. (1986), Learning diagnostic rules from incomplete and noisy data, AI Methods in Statistics, UNICOM Seminar, London, December 1986. Also in Interactions in AI and Statistics (ed. B. Phelps) London; Gower Technical Press, 1987.
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J. (1984), Classification and Regression Trees, Belmont, California: Wadsworth Int. Group.
Cestnik, B., Kononenko, I., Bratko, I. (1987), ASSISTANT 86: A Knowledge-Elicitation Tool for Sophisticated Users, Progress in Machine Learning, Eds. I. Bratko & N. Lavrac, Sigma Press, Wilmslow.
Cestnik, B. (1990), Estimating Probabilities: A Crucial Task in Machine Learning. In Proceedings of ECAI 90, Stockholm, August 1990.
Clark, P., Niblett, T. (1987), Induction in Noisy Domains, Progress in Machine Learning, Eds. I. Bratko & N. Lavrac, Sigma Press, Wilmslow.
Good, I.J. (1965), The Estimation of Probabilities, M.I.T. Press, Cambridge, Massachusetts.
Mingers, J. (1989), An Empirical Comparison of Pruning Methods for Decision Tree Induction, Machine Learning vol. 4, no. 2, Kluwer Academic Publishers.
Niblett, T., Bratko, I. (1986), Learning decision rules in noisy domains, Expert Systems 86, Cambridge University Press (Proceddings of Expert Systems 86 Conf., Brighton 1986).
Pearl, J. (1988), Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann: San Mateo, CA.
Quinlan, J.R. (1986), Learning from noisy data, Machine Learning vol. 2, Eds. R. Michalski, J. Carbonell and T. Mitchel, Palo Alto, CA: Tioga.
Quinlan, J.R. (1987), Simplifying decision trees, International Journal of Man-Machine Studies, 27, pp. 221–234.
Smyth, P., Goodman, R.M., Higgins, C. (1990), A Hybrid Rule-based/Bayesian Classifier, In Proceedings of ECAI 90, Stockholm, August 1990.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1991 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cestnik, B., Bratko, I. (1991). On estimating probabilities in tree pruning. In: Kodratoff, Y. (eds) Machine Learning — EWSL-91. EWSL 1991. Lecture Notes in Computer Science, vol 482. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0017010
Download citation
DOI: https://doi.org/10.1007/BFb0017010
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-53816-5
Online ISBN: 978-3-540-46308-5
eBook Packages: Springer Book Archive