Averaged Naive Bayes Trees: A New Extension of AODE
Pages 191 - 205
Abstract
Naive Bayes (NB) is a simple Bayesian classifier that assumes the conditional independence and augmented NB (ANB) models are extensions of NB by relaxing the independence assumption. The averaged one-dependence estimators (AODE) is a classifier that averages ODEs, which are ANB models. However, the expressiveness of AODE is still limited by the restricted structure of ODE. In this paper, we propose a model averaging method for NB Trees (NBTs) with flexible structures and present experimental results in terms of classification accuracy. Results of comparative experiments show that our proposed method outperforms AODE on classification accuracy.
References
[1]
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Machine learning 29(2), 131-163 (1997).
[2]
Chickering, D., Heckerman, D., Meek, C.: Large-sample learning of Bayesian networks is NP-hard. The Journal of Machine Learning Research 5, 1287-1330 (2004).
[3]
Hovold, J.: Naive Bayes spam filtering using word-position-based attributes. In: Proceedings of the Second Conference on Email and Anti-Spam, CEAS (2005).
[4]
Quinlan, J.: C4. 5: Programs for machine learning. Morgan Kaufmann, San Francisco (1993).
[5]
Kohavi, R.: Scaling up the accuracy of naive-Bayes classifiers: A decision-tree hybrid. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD), pp. 202-207 (1996).
[6]
Sahami, M.: Learning limited dependence Bayesian classifiers. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD), pp. 335-338 (1996).
[7]
Webb, G., Boughton, J., Wang, Z.: Not so naive Bayes: Aggregating one-dependence estimators. Machine Learning 58(1), 5-24 (2005).
[8]
Jiang, L., Zhang, H.: Weightily averaged one-dependence estimators. In: Yang, Q., Webb, G. (eds.) PRICAI 2006. LNCS (LNAI), vol. 4099, pp. 970-974. Springer, Heidelberg (2006).
[9]
Guyon, I., Elisseeff, A.: An Introduction to Variable and Feature Selection. The Journal of Machine Learning Research 3, 1157-1182 (2003).
[10]
Tang, E., Suganthan, P., Yao, X.: An analysis of diversity measures. Machine Learning 65, 247-271 (2006).
[11]
Cestnik, B., Bratko, I.: On estimating probabilities in tree pruning. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 138-150. Springer, Heidelberg (1991).
[12]
Mitchel, T.: Machine learning. McGraw Hill, New York (1997).
[13]
Breiman, L.: Random forests. Machine learning 45(1), 5-32 (2001).
[14]
Boser, B., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Proceedings of the fifth annual workshop on Computational learning theory (COLT), pp. 144-152 (1992).
[15]
Platt, J.: Sequential minimal optimization: A fast algorithm for training support vector machines. Advances in Kernel Methods-Support Vector Learning 208 (1999).
[16]
Keerthi, S., Shevade, S., Bhattacharyya, C., Murthy, K.: Improvements to Platt's SMO algorithm for SVM classifier design. Neural Computation 13(3), 637-649 (2001).
[17]
Asuncion, A., Newman, D.: UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences (2007).
[18]
Whitten, I., Frank, E.: DataMining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco (2005).
[19]
Nadeau, C., Bengio, Y.: Inference for the generalization error. Machine Learning 52(3), 239- 281 (2003).
[20]
Hsu, C., Huang, H., Wong, T.: Why discretization works for naive Bayesian classifiers. In: Proceedings of the Seventeenth International Conference on Machine Learning (ICML), pp. 399-406 (2000).
- Averaged Naive Bayes Trees: A New Extension of AODE
Recommendations
Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification
Averaged n-Dependence Estimators (AnDE) is an approach to probabilistic classification learning that learns by extrapolation from marginal to full-multivariate probability distributions. It utilizes a single parameter that transforms the approach ...
Not so naive Bayes: aggregating one-dependence estimators
Of numerous proposals to improve the accuracy of naive Bayes by weakening its attribute independence assumption, both LBR and Super-Parent TAN have demonstrated remarkable error performance. However, both techniques obtain this outcome at a considerable ...
Naive Bayes for optimal ranking
It is well known that naive Bayes performs surprisingly well in classification, but its probability estimation is poor. AUC (the area under the receiver operating characteristics curve) is a measure different from classification accuracy and probability ...
Comments
Information & Contributors
Information
Published In
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 03 November 2009
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Feb 2025