article

Individual attribute prior setting methods for naïve Bayesian classifiers

Authors:

Tzu-Tsung Wong,

Liang-Hao ChangAuthors Info & Claims

Pattern Recognition, Volume 44, Issue 5

Pages 1041 - 1047

https://doi.org/10.1016/j.patcog.2010.11.002

Published: 01 May 2011 Publication History

Abstract

The generalized Dirichlet distribution has been shown to be a more appropriate prior for naive Bayesian classifiers, because it can release both the negative-correlation and the equal-confidence requirements of the Dirichlet distribution. The previous research did not take the impact of individual attributes on classification accuracy into account, and therefore assumed that all attributes follow the same generalized Dirichlet prior. In this study, the selective naive Bayes mechanism is employed to choose and rank attributes, and two methods are then proposed to search for the best prior of each single attribute according to the attribute ranks. The experimental results on 18 data sets show that the best approach is to use selective naive Bayes for filtering and ranking attributes when all of them have Dirichlet priors with Laplace's estimate. After the ranks of the chosen attributes are determined, individual setting is performed to search for the best noninformative generalized Dirichlet prior for each attribute. The selective naive Bayes is also compared with two representative filters for the feature selection, and the experimental results show that it has the best performance.

References

[1]

M. Mizianty, L.A. Kurgan, M. Ogiela, Comparative analysis of the impact of discretization on the classification with naive Bayes and semi-naive Bayes classifiers, in: Proceedings of the Seventh International Conference on Machine Learning and Applications, San Diego, California. (2008) 823-828.

[2]

Yang, Y. and Webb, G.I., Discretization for naïve Bayes learning: managing discretization bias and variance. Machine Learning. v74. 39-74.

[3]

I. Kononenko, Semi-naive Bayesian classifier, in: Proceedings of the Sixth European Working Session on Learning, Porto, Portugal: Springer-Verlag. (1991) 206-219.

[4]

B. Cestnik, I. Bratko, On estimating probabilities in tree pruning, Machine Learning-EWSL-91, in: European Working Session on Learning, Berlin, Germany: Springer-Verlag. (1991) 138-150.

[5]

Mitchell, T.M., . 1997. McGraw-Hill.

[6]

Wong, T.T., Alternative prior assumptions for improving the performance of naive Bayesian classifiers. Data Mining and Knowledge Discovery. v18. 183-213.

[7]

J. Biesiada, W. Duch, A. Kachel, K. Maczka, S. Palucha, Feature ranking methods based on information entropy with parzen window, in: Proceedings of International Conference on Research in Electrotechnology and Applied Informatics, Katowice, Poland. (2005) 109-118.

[8]

Witten, I.H. and Frank, E., . 2005. second ed. Morgan Kaufmann.

[9]

P. Langley, S. Sage, Induction of Selective Bayesian Classifiers, in: Proceedings of the UAI-94 10th International Conference on Uncertainty in Artificial Intelligence, Seattle, WA. (1994) 399-406.

[10]

M. Hall, Correlation-based feature selection for discrete and numeric class machine learning, in: Proceedings of the 17th International Conference on Machine Learning. (2000) 359-366.

[11]

I. Kononenko, Estimating attributes: analysis and extensions of Relief, in: Proceedings of the European Conference on Machine Learning, Catania, Italy, Springer Verlag. (1994) 171-182.

Digital Library

[12]

Wilks, S.S., . 1962. John Wiley, New York.

[13]

Bier, V.M. and Yi, W., A Bayesian method for analyzing dependencies in precursor data. International Journal of Forecasting. v11. 25-41.

[14]

Wong, T.T., Perfect aggregation of Bayesian analysis on compositional data. Statistical Papers. v48. 265-282.

[15]

Connor, R.J. and Mosimann, J.E., Concepts of independence for proportions with a generalization of the Dirichlet distribution. Journal of the American Statistical Association. v64. 194-206.

[16]

Wong, T.T., Generalized Dirichlet distribution in Bayesian analysis. Applied Mathematics and Computation. v97. 165-181.

[17]

Asuncion, A. and Newman, D.J., . 2007. University of California, School of Information and Computer Science, Irvine, CA.

[18]

J. Dougherty, R. Kohavi, M. Sahami, Supervised and unsupervised discretization of continuous features, in: Proceedings of the 12th International Conference on Machine Learning, San Francisco, CA: Morgan Kaufmann. (1995) 194-202.

[19]

R. Kohavi, M. Sahami, Error-based and entropy-based discretization of continuous features, in: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, Portland, OR. (1996) 114-119.

Digital Library

[20]

Hsu, C.N., Huang, H.J. and Wong, T.T., Implications of the Dirichlet assumption for discretization of continuous attributes in naïve Bayesian classifiers. Machine Learning. v53. 235-263.

[21]

Pernkopf, F., Bayesian network classifiers versus selective k-NN classifier. Pattern Recognition. v38. 1-10.

Cited By

Bagui SKalaimannan EBagui SNandi DPinto A(2019)Using machine learning techniques to identify rare cyber‐attacks on the UNSW‐NB15 datasetSecurity and Privacy10.1002/spy2.912:6Online publication date: 7-Nov-2019
Zhao CZhang BHe J(2018)Vision-based Classification of Driving Postures by Efficient Feature Extraction and Bayesian ApproachJournal of Intelligent and Robotic Systems10.1007/s10846-012-9797-z72:3-4(483-495)Online publication date: 30-Dec-2018
https://dl.acm.org/doi/10.1007/s10846-012-9797-z

Individual attribute prior setting methods for naïve Bayesian classifiers
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Recommendations

Alternative prior assumptions for improving the performance of naïve Bayesian classifiers

The prior distribution of an attribute in a naïve Bayesian classifier is typically assumed to be a Dirichlet distribution, and this is called the Dirichlet assumption. The variables in a Dirichlet random vector can never be positively correlated and ...
Generalized Dirichlet priors for Naïve Bayesian classifiers with multinomial models in document classification

The generalized Dirichlet distribution has been shown to be a more appropriate prior than the Dirichlet distribution for naïve Bayesian classifiers. When the dimension of a generalized Dirichlet random vector is large, the computational effort for ...
An efficient parameter estimation method for generalized Dirichlet priors in naïve Bayesian classifiers with multinomial models

Generalized Dirichlet priors have been shown to be an effective way for improving the performance of naïve Bayesian classifiers with multinomial models, called multinomial naïve Bayesian classifiers, in document classification. For the sake of ...

Comments

Information & Contributors

Information

Published In

cover image Pattern Recognition

Pattern Recognition Volume 44, Issue 5

May, 2011

157 pages

ISSN:0031-3203

Issue’s Table of Contents

Copyright © Elsevier Ltd © 2010.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 May 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bagui SKalaimannan EBagui SNandi DPinto A(2019)Using machine learning techniques to identify rare cyber‐attacks on the UNSW‐NB15 datasetSecurity and Privacy10.1002/spy2.912:6Online publication date: 7-Nov-2019
Zhao CZhang BHe J(2018)Vision-based Classification of Driving Postures by Efficient Feature Extraction and Bayesian ApproachJournal of Intelligent and Robotic Systems10.1007/s10846-012-9797-z72:3-4(483-495)Online publication date: 30-Dec-2018
https://dl.acm.org/doi/10.1007/s10846-012-9797-z

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents