Classifying a Stream of Infinite Concepts: A Bayesian Non-parametric Approach
Pages 1 - 16
Abstract
Classifying streams of data, for instance financial transactions or emails, is an essential element in applications such as online advertising and spam or fraud detection. The data stream is often large or even unbounded; furthermore, the stream is in many instances non-stationary. Therefore, an adaptive approach is required that can manage concept drift in an online fashion. This paper presents a probabilistic non-parametric generative model for stream classification that can handle concept drift efficiently and adjust its complexity over time. Unlike recent methods, the proposed model handles concept drift by adapting data-concept association without unnecessary i.i.d. assumption among the data of a batch. This allows the model to efficiently classify data using fewer and simpler base classifiers. Moreover, an online algorithm for making inference on the proposed non-conjugate time-dependent non-parametric model is proposed. Extensive experimental results on several stream datasets demonstrate the effectiveness of the proposed model.
References
[1]
Ahmed, A., Low, Y., Aly, M., Josifovski, V., Smola, A.J.: Scalable distributed inference of dynamic user interests for behavioral targeting. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 114–122. ACM (2011)
[2]
Ahmed, A., Ho, Q., Eisenstein, J., Xing, E., Smola, A.J., Teo, C.H.: Unified analysis of streaming news. In: Proceedings of the 20th International Conference on World Wide Web, pp. 267–276. ACM (2011)
[3]
Ahmed, A., Xing, E.P.: Dynamic Non-Parametric Mixture Models and the Recurrent Chinese Restaurant Process: with Applications to Evolutionary Clustering. In: SDM, pp. 219–230 (2008)
[4]
Ahmed, A., Xing, E.P.: Timeline: A dynamic hierarchical Dirichlet process model for recovering birth/death and evolution of topics in text stream. arXiv preprint arXiv:1203.3463 (2012)
[5]
Andrieu C., De Freitas N., Doucet A., and Jordan M.I. An introduction to MCMC for machine learning Machine Learning 2003 50 1-2 5-43
[6]
Antoniak C.E. Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems The Annals of Statistics 1974 2 6 1152-1174
[7]
Bifet A. and Frank E. Pfahringer B., Holmes G., and Hoffmann A. Sentiment knowledge discovery in twitter streaming data Discovery Science 2010 Heidelberg Springer 1-15
[8]
Bifet, A., Pfahringer, B., Read, J., Holmes, G.: Efficient data stream classification via probabilistic adaptive windows. In: Proceedings of the 28th Annual ACM Symposium on Applied Computing, pp. 801–806. ACM (2013)
[9]
Blackwell, D., MacQueen, J.B.: Ferguson distributions via Plya urn schemes. The Annals of Statistics, 353–355 (1973)
[10]
Blei D.M. and Jordan M.I. Variational inference for Dirichlet process mixtures Bayesian Analysis 2006 1 1 121-143
[11]
Blei D.M. and Frazier P.I. Distance dependent Chinese restaurant processes The Journal of Machine Learning Research 2011 12 2461-2488
[12]
Cohen J. A coefficient of agreement for nominal scales Educational and Psychological Measurement 1960 20 1 37-46
[13]
Chu, W., Zinkevich, M., Li, L., Thomas, A., Tseng, B.: Unbiased online active learning in data streams. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 195–203. ACM (2011)
[14]
Davy M. and Tourneret J.Y. Generative supervised classification using dirichlet process priors IEEE Transactions on Pattern Analysis and Machine Intelligence 2010 32 10 1781-1794
[15]
Domingos, P.: Why Does Bagging Work? A Bayesian Account and its Implications. In: KDD, pp. 155–158 (1997)
[16]
Elwell R. and Polikar R. Incremental learning of concept drift in nonstationary environments IEEE Transactions on Neural Networks 2011 22 10 1517-1531
[17]
Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. The Annals of Statistics, 209–230 (1973)
[18]
Gama J., Sebastio R., and Rodrigues P.P. On evaluating stream learning algorithms Machine Learning 2013 90 3 317-346
[19]
Gama, J., Zliobaite, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A Survey on Concept Drift Adaptation. ACM Computing Surveys 46(4) (2014)
[20]
Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence (6), 721–741 (1984)
[21]
Gershman S.J. and Blei D.M. A tutorial on Bayesian nonparametric models Journal of Mathematical Psychology 2012 56 1 1-12
[22]
Gomes, J.B., Menasalvas, E., Sousa, P.A.: Learning recurring concepts from data streams with a context-aware ensemble. In: Proceedings of the 2011 ACM Symposium on Applied Computing, pp. 994–999. ACM (2011)
[23]
Graepel, T., Candela, J.Q., Borchert, T., Herbrich, R.: Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft’s bing search engine. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 13–20 (2010)
[24]
Hannah L.A., Blei D.M., and Powell W.B. Dirichlet process mixtures of generalized linear models The Journal of Machine Learning Research 2011 12 1923-1953
[25]
Harries, M.: Splice-2 comparative evaluation: Electricity pricing. Artificial Intelligence Group, School of Computer Science and Engineering, The University of New South Wales, Sidney, Tech.Rep. UNSW-CSE-TR-9905 (1999)
[26]
Heath D. and Sudderth W. De Finetti’s theorem on exchangeable variables The American Statistician 1976 30 4 188-189
[27]
Hosseini M.J., Ahmadi Z., and Beigy H. Cuzzocrea A. and Dayal U. New management operations on classifiers pool to track recurring concepts Data Warehousing and Knowledge Discovery 2012 Heidelberg Springer 327-339
[28]
Hoffman M.D., Blei D.M., Wang C., and Paisley J. Stochastic variational inference The Journal of Machine Learning Research 2013 14 1 1303-1347
[29]
Katakis I., Tsoumakas G., and Vlahavas I. Tracking recurring contexts using ensemble classifiers: an application to email filtering Knowledge and Information Systems 2010 22 3 371-391
[30]
Klinkenberg R. Learning drifting concepts: Example selection vs. example weighting Intelligent Data Analysis 2004 8 3 281-300
[31]
Minka, T.P.: Bayesian model averaging is not model combination. Technical Report (2000)
[32]
Minka, T.P.: Expectation propagation for approximate Bayesian inference. In: Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 362–369. Morgan Kaufmann Publishers Inc. (2001)
[33]
Neal R.M. Markov chain sampling methods for Dirichlet process mixture models Journal of Computational and Graphical Statistics 2000 9 2 249-265
[34]
Minku L.L. and Yao X. DDD: A new ensemble approach for dealing with concept drift IEEE Transactions on Knowledge and Data Engineering 2012 24 4 619-633
[35]
Paquet, U., Van Gael, J., Stern, D., Kasneci, G., Herbrich, R., Graepel, T.: Vuvuzelas & Active Learning for Online Classification. In: NIPS Workshop on Comp. Social Science and the Wisdom of Crowds (2010)
[36]
Shahbaba B. and Neal R. Nonlinear models using Dirichlet process mixtures The Journal of Machine Learning Research 2009 10 1829-1850
[37]
Zhang, J., Ghahramani, Z., Yang, Y.: A Probabilistic Model for Online Document Clustering with Application to Novelty Detection. In: NIPS, vol. 4, pp. 1617–1624 (2004)
[38]
Zhu X., Zhang P., Lin X., and Shi Y. Active learning from stream data using optimal weight classifier ensemble IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 2010 40 6 1607-1621
Recommendations
Classifying a stream of infinite concepts: a bayesian non-parametric approach
ECMLPKDD'14: Proceedings of the 2014th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part IClassifying streams of data, for instance financial transactions or emails, is an essential element in applications such as online advertising and spam or fraud detection. The data stream is often large or even unbounded; furthermore, the stream is in ...
A taxonomic look at instance-based stream classifiers
Large numbers of data streams are today generated in many fields. A key challenge when learning from such streams is the problem of concept drift. Many methods, including many prototype methods, have been proposed in recent years to address this ...
Study on Application of Bayesian Classifier Model in Data Stream
ICCIS '10: Proceedings of the 2010 International Conference on Computational and Information SciencesTraditional data classification algorithms can not be directly applied for unlimited data and concept drift problem of data stream, so it is accordingly proposed a real-time streaming data classification algorithm for data stream with the concept drift. ...
Comments
Information & Contributors
Information
Published In
748 pages
ISBN:978-3-662-44847-2
DOI:10.1007/978-3-662-44848-9
© Springer-Verlag Berlin Heidelberg 2014.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 10 March 2022
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025