Article

Topic models conditioned on arbitrary features with Dirichlet-multinomial regression

Authors:

Andrew McCallumAuthors Info & Claims

UAI'08: Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence

Pages 411 - 418

Published: 09 July 2008 Publication History

Abstract

Although fully generative models have been successfully used to model the contents of text documents, they are often awkward to apply to combinations of text data and document metadata. In this paper we propose a Dirichlet-multinomial regression (DMR) topic model that includes a log-linear prior on document-topic distributions that is a function of observed features of the document, such as author, publication venue, references, and dates. We show that by selecting appropriate features, DMR topic models can meet or exceed the performance of several previously published topic models designed for specific data.

References

[1]

D. Blei and M. Jordan. Modeling annotated data. In SIGIR, 2003.

[2]

D. Blei and J. D. McAuliffe. Supervised topic models. In NIPS, 2007.

[3]

D. Blei, A. Ng, and M. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022, January 2003.

[4]

L. Dietz, S. Bickel, and T. Scheffer. Unsupervised prediction of citation influences. In ICML, 2007.

[5]

E. Erosheva, S. Fienberg, and J. Lafferty. Mixed membership models of scientific publications. PNAS, 101(Suppl. 1):5220-5227, 2004.

[6]

P. Guimaraes and R. Lindrooth. Dirichlet-multinomial regression. Econometrics 0509001, Econ-WPA, Sept. 2005.

[7]

W. Li and A. McCallum. Pachinko allocation: DAG-structured mixture models of topic correlations. In ICML, 2006.

[8]

D. Liu and J. Nocedal. On the limited memory method for large scale optimization. Mathematical Programming B, 45(3):503-528, 1989.

[9]

A. McCallum, A. Corrada-Emmanuel, and X. Wang. Topic and role discovery in social networks. In IJCAI, 2005.

[10]

A. K. McCallum. MALLET: A machine learning for language toolkit. http://mallet.cs.umass.edu, 2002.

[11]

D. Mimno and A. McCallum. Expertise modeling for matching papers with reviewers. In KDD, 2007.

[12]

D. Newman, C. Chemudugunta, and P. Smyth. Statistical entity-topic models. In KDD, 2006.

[13]

M. Rosen-Zvi, T. Griffiths, M. Steyvers, and P. Smyth. The author-topic model for authors and documents. In UAI, 2004.

[14]

H. M. Wallach. Topic modeling: beyond bag-of-words. In ICML, 2006.

[15]

X. Wang and A. McCallum. Topics over time: a non-Markov continuous-time model of topical trends. In KDD, 2006.

[16]

X. Wang, N. Mohanty, and A. McCallum. Group and topic discovery from relations and their attributes. In NIPS, 2005.

Cited By

Hosseiny Marani ABaumer E(2023)A Review of Stability in Topic Modeling: Metrics for Assessing and Techniques for Improving StabilityACM Computing Surveys10.1145/362326956:5(1-32)Online publication date: 27-Nov-2023
https://dl.acm.org/doi/10.1145/3623269
Chauhan RAcharjya DAvasthi S(2022)Topic Modeling Techniques for Text Mining Over a Large-Scale Scientific and Biomedical Text CorpusInternational Journal of Ambient Computing and Intelligence10.4018/IJACI.29313713:1(1-18)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.4018/IJACI.293137
Wang ZZheng YZhu HYang CChen T(2022)Transferable adversarial examples can efficiently fool topic modelsComputers and Security10.1016/j.cose.2022.102749118:COnline publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1016/j.cose.2022.102749
Show More Cited By

Topic models conditioned on arbitrary features with Dirichlet-multinomial regression
1. Computing methodologies

Recommendations

Generalized Dirichlet priors for Naïve Bayesian classifiers with multinomial models in document classification

The generalized Dirichlet distribution has been shown to be a more appropriate prior than the Dirichlet distribution for naïve Bayesian classifiers. When the dimension of a generalized Dirichlet random vector is large, the computational effort for ...
Topic models conditioned on relations
ECMLPKDD'10: Proceedings of the 2010th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part III

Latent Dirichlet allocation is a fully generative statistical language model that has been proven to be successful in capturing both the content and the topics of a corpus of documents. Recently, it was even shown that relations among documents such as ...
An introduction to the imprecise Dirichlet model for multinomial data

The imprecise Dirichlet model (IDM) was recently proposed by Walley as a model for objective statistical inference from multinomial data with chances @q. In the IDM, prior or posterior uncertainty about @q is described by a set of Dirichlet ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

UAI'08: Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence

July 2008

609 pages

ISBN:0974903949

Editors:
David McAllester,
Petri Myllymaki
Complex Systems Helsinki Institute, Helsinki, Finland

Publisher

AUAI Press

Arlington, Virginia, United States

Publication History

Published: 09 July 2008

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hosseiny Marani ABaumer E(2023)A Review of Stability in Topic Modeling: Metrics for Assessing and Techniques for Improving StabilityACM Computing Surveys10.1145/362326956:5(1-32)Online publication date: 27-Nov-2023
https://dl.acm.org/doi/10.1145/3623269
Chauhan RAcharjya DAvasthi S(2022)Topic Modeling Techniques for Text Mining Over a Large-Scale Scientific and Biomedical Text CorpusInternational Journal of Ambient Computing and Intelligence10.4018/IJACI.29313713:1(1-18)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.4018/IJACI.293137
Wang ZZheng YZhu HYang CChen T(2022)Transferable adversarial examples can efficiently fool topic modelsComputers and Security10.1016/j.cose.2022.102749118:COnline publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1016/j.cose.2022.102749
Steuber FSchoenfeld MRodosek GChbeir RManolopoulos YAkerkar RMizera-Pietraszko J(2020)Topic Modeling of Short Texts Using Anchor WordsProceedings of the 10th International Conference on Web Intelligence, Mining and Semantics10.1145/3405962.3405968(210-219)Online publication date: 30-Jun-2020
https://dl.acm.org/doi/10.1145/3405962.3405968
Burkhardt SKramer S(2019)A Survey of Multi-Label Topic ModelsACM SIGKDD Explorations Newsletter10.1145/3373464.337347421:2(61-79)Online publication date: 26-Nov-2019
https://dl.acm.org/doi/10.1145/3373464.3373474
Wang HLi YZeng SWang GZhang PHui PJin D(2019)Modeling Spatio-Temporal App Usage for a Large User PopulationProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33144143:1(1-23)Online publication date: 29-Mar-2019
https://dl.acm.org/doi/10.1145/3314414
Elahi EWang WRay DFenton AJebara TBogers TSaid ABrusilovsky PTikk D(2019)Variational low rank multinomials for collaborative filtering with side-informationProceedings of the 13th ACM Conference on Recommender Systems10.1145/3298689.3347036(340-347)Online publication date: 10-Sep-2019
https://dl.acm.org/doi/10.1145/3298689.3347036
Kim JKim DOh ACulpepper JMoffat ABennett PLerman K(2019)Homogeneity-Based Transmissive Process to Model True and False News in Social NetworksProceedings of the Twelfth ACM International Conference on Web Search and Data Mining10.1145/3289600.3291009(348-356)Online publication date: 30-Jan-2019
https://dl.acm.org/doi/10.1145/3289600.3291009
Soleimani HMiller D(2019)Exploiting the value of class labels on high-dimensional feature spacesPattern Analysis & Applications10.1007/s10044-017-0629-422:2(299-309)Online publication date: 1-May-2019
https://dl.acm.org/doi/10.1007/s10044-017-0629-4
Fang ZZhang FYin LZhang D(2018)MultiCellProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32649162:3(1-25)Online publication date: 18-Sep-2018
https://dl.acm.org/doi/10.1145/3264916
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten