Abstract
Topic models are widely used to extra the latent knowledge of short texts. However, due to data sparsity, traditional topic models based on word co-occurrence patterns have trouble achieving accurate results on short texts. Researchers have recently proposed knowledge-based topic models for short texts to discover more coherent and meaningful topics. Every form of knowledge aims at representing specific and oriented information but is not wide-ranging. Single knowledge-enhanced topic models only take a form of knowledge into count, which is restricted and undesirable. The more forms of knowledge we incorporate, the more comprehensive our understanding of the short text. In this paper, we propose a novel short texts topic model, named MultiKE-DMM, which combines multiple forms of knowledge and the generalized P\(\acute{o}\)lya urn (GPU) model with Dirichlet Multinomial Mixture (DMM) model. The proposed approach boosts the multi-knowledge background-related words under the same topic. Access to multi-form knowledge permits the creation of an intelligent topic modelling algorithm that considers semantic and fact-oriented relationships between words, offering improved performance over four comparison models on four real-world short text datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3(Jan), 993–1022 (2003)
Cami, B.R., Hassanpour, H., Mashayekhi, H.: User preferences modeling using Dirichlet process mixture model for a content-based recommender system. Knowl.-Based Syst. 163, 644–655 (2019)
Chen, Z., Mukherjee, A., Liu, B., Hsu, M., Castellanos, M., Ghosh, R.: Leveraging multi-domain prior knowledge in topic models. In: Twenty-Third International Joint Conference on Artificial Intelligence (2013)
Dhelim, S., Aung, N., Ning, H.: Mining user interest based on personality-aware hybrid filtering in social networks. Knowl.-Based Syst. 206, 106227 (2020)
Garcia, K., Berton, L.: Topic detection and sentiment analysis in twitter content related to Covid-19 from brazil and the USA. Appl. Soft Comput. 101, 107057 (2021)
Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 50–57 (1999)
Li, C., Wang, H., Zhang, Z., Sun, A., Ma, Z.: Topic modeling for short texts with auxiliary word embeddings. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 165–174 (2016)
Mimno, D., Wallach, H., Talley, E., Leenders, M., McCallum, A.: Optimizing semantic coherence in topic models. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 262–272 (2011)
Nguyen, D.Q., Billingsley, R., Du, L., Johnson, M.: Improving topic models with latent feature word representations. Trans. Assoc. Comput. Linguist. 3, 299–313 (2015)
Phan, X.H., Nguyen, M.L., Horiguchi, S.: Learning to classify short and sparse text & web with hidden topics from large-scale data collections. In: Proceedings of the 17th International Conference on World Wide Web, WWW 2008, Beijing, China, 21–25 April 2008, pp. 91–100. ACM (2008)
Porteous, I., Newman, D., Ihler, A., Asuncion, A., Smyth, P., Welling, M.: Fast collapsed Gibbs sampling for latent Dirichlet allocation. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 569–577 (2008)
Qiang, J., Qian, Z., Li, Y., Yuan, Y., Wu, X.: Short text topic modeling techniques, applications, and performance: a survey. IEEE Trans. Knowl. Data Eng. 34(3), 1427–1445 (2020)
Rajani, N.F.N., McArdle, K., Baldridge, J.: Extracting topics based on authors, recipients and content in microblogs. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 1171–1174 (2014)
Wang, Y., Liu, J., Qu, J., Huang, Y., Chen, J., Feng, X.: Hashtag graph based topic model for tweet mining. In: 2014 IEEE International Conference on Data Mining, pp. 1025–1030. IEEE (2014)
Xu, J., et al.: Short text clustering via convolutional neural networks. In: Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, VS@NAACL-HLT 2015, Denver, Colorado, USA, 5 June 2015, pp. 62–69. The Association for Computational Linguistics (2015)
Xu, K., Qi, G., Huang, J., Wu, T.: Incorporating Wikipedia concepts and categories as prior knowledge into topic models. Intell. Data Anal. 21(2), 443–461 (2017)
Yan, X., Guo, J., Lan, Y., Cheng, X.: A biterm topic model for short texts. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 1445–1456 (2013)
Yao, L., et al.: Incorporating knowledge graph embeddings into topic modeling. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)
Yin, J., Wang, J.: A Dirichlet multinomial mixture model-based approach for short text clustering. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 233–242 (2014)
Zipf, G.K.: Selected studies of the principle of relative frequency in language. In: Selected Studies of the Principle of Relative Frequency in Language. Harvard University Press (2013)
Zubiaga, A., Ji, H.: Harnessing web page directories for large-scale classification of tweets. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 225–226 (2013)
Acknowledgement
This research work has been supported by the National key R & D program project (No. 2019YFC1605504).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
He, J., Chen, J., Li, M.J. (2023). Multi-knowledge Embeddings Enhanced Topic Modeling for Short Texts. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13625. Springer, Cham. https://doi.org/10.1007/978-3-031-30111-7_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-30111-7_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30110-0
Online ISBN: 978-3-031-30111-7
eBook Packages: Computer ScienceComputer Science (R0)