Multi-Domain Global Correlation Degree Branching Entropy Method for Microblog Text Word Segmentation
Authors: 
Jinlian Du, Xueyun Jin, Peng WangAuthors Info & Claims
Jinlian Du, Xueyun Jin, Peng WangAuthors Info & ClaimsPages 71 - 75
Abstract
Word segmentation is a basic topic in the field of natural language processing, and improving the accuracy of word segmentation is a key problem. With the popularity of microblog, accurate word segmentation for microblog text has become a hot spot. However, microblog texts often contain information about multiple related domains, ambiguous words in multi-domain will lead to the decline of word segmentation accuracy. Based on the model theory of word vector and branching entropy, this paper proposes a multi-domain global correlation degree branching entropy method for microblog text word segmentation. This model is applied to microblog text about house price topic in Beijing. The precision, recall and F-measure of this method are compared with branching entropy model proposed by Zhang[6], and the experimental results show that our method outperforms it.
References
[1]
Wei Yang and Longshu Li. 2014. Research on Film box office forecasting Model based on Weibo data. Electronic World, 21(Nov, 2014), 13--16.
[2]
Wenqing Zhao, Xiaoke Hou and Haihong Sha. 2014. Application of semantic rules to sentiment analysis of microblog hot topics. CAAI Transactions on Intelligent Systems, 9(2014), 121--125
[3]
Dongxia Zhang. 2013. Analysis and Discovery of Network Hotspot Based On Microblog for College Students. Southeast Communication, 6, (2013), 87--89.
[4]
Nianwen Xue. 2003. Chinese Word Segmentation as Character Tagging. Computational Linguistics and Chinese Language Processing, 8, (23, 2003), 29--48
[5]
Sujian Li, Qun Liu and Zhiyong Zhang. 2002. Method of Maximum Entropy Model for Language Processing. Computer Science, 7, (29, 2002), 108--110.
[6]
Libang Zhang, Yi Yuan, Jinfeng Yang. 2014. An Unsupervised Approach to Word Segmentation in Chinese EMRs. Intelligent Computer and Applications. 2, (Jan, 2014), 68--71.
[7]
Fuchun Peng, Fangfang Feng, and Andrew McCallum. 2004. Chinese Segmentation and New Word Detection Using Conditional Random Fields. In Proceedings of International Conference on Computational Linguistics, 2004, 562--568.
[8]
Tao Tang and Qiaoli Zhou. 2011. Term extraction based on the combination of Statistics and rules. Journal of Shenyang Aerospace University, 5, (28, 2011), 71--74.
[9]
Yang Zheng, Jianwen Mo. 2012. Chinese word Segmentation method based on Professional term extraction. Popular Science & Technology, 4, (14, 2012), 20--23.
[10]
Keda He, Zhengtao Zhu and Yu Cheng. 2016. Research on Text Categorization Based on Improved TF-IDF Algorithm. Journal of GUANGDONG university of Techonology, 5, (33, 2016), 49--53.
Index Terms
- Multi-Domain Global Correlation Degree Branching Entropy Method for Microblog Text Word Segmentation
Recommendations
Chinese Word Segmentation Based on Maximum Entropy
RSVT '19: Proceedings of the 2019 International Conference on Robotics Systems and Vehicle TechnologyChinese word segmentation has received extensive attention in recent years. The word segmentation method based on character-based tagging improves the performance of word segmentation greatly. This method transforms the word segmentation problem into a ...
Domain Neural Chinese Word Segmentation with Mutual Information and Entropy
ICIT '19: Proceedings of the 2019 7th International Conference on Information Technology: IoT and Smart CityChinese word segmentation (CWS) is an important basic task for NLP. However, the word segmentation model trained by the generic domain corpus has a significant decline in performance in the word segmentation task oriented to the specific domain. Aiming ...
Two-Word Collocation Extraction Using Monolingual Word Alignment Method
Statistical bilingual word alignment has been well studied in the field of machine translation. This article adapts the bilingual word alignment algorithm into a monolingual scenario to extract collocations from monolingual corpus, based on the fact ...
Comments
Information & Contributors
Information
Published In
May 2020
220 pages
ISBN:9781450375344
DOI:10.1145/3393527
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- Baidu Research: Baidu Research
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 October 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ACM TURC'20
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 27Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in