Cited By
View all- Nagar AJoshi NKatyayan PArora P(2024)Development of HMM Based Parts of Speech Tagger for HadotiComputation of Artificial Intelligence and Machine Learning10.1007/978-3-031-71484-9_15(170-178)Online publication date: 25-Sep-2024
Neural Joint Model for Part-of-Speech Tagging and Entity Extraction
Pages 239 - 245
Abstract
Part-of-speech tagging and named entity recognition (NER) are fundamental sequential labeling tasks in natural language processing (NLP), where joint learning of both tasks is an effective one-step solution. Limited efforts have been made by existing research to meet such needs for Sindhi language. As POS tagging and NER are highly correlative sequence tagging tasks, so most often, a word recognized by the NER system may be recognized as a noun by a POS tagger. Thus, in this paper, we propose a neural joint model based on a bidirectional long-short term memory (BiLSTM) network and adversarial transfer learning to incorporate syntactic information from two tasks by using task-shared information. The syntactic structure captures and provides the information of long-range dependencies among words. Moreover, the self-attention is employed to capture intra-sentence dependencies to the joint model explicitly. Empirical results on two benchmark datasets show that our proposed joint model consistently and significantly surpass the existing methods.
References
[1]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, 2016. Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI) 16). 265–283.
[2]
Wazir Ali, Asadullah Kehar, and Hidayatullah Shaikh. 2015. Towards Sindhi Named Entity Recognition: Challenges and opportunities. In 1st National Conference on Trends and Innovations in Information Technology.
[3]
Wazir Ali, Asadullah Kehar, and Hidayatullah Shaikh. 2016. Towards Sindhi named entity recognition: Challenges and opportunities. (2016).
[4]
Wazir Ali, Jay Kumar, Rajesh Kumar, and Zenglin Xu. 2021. SiPOS: A Benchmark Part-of-Speech Tagset for Low-resourced Language: Sindhi. (2021).
[5]
Wazir Ali, Jay Kumar, Junyu Lu, and Zenglin Xu. 2020. Word Embedding based New Corpus for Low-resourced Language: Sindhi. arXiv preprint arXiv:1911.12579(2020).
[6]
Wazir Ali, Jay Kumar, Zenglin Xu, Congjian Luo, Junyu Lu, Junming Shao, Rajesh Kumar, and Yazhou Ren. 2020. A Subword Guided Neural Word Segmentation Model for Sindhi. arxiv:2012.15079 [cs.CL]
[7]
Wazir Ali, Jay Kumar, Zenglin Xu, Junming Shao, and Yazhou Ren. 2021. A Deep Context-aware Bidirectional Neural Model for Sindhi Named Entity Recognition. (2021).
[8]
Wazir Ali, Junyu Lu, and Zenglin Xu. 2020. SiNER: A Large Dataset for Sindhi Named Entity Recognition. In Proceedings of The 12th Language Resources and Evaluation Conference. European Language Resources Association, 2946–2954.
[9]
Daniel M Bikel, Scott Miller, Richard Schwartz, and Ralph Weischedel. 1997. Nymble: a High-Performance Learning Name-finder. In Fifth Conference on Applied Natural Language Processing. 194–201.
[10]
Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain Separation Networks. In Proceedings of the Neural Information Processing Systems.
[11]
Tihana Britvić. 2018. Semi-supervised neural part-of-speech tagging. Ph.D. Dissertation. University of Zagreb. Faculty of Science. Department of Mathematics.
[12]
Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, and Shengping Liu. 2018. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 182–192.
[13]
Xinchi Chen, Zhan Shi, Xipeng Qiu, and Xuan-Jing Huang. 2017. Adversarial Multi-Criteria Learning for Chinese Word Segmentation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1193–1203.
[14]
Michael Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proceedings of the ACL-02 conference on Empirical Methods in Natural Language Processing, Vol. 10. 1–8.
[15]
Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural Language Processing (Almost) from Scratch. The Journal of Machine Learning Research 12 (2011), 2493–2537.
[16]
Rômulo César Costa de Sousa and Hélio Lopes. 2019. Portuguese POS Tagging Using BLSTM Without Handcrafted Features. In Iberoamerican Congress on Pattern Recognition. Springer, 120–130.
[17]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171–4186.
[18]
Mazhar Ali Dootio and Asim Imdad Wagan. 2019. Syntactic parsing and supervised analysis of Sindhi text. Journal of King Saud University-Computer and Information Sciences 31, 1(2019), 105–112.
[19]
Jeffrey L Elman. 1990. Finding structure in time. Cognitive science 14, 2 (1990), 179–211.
[20]
Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. PMLR, 1180–1189.
[21]
Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning Word Vectors for 157 Languages. In Language Resources and Evaluation Conference.
[22]
Tao Gui, Qi Zhang, Haoran Huang, Minlong Peng, and Xuan-Jing Huang. 2017. Part-of-speech tagging for twitter with adversarial neural networks. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2411–2420.
[23]
Onur Güngör, Suzan Uskudarli, and Tunga Güngör. 2018. Improving Named Entity Recognition by Jointly Learning to Disambiguate Morphological Tags. In Proceedings of the 27th International Conference on Computational Linguistics. 2082–2092.
[24]
Maqsood Ahmed Hakro, Intzar Ali Lashari, 2017. Sindhi Named Entity Recognition (SNER). The Government-Annual Research Journal of Political Science. 5, 5(2017).
[25]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
[26]
Chaoyi Huang, Youguang Chen, and Qiancheng Liang. 2019. Attention-based bidirectional long short-term memory networks for Chinese named entity recognition. In In Proceedings of the 4th International Conference on Machine Learning Technologies. 53–57.
[27]
Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv preprint arXiv:1508.01991(2015).
[28]
Hideki Isozaki and Hideto Kazawa. 2002. Efficient support vector classifiers for named entity recognition. In COLING 2002: The 19th International Conference on Computational Linguistics.
[29]
Wazir Ali Jamro. 2017. Sindhi Language Processing: A survey. In International Conference on Innovations in Electrical Engineering and Computational Technologies. 1–8.
[30]
Yaozong Jia and Xiaopan Ma. 2019. Attention in character-Based BiLSTM-CRF for Chinese named entity recognition. In In Proceedings of the 4th International Conference on Mathematics and Artificial Intelligence. 1–4.
[31]
Awais Khan Jumani, Mashooque Ahmed Memon, Fida Hussain Khoso, Anwar Ali Sanjrani, and Safeeullah Soomro. 2018. Named entity recognition system for Sindhi language. In International conference for emerging technologies in computing. Springer, 237–246.
[32]
Katharina Kann, Johannes Bjerva, Isabelle Augenstein, Barbara Plank, and Anders Søgaard. 2018. Character-level supervision for low-resource POS tagging. In Proceedings of the Workshop on Deep Learning Approaches for Low-Resource Natural Language Processing. 1–11.
[33]
Wahab Khan, Ali Daud, Jamal Abdul Nasir, Tehmina Amjad, Sachi Arafat, Naif Aljohani, and Fahd S Alotaibi. 2019. Urdu part of speech tagging using conditional random fields. Language Resources and Evaluation 53, 3 (2019), 331–362.
[34]
Joo-Kyung Kim, Young-Bum Kim, Ruhi Sarikaya, and Eric Fosler-Lussier. 2017. Cross-lingual transfer learning for pos tagging without cross-lingual resources. In Proceedings of the 2017 conference on empirical methods in natural language processing. 2832–2838.
[35]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).
[36]
Jay Kumar, Rajesh Kumar, Amin Ul Haq, and Sidra Shafiq. 2020. A Non-Parametric Multi-Lingual Clustering Model for Temporal Short Text. In 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP). IEEE, 58–61.
[37]
John D Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the Eighteenth International Conference on Machine Learning. 282–289.
[38]
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 260–270.
[39]
Hao Liu, Lirong He, Haoli Bai, Bo Dai, Kun Bai, and Zenglin Xu. 2018. Structured Inference for Recurrent Hidden Semi-markov Model. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. 2447–2453.
[40]
Maofu Liu, Yukun Zhang, Wenjie Li, and Donghong Ji. 2020. Joint Model of Entity Recognition and Relation Extraction with Self-attention Mechanism. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 19, 4 (2020), 1–19.
[41]
Pengfei Liu, Xipeng Qiu, and Xuan-Jing Huang. 2017. Adversarial Multi-task Learning for Text Classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1–10.
[42]
Xuezhe Ma and Eduard Hovy. 2016. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 1064–1074.
[43]
Javed Ahmed Mahar and Ghulam Qadir Memon. 2010. Rule based part of speech tagging of sindhi language. In 2010 International Conference on Signal Acquisition and Processing. IEEE, 101–106.
[44]
Javed Ahmed Mahar and Ghulam Qadir Memon. 2010. Sindhi part of speech tagging system using wordnet. International Journal of Computer Theory and Engineering 2, 4(2010), 538.
[45]
Raveesh Motlani, Harsh Lalwani, Manish Shrivastava, and Dipti Misra Sharma. 2015. Developing part-of-speech tagger for a resource poor language: Sindhi. In Proceedings of 7th Conference on Language and Technology, Poznan, Poland.
[46]
D Nawaz, SA Awan, ZA Bhutto, M Memon, and M Hameed. 2017. Handling Ambiguities in Sindhi Named Entity Recognition (SNER). Sindh University Research Journal-SURJ (Science Series) 49, 3(2017), 513–516.
[47]
Cicero D Santos and Bianca Zadrozny. 2014. Learning character-level representations for part-of-speech tagging. In Proceedings of the 31st international conference on machine learning (ICML-14). 1818–1826.
[48]
Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45, 11 (1997), 2673–2681.
[49]
Irum Naz Sodhar, Akhtar Hussain Jalbani, Muhammad Ibrahim Channa, and Dil Nawaz Hakro. 2019. Parts of Speech Tagging of Romanized Sindhi Text by applying Rule Based Model. International Journal of Computer Science and Network Security 19, 11(2019), 91.
[50]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.
[51]
Farhan Ali Surahio and Javed Ahmed Mahar. 2018. Prediction system for sindhi parts of speech tags by using support vector machine. In 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET). IEEE, 1–6.
[52]
Masaya Suzuki, Kanako Komiya, Minoru Sasaki, and Hiroyuki Shinnou. 2018. Fine-tuning for Named Entity Recognition Using Part-of-Speech Tagging. In Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation.
[53]
Ioannis Tsochantaridis, Thomas Hofmann, Thorsten Joachims, and Yasemin Altun. 2004. Support vector machine learning for interdependent and structured output spaces. In Proceedings of the twenty-first international conference on Machine learning. 104.
[54]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998–6008.
[55]
YaoSheng Yang, Meishan Zhang, Wenliang Chen, Wei Zhang, Haofen Wang, and Min Zhang. 2018. Adversarial learning for Chinese NER from crowd annotations. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
[56]
Meishan Zhang, Nan Yu, and Guohong Fu. 2018. A simple and effective neural model for joint word segmentation and POS tagging. IEEE/ACM Transactions on Audio, Speech, and Language Processing 26, 9(2018), 1528–1538.
[57]
Yuan Zhang, Regina Barzilay, and Tommi Jaakkola. 2017. Aspect-augmented adversarial networks for domain adaptation. Transactions of the Association for Computational Linguistics 5 (2017), 515–528.
[58]
Junsheng Zhou, Weiguang Qu, and Fen Zhang. 2013. Chinese named entity recognition via joint identification and categorization. Chinese journal of electronics 22, 2 (2013), 225–230.
Recommendations
Rule Based Part of Speech Tagging of Sindhi Language
ICSAP '10: Proceedings of the 2010 International Conference on Signal Acquisition and ProcessingPart of Speech (POS) tagging is a process of assigning correct syntactic categories to each word in the text. Tag set and word disambiguation rules are fundamental parts of any POS tagger. No work has hitherto been published of tag set in Sindhi ...
Korean Part-of-speech Tagging Based on Morpheme Generation
Two major problems of Korean part-of-speech (POS) tagging are that the word-spacing unit is not mapped one-to-one to a POS tag and that morphemes should be recovered during POS tagging. Therefore, this article proposes a novel two-step Korean POS tagger ...
Part-of-speech tagging of modern hebrew text
Words in Semitic texts often consist of a concatenation of word segments, each corresponding to a part-of-speech (POS) category. Semitic words may be ambiguous with regard to their segmentation as well as to the POS tags assigned to each segment. When ...
Comments
Information & Contributors
Information
Published In
February 2021
601 pages
ISBN:9781450389310
DOI:10.1145/3457682
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 June 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- National Key R&D Program of China
Conference
ICMLC 2021
ICMLC 2021: 2021 13th International Conference on Machine Learning and Computing
February 26 - March 1, 2021
Shenzhen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 91Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)1
Reflects downloads up to 10 Feb 2025
Other Metrics
Citations
Cited By
View all- Nagar AJoshi NKatyayan PArora P(2024)Development of HMM Based Parts of Speech Tagger for HadotiComputation of Artificial Intelligence and Machine Learning10.1007/978-3-031-71484-9_15(170-178)Online publication date: 25-Sep-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format