A Study on the Importance of Linguistic Suffixes in Maithili POS Tagger Development

This paper presents our study on the effect of morphological inflections in the performance of a Maithili Part of Speech (POS) tagger. In the last few years, substantial effort is devoted to developing morphological analyzers and POS taggers in several Indian languages including Hindi, Bengali, Tamil, Telugu, Kannada, Punjabi and Marathi. But we did not find any open POS tagger or morphological analyzers in Maithili. However, Maithili is one of the official languages of India with around 50 million native speakers. So, we worked on developing a POS tagger in Maithili. For the development, we used a manually annotated in-house Maithili corpus containing 52,190 tokens. The tagset contains 27 tags. We first trained conditional random fields (CRF) classifier with various combination of word unigram, bigram, fixed-length suffix, and prefix features. There we observed that the fixed-length suffixes do not show the expected accuracy improvement. However, during the manual corpus annotation, we observed that suffixes played as a helpful clue. So, instead of using the fixed-length suffixes, we worked on identifying the morphological inflections in Mathili. When we used these morphological suffixes in the system, we found a noticeable performance improvement.