2007. Bengali Part of Speech Tagging using Conditional Random Field

This paper reports about the task of Part of Speech (POS) tagging for Bengali using the statistical Conditional Random Fields (CRFs). The POS tagger has been devel-oped using a tagset1 of 26 POS tags, de-fined for the Indian languages. The system makes use of the different contextual in-formation of the words along with the vari-ety of features that are helpful in predicting the various POS classes. The POS tagger has been trained and tested with the 72,341 and 20K wordforms, respectively. It has been experimentally verified that the lexi-con, named entity recognizer and different word suffixes are effective in handling the unknown word problems and improve the accuracy of the POS tagger significantly. Experimental results show the effective-ness of the proposed CRF based POS tag-ger with an accuracy of 90.3%.