Use of Weighted Finite State Transducers in Part of Speech Tagging

This paper addresses issues in part of speech disambiguation using finite-state transducers and presents two main contributions to the field. One of them is the use of finite-state machines for part of speech tagging. Linguistic and statistical information is represented in terms of weights on transitions in weighted finite-state transducers. Another contribution is the successful combination of techniques -- linguistic and statistical -- for word disambiguation, compounded with the notion of word classes. 1 Introduction Finite-state machines have been extensively used in several areas of natural language processing, including computational phonology, morphology, and syntax. Nevertheless, less has been done in the area of part of speech disambiguation with finite-state transducers (Silberztein1993; Roche and Schabes1995; Chanod and Tapanainen1995). Part of speech tagging consists of assigning to a word its disambiguated part of speech in the sentential context in which this word is ..