A NON-PARAMETRIC MODEL FOR THE DISCOVERY OF INFLECTIONAL PARADIGMS FROM PLAIN TEXT USING GRAPHICAL MODELS OVER STRINGS

The field of statistical natural language processing has been turning toward morpholog-ically rich languages. These languages have vocabularies that are often orders of magnitude larger than that of English, since words may be inflected in various different ways. This leads to problems with data sparseness and calls for models that can deal with this abun-dance of related words—models that can learn, analyze, reduce and generate morphologi-cal inflections. But surprisingly, statistical approaches to morphology are still rare, which stands in contrast to the many recent advances of sophisticated models in parsing, grammar induction, translation and many other areas of natural language processing. This thesis presents a novel, unified statistical approach to inflectional morphology, an approach that can decode and encode the inflectional system of a language. At the center of this approach stands the notion of inflectional paradigms. These paradigms cluster the large vocabulary of a language into structured chunks; inflections of the same word, like break, broke, breaks, breaking,..., all belong in the same paradigm. And moreover, each of these inflections has an exact place within a paradigm, since each paradigm has designate