Feature-based Thai Word Segmentation

Word segmentation is a problem in several Asian languages that have no explicit word boundary delimiter, e.g. Chinese, Japanese, Korean and Thai. We propose to use featurebased approaches for Thai word segmentation. A feature can be anything that tests for specific information in the context around the word in question, such as context words and collocations. To automatically extract such features from a training corpus, we employ two learning algorithms, namely RIPPER and Winnow. Experimental results show that both algorithms appear to outperform the existing Thai word segmentation methods, especially for context-dependent strings. 1 Introduction Word segmentation is a crucial problem in natural language processing for several Asian languages that have no explicit word boundary delimiter, e.g. Chinese, Japanese, Korean and Thai. The problem can be formally defined as finding arg max W i P (W i jC) = arg max W i P (W i )P (CjW i )=P (C) (1) where C = c 1 c 2 : : : c m is an input..