Mining Atomic Chinese Abbreviation Pairs with a Probabilistic Single Character Word Recovery Model

Abstract. An HMM-based Single Character Recovery (SCR) Model is proposed in this paper to extract a large set of “atomic abbreviation pairs”from a text corpus. By an “atomic abbreviation pair,”it refers to an abbreviated word and its root word (i.e., unabbreviated form) in which the abbreviation is a single Chinese character. This task is important since Chinese abbreviations cannot be enumerated exhaustively but the abbreviation process for Chinese compound words seems to be“compositional”; one can often decode an abbreviated word, such as“台大”(Taiwan University), character-by-character back to its root form,“台灣”plus“大 學”. With a large“atomic abbreviation dictionary,”one may be able to handle problems associated with multiple-character abbreviations more easily based on the compositional property of abbreviations