Incorporating Linguistic Rules in Statistical Chinese Language Model for Pinyin2to2character Conversion Ξ

An N2gram Chinese language model incorporating linguistic rules is presented. By constructing elements lattice, rules information is incorporated in statistical frame. To facilitate the hybrid modeling, novel methods such as MI2 based rule evaluating, weighted rule quantification and element2based n2gram probability approximation are present2 ed. Dynamic Viterbi algorithm is adopted to search the best path in lattice. To strengthen the model, transforma2 tion2based error2driven rules learning is adopted. Applying proposed model to Chinese Pinyin2to2character conver2 sion, high performance has been achieved in accuracy, flexibility and robustness simultaneously. Tests show correct rate achieves 94. 81 % instead of 90. 53 % using bi2gram Markov model alone. Many long2distance dependency and recursion in language can be processed effectively