SEMANTIC N-GRAM LANGUAGE MODELING WITH THE LATENT MAXIMUM ENTROPY PRINCIPLE

In this paper, we des ribe a unied probabilisti framework for statisti al language modeling|the latent maximum en-tropy prin iple|whi h an ee tively in orporate various aspe ts of natural language, su h as lo al word intera-tion, synta ti stru ture and semanti do ument informa-tion. Unlike previous work on maximum entropy methods for language modeling, whi h only allow expli it features to be modeled, our framework also allows relationships over hidden features to be aptured, resulting in a more ex-pressive language model. We des ribe eÆ ient algorithms for marginalization, inferen e and normalization in our ex-tended models. We then present experimental results for our approa h on the Wall Street Journal orpus. 1