An Integrated Approach to Evaluating Text Coherence and its Application to the Prevention of Reader Misconceptions (or The Joy of Detecting Incoherence in Texts)

We examine the problem of incoherence in text and propose an integrated framework for judging it. Incoherence is one source of misconception during natural language processing --- given a text which is not sufficiently coherent, a listener will be unable to interpret the intended meaning; moreover, the listener may produce an interpretation which is not the the one intended by the speaker. We show how the inclusion of a coherence filter in various natural language processing systems may be useful in flagging problematic input so that misconceptions may be deflected. Why Study Incoherence? While it is reasonable to require that only sensible, grammatical, correctly spelled, etc. input be given to NLP systems, this idealization must eventually give way to more robust systems able to deal with any kind of input. If a user is providing the input, this user can make mistakes, become confused, or even be malicious. If a system is automatically generating this input, then certainly errors m..