Robust Processing of Real-World NaturalLanguage Texts

It is often assumed that when natural language processing meets the real world, the ideal of aiming for complete and correct interpretations has to be abandoned. However, our experience with TACITUS, espe-cially in the MUC-3 evaluation, has shown that principled techniques for syntactic and pragmatic analysis can be bolstered with methods for achieving robustness. We describe and evaluate a method for dealing with unknown words and a method for filtering out sentences irrele-vant to the task. We describe three techniques for making syntactic analysis more robust-an agenda-based scheduling parser, a recovery technique for failed parses, and a new technique called terminal sub-string parsing. For pragmatics processing, we describe how the method of abductive inference is inherently robust, in that an interpretation is always possible, so that in the absence of the required world knowl-edge, performance degrades gracefully. Each of these techniques have been evaluated and the results of the evaluations are presented.