From Sentence Processing to Multimedia Information Access

The Problem: It has become clear that a robust full-text natural language question-answering system cannot realisti-cally be expected any time soon. Numerous problems such as intersentential reference and paraphrasing, summariza-tion, common sense implication, and many more, will take a long time to solve to everybody’s satisfaction. At the same time, however, it turns out that given a sophisticated grammar, a large lexicon enhanced by advances in Lexical Semantics, and an inference engine, it is possible to build a natural language system with satisfactory sentence-level performance. Motivation: We need a mechanism that will let us bridge the gap between our ability to analyze natural language sentences and our appetite for processing huge amounts of natural language text and multimedia. Previous Work: Our work builds on the START natural language system [1], which has been used by researchers at MIT and other universities and research laboratories for constructing and querying knowledge bases using English.1 The START system analyzes English text and produces a knowledge base which incorporates, in the form of embedded ternary expressions, the information found in the text. One can think of the resulting entry in the knowledge base as a “digested summary ” of the syntactic structure of an English sentence. A user can retrieve the information stored in the knowledge base by querying it in English. The system will then produce an English response. A representation mimicking the hierarchical organization of natural language syntax has one undesirable consequence