What is the role of NLP in text retrieval

This paper addresses the value of linguistically-motivated indexing (LMI) for document and text retrieval. After reviewing the basic concepts involved and the assumptions on which LMI is based, namely that complex index descriptions and terms are necessary, I consider past and recent research on LMI, and specifically on automated LMI via NLP. Experiments in the first phase of research, to the late eighties, did not demonstrate value in LMI, but were very limited; but the much larger tests of the Nineties, with full text, have not done so either. My conclusion is that LMI is not needed for effective retrieval, but has other important roles within information-selection systems. The rapid growth of full text databases, together with developments in natural language processing (NLP) technology, has prompted those engaged with NLP to suggest that it could be usefully applied to text retrieval, primarily for indexing purposes but perhaps also for more or less related tasks such as document ‘abstracting ’ or extracting; it could be applied at shallow text as well as at deep content levels, and for user display or for database creation. Retrieval itself has various modes, including filtering or routing as well as one-off searching