How much will a RE-based preprocessor help a statistical parser

The present paper describes experiments on combining a statistical parser with a preprocessor built of regular expres-sions. While the syntactic structure of a sentence may be very complex and will never be fully covered by a finite-state machine, there are phenomena such as noun phrases and simple coordinations that do quite well. Even these “chunks” may theoretically be unlimitedly complex but we show that in reality they rarely do. So a shallow finite state parser can be built to preprocess the input text and solve the simple chunks without introducing too many errors. We discuss one implementation of such preprocessor that is very easy to write and covers roughly 20 % of input words with an accuracy of over 90%. Then we describe two ways of combining the preprocessor with a parser and show that the performance of the parser improves both in speed and accuracy. 1