Treebank parsing and knowledge of language: A cognitive perspective

Over the past 15 years, there has been increasing use of linguis-tically annotated sentence collections, such as the Penn Tree-bank (PTB), for constructing statistically based parsers. While these parsers have generally been built for engineering pur-poses, more recently such approaches have been advanced as potentially cognitively relevant, e.g., for addressing the prob-lem of human language acquisition. Here we examine this pos-sibility critically: we assess how well these Treebank parsers actually approach human/child language competence. We find that such systems fail to replicate many, perhaps most, em-pirically attested grammaticality judgments; seem overly sen-sitive, rather than robust, to training data idiosyncrasies; and easily acquire ”unnatural ” syntactic constructions, those never attested in any human language. Overall, we conclude that existing statistically based treebank parsers fail to incorporate much “knowledge of language ” in these three senses