SPIED: Stanford Pattern-based Information Extraction and Diagnostics

This paper aims to provide an effective interface for progressive refinement of pattern-based information extraction sys-tems. Pattern-based information extrac-tion (IE) systems have an advantage over machine learning based systems that pat-terns are easy to customize to cope with errors and are interpretable by humans. Building a pattern-based system is usually an iterative process of trying different pa-rameters and thresholds to learn patterns and entities with high precision and recall. Since patterns are interpretable to humans, it is possible to identify sources of errors, such as patterns responsible for extract-ing incorrect entities and vice-versa, and correct them. However, it involves time consuming manual inspection of the ex-tracted output. We present a light-weight tool, SPIED, to aid IE system develop-ers in learning entities using patterns with bootstrapping, and visualizing the learned entities and patterns with explanations. SPIED is the first publicly available tool to visualize diagnostic information of multi-ple pattern learning systems to the best of our knowledge.