A Joint Model for Discovering and Linking Entities

Entity resolution, the task of automatically determining which mentions refer to the same real-world entity, is a cru-cial aspect of knowledge base construction and management. However, performing entity resolution at large scales is chal-lenging because (1) the inference algorithms must cope with unavoidable system scalability issues and (2) the search space grows exponentially in the number of mentions. Current con-ventional wisdom declares that performing coreference at these scales requires decomposing the problem by first solv-ing the simpler task of entity-linking (matching a set of mentions to a known set of KB entities), and then perform-ing entity discovery as a post-processing step (to identify new entities not present in the KB). However, we argue that this traditional approach is harmful to both entity-linking and overall coreference accuracy. Therefore, we embrace the challenge of jointly modeling entity-linking and entity-discovery as a single entity resolution problem. In order to achieve scalability we (1) present a model that reasons over compact hierarchical entity representations, and (2) propose a novel distributed inference architecture that does not suffer from the synchronicity bottleneck which is inherent in map-reduce architectures. We demonstrate that more test-time data actually improves the accuracy of coreference, and show that joint coreference is substantially more accurate than traditional entity-linking, reducing error by 75%. 1