A Method for Estimating the Precision of Place Name Matching

Abstract. Information in digital libraries and information systems frequently refers to locations or objects in geographic space. Digital gazetteers are commonly employed to identify the referred place names, in information integration and data cleaning procedures. This process fails due to missing information in the gazetteer, multiple matches or false unique matches. We study the cases of success and the cases for failure of mapping process to gazetteers. In this study, we present a statistical method that permits to estimate the completeness of a gazetteer with respect to a specific target area and application, the expected precision and recall of one-to-one mappings of source place names to the gazetteer, the semantic inconsistency that remains in one-to-one mappings and the degree to which precision and recall are improved under knowledge of the identity of higher levels in a hierarchy of places. Our work is the first that provides estimations of completeness of a digital gazetteer and correctness of the matching process. It is useful (i) to provide decision support for gazetteer use and gazetteer development issues, (ii) to determine the error propagation introduced by mismatch into statistical analysis based on the matched data and (iii) to determine the degree to which the automatic mapping process is improved under knowledge of the identity of higher levels in a hierarchy of places. The presented method is based solely on statistical analysis of the results of the matching process itself and as such, it does not depend on any other source of additional information. This method is general and can probably be easily adopted to matching terms against other hierarchies of concepts