What implementation and translation teach us: the case of semantic similarity measures in wordnets

Wordnet::Similarity is an important instru-ment used for many applications. It has been available for a while as a toolkit for English and it has been frequently tested on English gold standards. In this pa-per, we describe how we constructed a Dutch gold standard that matches the En-glish gold standard as closely as possi-ble. We also re-implemented the Word-Net::Similarity package to be able to deal with any wordnet that is specified in Wordnet-LMF format independent of the language. This opens up the possibility to compare the similarity measures across wordnets and across languages. It also provides a new way of comparing wordnet structures across languages through one of its core aspects: the synonymy and hy-ponymy structure. In this paper, we report on the comparison between Dutch and En-glish wordnets and gold standards. This comparison shows that the gold standards, and therefore the intuitions of English and Dutch native speakers, appear to be highly compatible. We also show that our pack-age generates similar results for English as reported earlier and good results for Dutch. To the contrary of what we ex-pected, some measures even perform bet-ter in Dutch than English.