Relevance Scores for Triples from Type-Like Relations

We compute and evaluate relevance scores for knowledge-base tri-ples from type-like relations. Such a score measures the degree to which an entity “belongs ” to a type. For example, Quentin Tarantino has various professions, including Film Director, Screen-writer, and Actor. The first two would get a high score in our set-ting, because those are his main professions. The third would get a low score, because he mostly had cameo appearances in his own movies. Such scores are essential in the ranking for entity queries, e.g. “american actors ” or “quentin tarantino professions”. These scores are different from scores for "correctness " or "accuracy " (all three professions above are correct and accurate). We propose a variety of algorithms to compute these scores. For our evaluation we designed a new benchmark, which includes a ground truth based on about 14K human judgments obtained via crowdsourcing. Inter-judge agreement is slightly over 90%. Ex-isting approaches from the literature give results far from the op-timum. Our best algorithms achieve an agreement of about 80% with the ground truth. 1