Entity resolution (ER) from multiple data sets

Entity resolution (ER) within or between datasets is a challenging problem. In this report, we propose a novel methodology and an algorithm to match entities from a canonical dataset, i.e. the data is correct, complete and accurate, to a non canonical dataset (social media website) where the data may be missing or invalid. The proposed methodology is that we inspect a user's friends in order to determine the user's attributes. We make the assumption that it is easier for a single user on the social media website to display invalid information, than to have multiple friends that also fake their information. By inspecting the user's friends we can calculate a score and with this score we match them with the entities from the canonical dataset. Because of privacy issues and against the social media website’s terms of service we create synthetic datasets to scale the experiments to measure the efficiency of the proposed algorithm. We base the synthetic datasets on statistic distributions and use a social homophile approach in order to mimic real life data. To measure the effectiveness of the methodology we manually traverse the social media website. We look at the joint attributes from the canonical and non canonical dataset. We calculate the matching score and match them with entities from the canonical datasets. In the experimental evaluations we are able to match 31% of the entities from the canonical dataset with 31% of the entities in the the non canonical dataset. As a conclusion, with the proposed methodology we manage to match entities, infer and detect invalid values for entities in the non canonical dataset. We also manage to scale the experiments by creating synthetic data and use the proposed algorithm with these datasets