Use of locality Sensitive Hashing (LSH) Algorithm to match Web of Science and SCOPUS

The objective of this study is to find the most appropriate parameters and text components for item-wise matching the two large bibliographic datasets: Clarivate Analytics Web of Science (WoS) and Elsevier’s Scopus. Our focus is on detecting exact matches, that is, no false positives are tolerated at all. To this end, we follow a twofold matching procedure. First, a locality sensitive hashing (LSH) algorithm [15] is applied, which provides fast approximate nearest neighbours and similarities, in order to obtain WoS-Scopus pair suggestions. We experiment with three different combinations of text components (i.e., only publication titles, titles + journal names, co-author names + titles + journals) as input for the matching process. In addition, different values for LSH input parameters (i.e., number of random vectors, number of different random vector sets, number of neighbours, similarity threshold) are tested. Second, for each suggested pair, different heuristics are applied to identify those pair of records that indeed refer to the same publication. For example, the pairs are classi- fied as correct matches if the journal name, volume, issue and begin page do match. We achieved the best results when only titles were matched and 50-50-50-0.80 or 100-30-30-0.80 input parameters are used. We observe that at least 70% of WoS pub- lications are also indexed by Scopus. Last but not the least, when the parameters lead- ing to the best matching results were applied, it took just about an hour to match 1.6 million vs 2.2 million.status: publishe