The Herrero-Villar approach to citation impact

Scoring rules provide an evaluation of the impact of any research unit in a scientific field based upon a partition of the field citations into ordered categories, along with some external weighting system (the scores) to weigh those categories. Many important citation impact indicators widely used in practice can be formulated as scoring rules. This paper introduces a new ranking procedure &-the HV procedure, after Herrero & Villar (2013)&- that is not a scoring rule. Given a set of ordered categories, the HV procedure measures the relative performance of the different research units in terms of a series of tournaments in which each unit is repeatedly confronted with all others. Although the evaluation of each unit is relative to all other units, the HV method provides not only a ranking but also a cardinal evaluation of all units. Moreover, it does not need an external weighting scheme. Using a large dataset of publications in 22 scientific fields assigned to 40 countries, we compare the performance of several scoring rules &-the Relative Citation Rate, four percentile‐based ranking procedures, and two average‐based high‐impact indicators&- and the corresponding HV procedures under the same set of ordered categories. Comparisons take into account re‐rankings, and differences in the discriminatory power, measured by the coefficient of variation, the range, and the ratio between the maximum and minimum index values. Together with their interesting conceptual properties, our results show that HV procedures have good empirical properties.The dataset has been acquired thanks to a grant by Santander Universities Global Division of Banco Santander. P. Albarrán acknowledges financial support from the Spanish MEC through grant ECO2012‐31358. C. Herrero acknowledges financial support from Generalitat Valenciana under PROMETEO 2013/037, and from the Spanish Ministry of Economics and Innovation under project ECO2015‐65820‐P. J. Ruiz‐Castillo acknowledges financial support from the Spanish MEC through grant ECO2014‐55953‐P, as well as grant MDM 2014‐0431 to his Departamento de Economía, and grant MadEco‐CM (S2015/HUM‐3444) from the Comunidad Autónoma de Madrid. A. Villar acknowledges financial support from the Spanish Ministry of Economics and Innovation under project ECO2015‐65408‐R (MINECO/FEDER)