The Herrero-Villar approach to citation impact

This paper focuses on the evaluation of research institutions in terms of size-independent indicators. There are well-known procedures in this context, such as what we call additive rules, which provide an evaluation of the impact of any research unit in a scientific field based upon a partition of the field citations into ordered categories, along with some external weighting system to weigh those categories. We introduce here a new ranking procedure that is not an additive rule – the HV procedure, after Herrero & Villar (2013) – and compare it those conventional evaluation rules within a common setting. Given a set of ordered categories, the HV procedure measures the performance of the different research units in terms of the relative probability of getting more citations. The HV method also provides a complete, transitive and cardinal evaluation, without recurring to any external weighting scheme. Using a large dataset of publications in 22 scientific fields assigned to 40 countries, we compare the performance of several additive rules – the Relative Citation Rate, four percentile-based ranking procedures, and two average-based high-impact indicators – and the corresponding HV procedures under the same set of ordered categories. Comparisons take into account re-rankings, and differences in the outcome variability, measured by the coefficient of variation, the range, and the ratio between the maximum and minimum index values.The dataset has been acquired thanks to a grant by Santander Universities Global Division of Banco Santander. P. Albarrán acknowledges financial support from the Spanish MEC through grant ECO2014-58434-P. C. Herrero acknowledges financial support from Generalitat Valenciana under PROMETEO 2013/037, and from the Spanish MEC under project ECO2015-65820-P. J. Ruiz-Castillo acknowledges financial support from the Spanish MEC through grant ECO2014-55953-P, as well as grant MDM 2014-0431 to his Departamento de Economía, and grant MadEco-CM (S2015/HUM-3444) from the Comunidad Autónoma de Madrid. A. Villar acknowledges financial support from the Spanish MEC under project ECO2015-65408-R(MINECO/FEDER)