To what extent is inclusion in the Web of Science an indicator of journal 'quality'?

The assessment of research based on the journal in which it is published is a widely adopted practice. Some research assessments use the Web of Science (WoS) to identify ‘high quality’ journals, which are assumed to publish excellent research. The authority of WoS on journal quality stems from its selection of journals based on editorial standards and scientific impact criteria. These can be considered as universalistic criteria, meaning that they can be applied to any journal regardless of its place of publication, language, or discipline. In this article we examine the coverage by WoS of journals produced in Latin America, Spain, and Portugal. We use a logistic regression to examine the probability of a journal to be covered by WoS given universalistic criteria (editorial standards and scientific impact of the journal) and particularistic criteria (country, language, and discipline of the journal). We find that it is not possible to predict the inclusion of journals in WoS only through the universalistic criteria because particularistic variables such as country of the journal, its discipline, and language are strongly related to inclusion in WoS. We conclude that using WoS as a universalistic tool for research assessment can disadvantage science published in journals with adequate editorial standards and scientific merit. We discuss the implications of these findings within the research evaluation literature, specifically for countries and disciplines not extensively covered by WoS.The authors thank three anonymous reviewers for their insights. D.C. is grateful to Colciencias—Colombia for a scholarship to course a PhD on Science and Technology Policy in SPRU—UK and to EU-SPRI and Ingenio—València for a research visit during which the first version of this article was written