LIBER Webinar: Metrics & Machines

This webinar, presented on 6 March, was organised by LIBER's Innovative Metrics Group. The webinar looked at why metrics is currently such a hot topic in academia, and at how new text mining technology could deepen our understanding of the ‘knowledge potential’ of research. A recording of the webinar is available on YouTube. In the first presentation, Dr Charlotte Wien (Professor of Scholarly Communication at the University Library of Southern Denmark) addressed the question 'Why Metrics?'. "Metrics are currently the subject of many heated academic debates. Researchers want fair assessment while decision makers need simple ways of overseeing research and its sea of complexity. So far no ‘one size fits all’ standard performance measure has been established. The methods and theories of various scientific disciplines — plus differences in scopes, aims, publication traditions, and so on — has made it virtually impossible to establish metrics covering all disciplines, providing justice to research and at the same time establishing the overview so much in demand. But why is it in demand? What brought us here?" The second presentation from Gustaf Nelhans (Senior Lecturer at the Swedish School of Library and Information Science) looked at connections between metrics and text mining. "Traditionally, citation analysis and text mining cover two distinct fields. One is used as an indicator of some property of ‘quality’ while the other intends to read texts at a massive scale. In this lecture, we will investigate the possibility of adding the semantic content of texts to the study of citations and argue that this opens for new means of research in the field. Our line of reasoning is that words are used in a specific or more general way, and their meaning changes through use. Correspondingly, we argue that the meaning of a cited reference is defined by its use. Using machine learning and so-called ‘word embeddings’, we create a conceptual space of cited documents using a window of text around the references to extract the “meaning” of the cited document. By visualising the results, we can explore literature in its cited context, meaning that we can characterize research based on how it is used, rather than based on content. The reflexive question here is: in what way can these metrics help unveil the “knowledge potential” of published research by looking at its use?" The webinar was hosted by Sofia Fernandes, Open Research Manager at the University of Exeter