Evaluating altmetrics acts through their creators – how to advance?

<p>One of the grand challenges in the meaningful use and interpretation of altmetrics is the heterogeneity of the acts behind them (Haustein, 2016). On the one hand, the diversity of the online interactions as a measure of scholarly impact is part of what makes altmetrics such a promising complement to traditional impact measures. On the other hand, this diversity makes the interpretation of altmetrics a difficult endeavor, as altmetrics derived from different social media platforms are shaped by significantly different premises. Although different actions on those platforms are in many cases fundamentally different regarding both the respective user’s degree of involvement and intention, their scores are displayed side by side by altmetrics providers without much further explanation of their diverse premises. For example, bookmarking a publication in Mendeley has a substantially different meaning from writing a post about the same publication in Facebook. In order to account for semantic differences between acts from different sources for altmetrics, efforts have been made to classify interactions regarding the required degree of involvement (Haustein, Bowman, & Costas, 2016) or their stakeholders’ main use cases („NISO RP-25-2016, Outputs of the NISO Alternative Assessment Project - National Information Standards Organization“, 2016). One largely unexplored premise that should be considered when interpreting altmetrics are differences regarding the platforms’ userships – the users that are responsible for the interactions underlying altmetrics. Referring to past studies, the share of academics among the users interacting with scientific articles seems to vary considerably between platforms: while for example Jin-Cheon Na & Yingxin Estella Ye (2017) found a distinct predominance of non-academic users in discussions of psychological academic articles on Facebook, Vainio & Holmberg (2017) found researchers to be strongly represented among Twitter users responsible for tweeting scientific articles. And even for those platforms for which we can assume that the relevant share of interactions with scientific publications is committed by researchers such as Mendeley and ResearchGate (Sugimoto, Work, Larivière, & Haustein, 2016), there still might be considerable differences regarding the overall researchers’ professional experience, productivity in terms of traditional publications or represented fields of research between individual platforms. </p> <p>The differentiation between the semantics of acts in altmetrics – and their weighting based on this differentiation – leads to many questions regarding a sensible methodology and also its overall desirableness.</p> <p> </p> <p>