Does quality and content matter for citedness? A comparison with para-textual factors and over time

Using (binomial) regression analysis, we run models using citation windows of one to ten years with both annual citation and cumulative citations as dependent variables, and with both bibliometric and quality indicators (judgments of peers) as independent variables. The bibliometric variables are the Journal Impact Factor (JIF) of the publication medium, the numbers of authors and pages, and the statistical citedness of the references used within the paper. We find that the JIF has a larger influence on the citation impact of a publication than the quality (measured by judgments of peers). However, the number of pages and the quality of the references are less influential. The influence of JIF peaks after three years and then declines (in most regression analyses), but remains higher than the influence of quality judgments even after ten years. These results call into question a discrepancy between the algorithmically based indicators and the qualitative judgments by experts. The latter seems less predictive for future citation than a combination of algorithmic constructs. The results of this study can contribute to the empirical specification of the relevance of a normative versus a constructivist theory of citation