A Scientometric Investigation into the Validity of IS Journal Quality Measures

In this study we investigated the measurement validity of the findings in the IS journal quality stream over the past ten years. Our evaluation applied a series of validation tests to the metrics presented in these studies using data from multiple sources. The results of our tests for content, convergent, and discriminant validity, as well as those for parallel-form, test-retest, and item-to-total reliability, were highly supportive. From these findings, we conclude that recent studies in the IS journal quality stream are credible. As such, these IS journal quality measures provide appropriate indicators of relative journal quality. This conclusion is important for both academic administrators and scientometric researchers, the latter of whom depend on journal quality measures in the evaluation of published IS research