Factors influencing the stability and instability of international rankings in higher education

Whether rankings can reflect the real-life situation within universities or create an artificial illusion of universities is a question in our mind when discussing rankings of universities. This question is even more interesting when associated with a discussion of the world class universities. The two most influential rankings of world universities: the Academic Ranking of World Universities (ARWU) by Shanghai Jiao Tong University and the World University Rankings (WUR) by Times Higher Education- Quacquarelli Symonds commenced in 2003 and 2004 respectively. The short period of publication of these rankings means little research has been conducted to answer this question. This dissertation addresses this issue. To grasp the features of the ARWU and the WUR, various statistical methods are employed first, followed by in-depth multiple case studies to verify the values in the two rankings and to test if they reflect the actual situation in universities. To ensure the accuracy of data, data for the multiple case studies are acquired from different sources for triangulation. It is found that all indicators in the ARWU are positively correlated. The multiple regression analysis reveals that the actual weightings in the ARWU correspond with the alleged weightings. Two components are extracted by the principal components analysis. The biggest contributors to the first component are Alumni Prize Winners and Staff Prize Winners. The biggest contributor to the second component is Publication. The two components are positively correlated. Thus, it is reasonable to combine the indicators into one ranking. In contrast, not all indicators in the WUR are positively correlated. The correlation coefficients between International Student Number and International Staff Number with Citation are negative. The multiple regression analysis reveals that the actual weightings in the WUR do not correspond with the alleged weightings. The actual weightings change year by year. Three components are extracted by the principal components analysis. The biggest contributors to the first component are Peer Review and Employer Review. The biggest contributors to the second component are International Student Number and International Staff Number. The only contributor to the third component is Student to Staff Ratio. The three components are not correlated with each other. Actually, within the second components, the relationship between International Student Number and International Staff Number with Citation are negative. All the evidences show that the indicators of the WUR should not be combined into a single ranking. The multiple case studies show that universitiesâ rankings in the ARWU are relatively stable but shift dramatically in the WUR. The major factors help universities improve their rankings in the ARWU are Alumni Prize Winners and Staff Prize Winners. The dramatic changes of rankings in the WUR are not attributed to the actual changes in universities but the frequently changes in ranking methodology and the use of incorrect data