Worldwide Inequality and Poverty in Cognitive Results: Cross-sectional Evidence and Time-based Trends

The Sustainable Development Goals (SDGs) for education represent a major departure from the Millennium Development Goals (MDGs) - at least if educational leaders act seriously in their pursuit - in at least two important respects. First, the goals now pertain to learning outcomes. Out of eleven indicators, four focus directly on learning or developmental outcomes, and, in addition, one of the eleven goals (4.7) would require measurement of learning outcomes, even though the relevant indicator currently does not explicitly mention learning outcomes. In presentational terms, the lead indicator (the one presented first in the list) is all about learning and is really composed of three sub-indicators. This is a major departure from the previous set of global goals, the MDGs. These had only one indicator which could arguably be said to pertain to learning outcomes produced by an education system in relative “real time,” namely the literacy rate of 15- to 24-year-olds. However, measurement was often based on self-reports in surveys or censuses, or was based on imputations derived from completed years of education. The headline indicators for the MDGs were really all about access. The formal indicator expressed in the MDG list was completion of primary schooling, but the discussion among global educational leadership focused almost entirely on the number of children out of school and on gender differences in these numbers. The debate started to shift in the early and mid-2000s. The Global Monitoring Report for the Education for All movement, produced in 2012, estimated that some 250 million children were in school but essentially learning nothing. This got a large amount of coverage. The implicit comparison against some 60 million children out of school was sobering. In addition, scholars, non-governmental organizations (NGOs), and development agencies started calling for something akin to “Millennium Learning Goals.” Perhaps more telling than the absolute-amount estimate of 250 million hardly-learning was Filmer, Hasan, and Pritchett’s (2006) early estimate that the median student in poor countries learns at about the same level as children around the 5th percentile in OECD countries. Thinking in terms of percentiles suggests that the median (or below) child in poor countries would be a candidate, or a near candidate, for disability-related support in the OECD—a sobering perspective. The focus on learning is more demanding, but also more meaningful, than a focus on mere access to schooling, because there is far more inequality in learning around the world, than there is in access to schooling. As an example, consider reading skills. A crude but serviceable index of worldwide inequality of this indicator, just within the PISA 2015 reading database, can be crafted by taking the percentage of students performing at or above an acceptable minimum level (Level 2 in PISA 2015 reading), taking the range of this percentage between the three highest and three lowest-performing countries, and dividing this by the median performance. That produces an index value of 0.81. Now, taking two indicators of access, namely the gross enrolment ratios for secondary and tertiary education (the next two access frontiers), a similar calculation yields 0.30 and 0.57 respectively, for the same countries. In a loose sense, and using a database that does not include the worst-off countries in the world, the inequality in learning outcomes is 170 percent greater than the inequality in access to secondary education, and 43 percent greater than the inequality in access to tertiary education. Second, there is a great deal of focus on inequality in the SDGs. In the MDGs, since the goal was in any case 100 percent access, and access is a binary issue (one is either in school or not, one completes primary school or not[2]), a goal of equality is implicit in a goal of 100 percent access. In addition, much of the declarative emphasis was on gender inequality. However, gender inequality is a relatively weak proxy for overall inequality. The SDGs are different. They emphasize inequality as created by quite a few factors (gender, region, income, disability, etc.). (Though they do not cover what one might call “pure” or “total” inequality, that is, the total dispersion in scores, due to factors such as income and region, but also, importantly, due to lack of quality assurance and standards.) This is an important lack in the SDGs, but researchers and policy-makers are likely to pay attention to the issue anyway. Moreover, learning outcomes are a much less binary and more complicated phenomenon than access to schooling, and therefore so is the measurement of its inequality. Taking note of this new dual emphasis of the SDGs, this paper assembles the largest database of learning outcomes inequality data that we know of, and explores key issues related to the measurement of inequality in learning outcomes, with a view to helping countries and international agencies come to grips with the key dimensions and features of this inequality. Two issues in particular are explored. First, whether, as countries improve their average cognitive performance (as measured by international learning assessments) from the lowest to middling levels, they typically reduce cognitive skill inequality or, more importantly perhaps, whether they reduce absolute lack of skills. Second, whether most of cognitive skills inequality is between or within countries. In dealing with these measurement issues, the paper also explores the degree to which measures of cognitive skills are “proper” cardinal variables lending themselves to generalizations from the field of income and wealth distribution—the field for which many measures of inequality and its decomposition were first applied. To do this, we look into whether using the item response theory (IRT) test scores of programmes such as TIMSS influence these types of findings, relative to the use of the underlying and more intuitive classical test scores. Patterns emerging from the classical scores are far less conclusive than those of the IRT scores, in part due to the greater ability of the IRT scores to discriminate between pupils at the bottom end of the performance spectrum. An important contribution of the paper is to examine the sensitivity of standard measures of inequality to different sets of test scores. The sensitivity is high, and the conclusion is that meaningful comparisons between test score inequality and, for instance, income inequality are not possible, at least not using the currently available toolbox of inequality statistics. Finally, the paper explores the practical use of school-level statistics from the test data to inform strategies for reducing inequalities