The Politics of Indicator Development in the Education 2030 Framework for Action By Tore Bernt Sørensen

By Tore Bernt Sørensen, University of Bristol.

As the process of defining the Education 2030 Framework for Action becomes nearer to completion, this post takes a critical look at one prominent submission to the post-2015 education debate: the OECD Universal Basic Skills report.

In the report, the OECD effectively proposes that PISA should be extended to the entire globe to monitor and drive that “all youth acquire basic skills” (page 15). The bold OECD proposal is not so surprising, even though it is hard to take seriously given the substantial critiques of the programme. The first aspiration of organisations is to increase their influence and through that the values giving direction to its initiatives. As PISA is the most successful enterprise[1] in comparative education research ever seen, OECD seeks to build on the momentum with initiatives like the PISA based Test for Schools and PISA for Development.

Indicator development is associated with politics and values. So, the Education 2030 agenda along with its construction of assessment markets is bound to involve indicator politics. Indicators are at once enabling and constraining, providing opportunities and sidelining alternative perspectives. Measurement is power, for better and for worse.

This gives us all the more reason to critically evaluate the input from policy actors in the light of the Incheon Declaration, adopted at the recent World Education Forum. Indicator development for the Education 2030 agenda raises hard questions that have to be answered in order to take the agenda forward.

Let us dwell a bit with the recent OECD report. This submission is disappointing for two reasons: the self-promoting proposal to extend PISA globally,[2] and the surprisingly weak research basis. Together, these two combine to construe a simplistic ‘flat world’ and ‘level playing field’ along the lines of Thomas Friedman. In the context of the Sustainable Development Goals, this makes for unsettling reading indeed. The world is neither flat nor a ‘level playing field’. Rather, existing power structures and inequalities affect countries and the individuals inhabiting them. They thus have very different starting points, opportunities and constraints, now and in their future development.

Conjuring up a flat world

The OECD report was authored by prominent scholars Eric A. Hanushek and Ludger Woessmann and builds on their extensive research. Their meta-analysis of the causal relationship between national knowledge capital and economic growth is intriguing and provocative. But their extraordinary claims are based on weak data.

Their main argument is that a population’s knowledge capital is by far the most important determinant of a country’s economic growth as measured by GDP (p.26). To support this argument, Hanushek and Woessmann draw on aggregate test scores from 12 international mathematics and science tests conducted by IEA and OECD between 1964 and 2003.

Yet, their analysis appears highly speculative. A closer look at their data material reveals the many ways in which the 12 tests vary:

• Who were tested? The target population for the tests varied from 9-15 year olds. Moreover, some tests were grade-based, others age-based.
• Where were the tests conducted? Between 11-45 countries participated in the tests. USA is the only country that participated in all of the included tests.
• What was tested? The tests focused on either curriculum-based knowledge or real-world application of knowledge
• What sort of test scale? Percentage-correct format vs. item-response-theory proficiency scale

On the basis of these weak data, Hanushek and Woessmann present excessive claims concerning the future potential for countries in raising their national knowledge capital. This is where the global expansion of the PISA assessment framework enters the stage. The OECD report (page 15) suggests that “a central post-2015 development goal for education should be that all youth achieve at least basic skills as a foundation for work and further learning”. The OECD goes on to suggest that the indicator for basic skills should be set at 420 points on the PISA scale, equivalent to “Level 1 skills” and “functional literacy”. In supporting their proposal, the OECD report then claims that if for instance Ghana increased its PISA score by 170 points – from 250 to 420 points – the country’s GDP would increase 38 times.

Furthermore, the methodological nationalism implied in the analysis undermines the OECD argument. Roger Dale wrote on this blog about the fallacies of this particular worldview which is represented in a contradiction running through the entire OECD report. On the one hand, the report stresses global interdependence and that economic markets are liked internationally. Yet, on the other, the analysis treats each country and their potential economic growth in isolation. So, OECDs otherwise promising ‘one-world’ proposal of universal basic skills is not accompanied by any meaningful analysis of what sort of opportunities and constraints apply for low-income and high-income countries in our current global era.

Finally, looking into the foundation of these claims, we see that a number of political assumptions are implied. So, the otherwise interesting discussion of institutional measures reinforces the picture that the OECDs overarching objective, besides furthering its own influence in global educational governance, is to promote a particular political project.

The statistical models in the OECD report thus imply that economies are open to international trade (p.73). Accordingly, the report suggests that developing countries should correct their “imperfect economic institutions” along these lines (p.70). To say that this argument is controversial within the SDGs context would be an understatement.

Furthermore, the US is astonishingly invoked as the model country for production and labour markets since they are “regarded generally as the least distorted” in identifying most clearly how individual skills affect productivity and potential earnings in the labour market (p.76).

In this light, it does not come as a surprise that Hanushek and Woessmann’s book includes straightforward political proposals calling for effective accountability, choice and competition, and direct rewards for good performance.

This is the flat world conjured up by the OECD: Putting itself in charge of a global assessment framework and making a rather US-centric case that there is one single direction in terms of economic and social development. This reader was left to wonder to what extent the OECD proposal of universal basic skills leaves room for adaptation and contextualisation at country level.

The Incheon Declaration calls for “a humanistic vision of education and development based on human rights and dignity; social justice; inclusion; protection; cultural, linguistic and ethnic diversity; and shared responsibility and accountability”.

How is the OECD submission able to support this vision in any way?

Tore Bernt Sørensen is PhD student at the Graduate School of Education, University of Bristol. His PhD project concerns the reasons for and implications of the increased attention directed towards the teaching profession in global educational governance. Contact: t.b.sorensen@bristol.ac.uk

 >>View all NORRAG Blogs on Education Indicators

NORRAG (Network for International Policies and Cooperation in Education and Training) is an internationally recognised, multi-stakeholder network which has been seeking to inform, challenge and influence international education and training policies and cooperation for almost 30 years. NORRAG has more than 4,200 registered members worldwide and is free to join. Not a member? Join free here.

[1] As an enterprise, PISA can only be regarded as a success. However, this does not mean that the programme cannot be the subject of critique.

[2] The OECD report does not go into further details on how PISA for Development relates to the report proposal of extending PISA globally. Rather, OECD presents PISA as one single ‘brand’ of measurement which they argue is relevant for the purpose of the SDGs.