Learning Assessments as Public Goods? By Michel Carton and Velibor Jakovleski

By Michel Carton and Velibor Jakovleski, NORRAG

The intentions behind the “data revolution” in terms of policy-making are relatively straightforward. Data are meant to guide policy and help close existing gaps between developed and developing countries, in effect making the development process more efficient. It follows the same reasoning that popular American gym L.A. Fitness uses to entice its clients: “what gets measured gets improved.”

According to the High Level Panel (HLP) appointed by the UN Secretary General to advise on the post-2015 global development agenda:

Better data and statistics will help governments track progress and make sure their decisions are evidence-based; they can also strengthen accountability. This is not just about governments. International agencies, CSOs and the private sector should be involved. A true data revolution would draw on existing and new sources of data to fully integrate statistics into decision making, promote open access to, and use of, data and ensure increased support for statistical systems.

The HLP states that all actors should be involved in the process and that data should be openly accessible to them. Based on those conditions, assessment practitioners and scholars alike consider quantitative international assessments as public goods (e.g. see Wagner, 2012).

Public goods are goods or services that can be used by all actors without restriction (non-excludable), and without diminishing the benefit to other actors when they are used (non-rivalrous). The treatment of assessments as public goods inevitably raises the question of what “public” are we referring to? In other words, who really benefits?

If a state signs up for an international learning assessment regime, it does not necessarily diminish the potential benefits to another state joining and benefiting from the regime. But without their substantive adaptation, assessments have inherent exclusionary mechanisms which, at least for now, prevent them from being considered as public goods in the true sense of the term. The two main reasons are: 1) material – the cost and capability of implementation at the national level; 2) normative – what learning actually entails in a given national context.

In terms of material exclusion, assessments are usually developed and orientated towards use in wealthier countries, which have more readily bought into the principles of new public management and, perhaps more importantly, have the resources and capacity to implement them. Because the lowest income countries have the least capacity to develop, improve, or implement assessment systems, they can be excluded from participating based on material ground alone.

However, the ability to take part in international learning assessments and the ability to effect learning outcomes positively should not be confused. Once a state has joined an international assessment regime, national actors are incentivized to adapt their strategies and behaviours in order to reflect and reinforce global (and usually neo-liberal) standards. But perhaps the most glaring issue in applying such assessments nationally is what they actually measure. A problem with prioritizing only learning outcomes is that they mostly focus on literacy and numeracy, with little emphasis on social learning and the context in which that occurs. As Andreas Schleicher comments:

The world has become indifferent to tradition and past reputations, unforgiving of frailty and ignorant of custom and practice. Success will go to those individuals, institutions and countries which are swift to adapt, slow to complain and open to change.

It points to the second exclusionary mechanism, which is a normative one. National education ministries focus on tracking inputs such as budgets, student attendance and graduation rates, but not learning outputs and outcomes – which would be the only way to assess the efficiency of schooling. The overreliance on such quantitative indicators contributes to a discrepancy between what is actually learned and what is measured, which is referred to as the “schooling-learning gap”. By not taking into account the local, cultural and historical contexts of learning, quantitative indicators employed by some international assessments miss the mark, with data not representing the nuances of learning on the ground.

So even if the poorest countries are supported materially to take part in international learning assessments, this value-based exclusionary barrier still prevents learning assessments from being a public good. This implies that international assessments are more like club goods, which are similar to public good but they have some barrier which prevents some actors from benefiting from the good or service on offer (e.g. a social club, or even the European Union).

Aware of these problems, one of the core results that the Learning Metrics Task Force (LMTF) 2.0 aimed to achieve by the end of 2015 was to universally treat “assessments as a public good”. To do so, the LMTF made resources available to ensure that “even the poorest countries are able to access the technical and financial capability to develop, reinforce, maintain and use assessment data to improve learning.” Their “Learning Champions” initiative also focused on broadening inclusion, involving national stakeholders like students, teachers, civil servants, civil society, and development agencies. The initiative was applied in fifteen countries, which adapted LMTF recommendations to their national contexts. Participants included: Argentina (Buenos Aires), Botswana, Canada (Ontario), Colombia (Bogotá), Ethiopia, Kenya, Kyrgyz Republic, Nepal, Pakistan, Palestine, Rwanda, Senegal, Sudan, Tunisia, and Zambia.

Such efforts are commendable because they help to remove the two exclusionary mechanisms by providing resources and by being sensitive to local context and needs. They also touch upon the important principle of Common But Differentiated Responsibility (CBDR). It implies asymmetrical obligations, meaning all international actors should have a role to play in the development, financing and implementation of education assessments, but with differentiated responsibilities based on special needs or circumstances, such as the history, culture, and degree of development of a certain country.

With LMTF 2.0 now winding down, it is up to debate to what extent their efforts were successful in treating international learning assessments as public goods. The proliferation of different assessment regimes hints at their potential benefits, but ultimately measurements need to be translated to learning outcomes. For that to happen, perhaps we should change the conversation, and re-focus on learning, rather than the assessments of that learning, as being the public good in question.

Michel Carton is the Executive Director of NORRAG. Email: michel.carton@graduateinstitute.ch

Velibor Jakovleski is a Research and Programme Officer for NORRAG. Email: velibor.jakovleski@graduateinstitute.ch 

>>View all NORRAG Blogs on Learning Assessments

NORRAG (Network for International Policies and Cooperation in Education and Training) is an internationally recognised, multi-stakeholder network which has been seeking to inform, challenge and influence international education and training policies and cooperation for almost 30 years. NORRAG has more than 4,500 registered members worldwide and is free to join. Not a member? Join free here.