PISA for Development: One World, One Measure for Learning? By Angeline M. Barrett

By Angeline M. Barrett, University of Bristol.

Technical work has started on PISA for Development, the three year project, conceived as a pilot, which aims to “enhance” PISA survey instruments to make them more relevant for “developing countries” whilst at the same time producing scores on the same scales as the “main” PISA assessment.  The OECD’s brochure (OECD, 2013) on the three year project links the ambition behind PISA for Development to the post-2015 UN development goal for education, which it anticipates will focus on learning, not just schooling, and secondary education, not just primary. The 2015 PISA cycle, it argues, can provide a baseline against which to measure progress towards the new goal. It is hoped that developing countries implementing PISA will see “spill-over benefits to other parts of the educational sector.” The ambitions behind PISA for Development raise important questions. How feasible and desirable is it to measure learning across the world along one set of scales? What implications does a single internationally recognised measure have for school curricula? How will spill-over generate capacity to measure and improve education quality at the national level?

One PISA, one set of scales for measuring learning

Like any technically complex tool, PISA can work in different directions and towards different ends. PISA data can show us who is being disadvantaged by education systems. This demonstrated, for example, by Bloem’s (2013) analysis of PISA 2012 data from low and middle income countries. On the other hand, policy research (e.g. Takayama, 2008) shows how PISA’s arrangement of countries along its “main scales” generates a momentum to policy convergence as policy makers seek solutions to improving education or to legitimize their own priorities for change. Through the credibility of its measures, PISA has also been credited with extending OECD’s influence on the global governance of education (Grek, 2013; Sellar and Lingard, 2013).

PISA creates a pressure for curricular convergence in the following way. Like the Learning Metrics Task Force (LMTF) (referenced in OECD’S brochure), it uses an inductive approach to construct a framework for identifying knowledge and skills based on commonalities across curricula. The framework thus constructed becomes an international model that informs curriculum design in countries seeking to demonstrate their economic competitiveness through climbing the PISA league tables. Hence, PISA comes to define the knowledge and skills that are valued around the world. Assessment experts acknowledge the cultural specificity of PISA’s pool of test items and seek out ways to address this. However, the issue deserves critical scrutiny. One set of scholarship that can inform such scrutiny is that of Boaventura de Sousa Santos and his colleagues contributing to the book Another Knowledge is Possible: Beyond Northern Epistemologies. They highlight the influence of Western Europe’s former imperialism and the current logic of neoliberalism on scientific knowledge. These influences are also inscribed to various degrees in school curricula around the world. An unintended effect of PISA and LMTF may be to reinforce the status quo in national curricula and act as a disincentive for local level curricular innovation.

PISA for Capacity Development?

Realising the full potential of PISA to reveal inequalities within education systems depends on capacity to address secondary data analysis to questions of national concern. ‘Spill-over’ seems a precarious plan for capacity building. Indeed, at the present time much assessment expertise is concentrated in a small number of research institutions in the economic ‘North’ able to bid competitively for PISA contracts. When Hasan, Filmer and Pritchett first proposed a learning goal for education in 2006, they used PISA data to illustrate what form a learning goal might take but stated that “societies should be free to set whatever goals they choose” (p. 40) and suggested goals are set at the country level or by a set of countries. More recently Lant Pritchett and Amanda Beatty (2012) have re-iterated the case for a learning goal ending their paper by encouraging countries to “use their own homegrown series of tests and set country-specific goals” at the same time as participating in international assessments such as TIMMS, PISA and SACMEQ. I believe that this case for a multiplicity of measures of different forms of learning across different scales and that is useful to decision makers across different scales needs to be made more strongly. This demands systematic planned investment in capacity building at the national and regional level with an emphasis of developing regional centres of expertise in the design of education evaluations and data analysis.

However, we need also to maintain a sense of proportion around measures of learning. They do not in and of themselves fix education systems. It is educational structures and processes that generate learning outcomes (see Hugh McLean’s article on this) and measures of learning are only valuable insofar as they inform improvements to processes. Measures of learning are partial indicators of quality that should be interpreted with reference to other information, including those which collect qualitative data on processes such as school evaluations and inspection reports.

PISA is not big enough for the world

The world is big enough for an upgraded, scaled-up version of PISA but PISA is not big enough for the world. PISA should not be allowed to contract the knowledge and learning that is valued across societies worldwide or to shift decision-making about what learning we value in schools to the global level. As a tool for global governance, it has limitations and pitfalls. Not least amongst these is its ownership by an organisation that is a coalition of high income countries. In the end, investing in an assessment that belongs to the OECD will not improve education quality in low income countries. That will come about through investing in education systems in low income countries.

Angeline M. Barrett is lecturer at the Graduate School of Education, University of Bristol. Email: angeline.barrett@bristol.ac.uk

Other blogs in this series:

• PISA for Development and the Post-2015 Agenda, by Michael Davidson, Michael Ward and Alejandro Gomez Palma, OECD.
• PISA in Low and Middle Income Countries, By Simone Bloem, formerly OECD.