The Countdown to Defining What Counts: Measurement and Education Post-2015 By Robert Palmer
By Robert Palmer, NORRAG.
This blog is the 4th in a series of post-2015 reflection blogs (see blog 1, 2,and 3 here); a synthesis review of NORRAG NEWSBite’s post-2015 education blogs over the last couple of years.
Improved data and measurement are obviously key to the success of monitoring the post-2015 agenda. The May 2013 report of the UN Secretary-General’s (UNSG) High Level Panel on Post-2015 called for a data revolution, and in August 2014 the UNSG set up an Independent Expert Advisory Group on the Data Revolution for Sustainable Development; the group published their report in November 2014.
For the education sector, the latest initiative to help identify what to measure post-2015 comes in the form of an ongoing UIS-led online consultation on the proposed post-2015 global education indicators (17 November 2014 to 30 January 2015).
NORRAG NEWSBite blogs have certainly had a fair bit to say on the issues of data and measurement, and as usual we aim to present several sides of the story.
Learning assessments and data collection initiatives for education
New assessments and data collection initiatives using international metrics and bench-marking (e.g. PISA for Development, SABER) certainly claim to emphasize quality improvements.
Anderson Simons, technical lead of the Learning Metrics Task Force (LMTF), outlines the LMTF’s recommendations for how learning should be measured across seven domains: physical well-being; social and emotional; culture and the arts; literacy and communication; learning approaches and cognition; numeracy and mathematics; and, science and technology.
Davidson, Ward and Palma of the OECD discuss the relevance of the PISA for Development initiative to the post-2015 global learning agenda, noting, for example, that it could become a ‘single reference point against which to rigorously gauge progress towards targets for the quality and equity of learning outcomes’.
In a similar vein, Malpel argues that learning assessment are not a threat, but a necessity, noting that assessment is key to improving the quality of education and accelerating learning. He draws on the example of CONFEMEN’s[1] PASEC (Programme for the Analysis of Education Systems) as an example of a large scale approach to learning assessment.
Even those bloggers who are less enthusiastic about learning assessments (such as PISA), note that other approaches such as the Brookings-UIS Africa Learning Barometer are doing an excellent job using learning metrics to expose the shallowness of another metric: enrollment rates.
We treasure what we measure?
The drive towards more standardized and quantifiable data, including the measurement of learning outcomes, brings up a number of questions on the policy implications for education and development. Collectively, these voices are concerned about a too reductionist approach to the post-2015 education framework, and the danger of falling back only on what can be neatly counted.
A NORRAG meeting on “The Brave New World of Data for Education and Development” in September 2013 as well as other blogs (e.g. Draxler, Carton) raised several questions about the drive towards more standardized and quantifiable data:
- What data will be given priority, how will it be collected and used, by whom, and at what cost?
- How and to what extent can quality education and lifelong learning be measured?
- Are international metrics the priority tools for helping countries improve the quality and equality of education?
- Does widespread standardized testing reliably inform about learning acquisition, provide appropriate accountability, and improve teacher and learner performance?
- Who stands to gain the most from the use of worldwide standardized performance indicators?
Several of the most known proposals for learning assessment and approaches to bench-marking were critiqued in a few blogs:
Draxler takes a critical look at the “Global Framework of Learning Domains” that the LMTF is working towards. McLean argues that the LMTF, ‘driven by special donor-interests, is focused on improving measurement, not learning’.
Barrett questions the utility of PISA for Development, noting that the ambitions behind PISA for Development raise important questions; for example, how feasible and desirable it is to measure learning across the world along one set of scales. She argues that ‘in the end, investing in an assessment that belongs to the OECD will not improve education quality in low income countries’.
Menefee echoes the disdain felt by some towards PISA, and notes that easily accessible non-proprietary learning metric methodologies would be a valuable asset to the international education community. He also comments that metrics themselves are not the problem; it is that they make terrible managers.
A NORRAG meeting notes that a ‘bottom-up approach to data, education and development is completely lacking at the moment, and a small group of institutions (World Bank, Brookings, LMTF) appear to be proposing what to learn and what to measure’.
Verger argues that we need a comprehensive measure of educational development post-2015, and proposes a methodological and conceptual framework that could help to envision a comprehensive measure of educational progress.
The countdown to defining what counts
Clearly there will be metrics defined to track post-2015 education targets, including for assessing learning. Draxler argues that, rather than the overall goal wording itself, ‘it is the targets and the ways of measuring those targets that are the most influential drivers of the development goals and of education as part of those goals’.
With about six months to go to the World Education Forum in South Korea, those with some suggestions and concerns about the shape of the education post-2015 agenda better start speaking more loudly as there are already several dominant voices in the room. One place to start might be by engaging directly with one of these dominant voices, the UIS, via their online consultation on the proposed post-2015 global education indicators.
In the last blog in this series, we look at the mechanisms and means of implementation of education targets post-2015.
Robert Palmer is an independent education and skills consultant. He also supports the Editor of NORRAG News and runs NORRAG NEWSBite. Email: rpalmer00@gmail.com Tweets @SkillsImpact
>>View all NORRAG Blogs on Education 2030
NORRAG (Network for International Policies and Cooperation in Education and Training) is an internationally recognised, multi-stakeholder network which has been seeking to inform, challenge and influence international education and training policies and cooperation for almost 30 years. NORRAG has more than 4,200 registered members worldwide and is free to join. Not a member? Join free here.
[1] Conference of Education Ministers of Francophone Countries