Brookings, UNESCO and the Post-2015 Metrics Steamboat
By Alexandra Draxler, former UNESCO education specialist.
This blog is a reflection on some of the discussion on educational quality post-2015 that are taking place all week at the Comparative and International Education Society (CIES) 2013 Conference in New Orleans.
The predictable trade-off between quality and quantity
The last few years of the Education for All (EFA) movement have seen hand-wringing about the fact that what was a predictable and perverse effect of EFA has come to pass, that is a negative trade-off between quality and quantity in the rush to achieve universal primary education. Quality, and how to measure it, has therefore also recently returned to the forefront of most international discussions about the future of educational development.
The scramble for post-2015: enter the Learning Metrics Task Force at full throttle
What we might call the education and development industry – the institutions, think tanks, consulting firms, NGOs, intergovernmental organizations, bilateral aid agencies and individuals from OECD countries involved in various capacities in education and development in emerging economies – is concentrating hard on negotiating meanders of the various institutional initiatives leading up to a post-2015 agenda. There are excellent reasons for this, as international priorities and financial flows will be to a very large extent governed by this agenda. Private funders will be inspired by the public blueprint.
What does all this have to do with developing countries, or indeed national policies everywhere? The preferred answer, or rather the atmospheric environment, depends which panel one attends at CIES. In one panel, “Envisioning a critical international education agenda” a group of mainly academic “critical friends” of EFA question the need for any post-2015 programme, or indeed the legitimacy of the principal proponents of EFA to lay out targets and standards for the developing countries yet again. On the other hand, the Brookings Institution, financed and supported by an impressive array of institutions, possibly some with a vested interest in the outcome, is showcasing in partnership with UNESCO progress by its Learning Metrics Task Force (LMTF). Its early progress towards a “Global Framework of Learning Domains” is being presented under the title “Towards Universal Learning”. This will eventually detail learning goals and how to measure them “reliably and comparably” across countries. A recent meeting in Dubai of the LMTF (February 20 – 21) recommends a global post-2015 goal that “all children and youth have equitable learning opportunities to become successful global citizens” and attached to this goal six domains of measurement.
Over the past few years, Brookings’ Centre for Universal Education has been advancing rapidly in the education arena, becoming a powerful voice for international collaboration and for continued priority to education internationally. Its “Global Compact on Learning” has planted a flag on the terrain of ideas and research. Its products like the global framework are developed through careful and widespread consultation, and it is responsive to friends and critics alike.
A critical look at the “Global Framework of Learning Domains”
Here follows a selection of a few of the observations and questions raised in the consultations at CIES:
- What is the demand for a global framework, and what is the evidence of demand?
- How will this initiative build on, use or collaborate with existing international measurement initiatives and programmes?
- What evidence is there that the notion of “global citizenship” is or could be a shared vision for post-2015, and is such a vision desirable? Several participants stressed that there is much evidence to the contrary.
- What roles will teaching and teachers have in the implementation and evaluation of such a framework? How and by whom will their roles be defined?
- The LMTF freely admits that the process of developing new universal metrics to assess learning is fraught with controversy and disagreement. Why not, then, make this controversy and disagreement a creative part of its final product, sharing with eventual users the foreseeable drawbacks of different types of metrics, and the probable perverse effects or unintended consequences of various choices?
- How can an international measurement system avoid exacerbating the trend towards using standardized benchmarks as a high-stakes contest for individual learners and governments alike?
- The LMTF acknowledges the impossibility in the early stages of developing measurement tools for all the seven learning domains, and a likely initial focus on measuring literacy (reading) and numeracy (counting and basic maths in real-life situations). Won’t this have the effect of narrowing rather than broadening overall learning goals and curriculum in the countries that use the framework?
- The sixth domain of measurement is “an adaptable, flexible skill set to meet the demands of the 21st century”. Will the elaboration of this domain include vocational skills, or attempt to encompass non-formal learning?
- How will such a framework serve some of the overarching objectives emerging from post-2015: education as an integrated element of societal progress in reducing inequality, promoting human rights and transparent and responsible government, moving towards sustainable development?
Alexandra Draxler was an education specialist at UNESCO. She is now an independent consultant. Email: a.draxler@gmail.com
Pingback : Education Post-2015: Re-Invigorating the Agenda, or Reshuffling the Lobbies? NORRAG Panel at CIES 2013 | NORRAG NEWSBite
The final learning metrics report came out in September 2-2013. There is no mention of teaching, just testing! Countries are somehow supposed to modify instruction from tests, but everyone in the international education business knows that this has never happened. The ‘learning metrics’ is quite obviously a honeypot for companies like RTI that are looking for new testing business. (Coincidentally, Luis Crouch who is an RTI vice president was a founding member of learning metrics in his second stint at the World Bank.)
Diane Ravitch has a blog article on the learning metrics: Global Scandal: If You Can’t Teach Them, Test Them, October 23, 2013. It is hard to locate on her website, so it is copied below.
———-
A reader who works for an international agency sent me this essay about a pressing problem. For obvious reasons, he will remain anonymous, but his sources are cited.
Learning Metrics Taskforce: If you can’t teach the students of poor countries, just test them!
Much has been written about testing problems and corporate interests in the US. Could similar forces be operating outside the US? Here is a story that few readers probably know.
In poor countries education is mainly for the middle class. Most citizens of countries such as Rwanda, Congo, or Papua New Guinea have traditionally remained illiterate. In 1990 a worldwide initiative was launched, called “Education for All”. It was led by the World Bank and has evolved into a multi-billion dollar fund. About 55 low-income governments have received grants to build schools, buy books, and recruit teachers. Parents desperately want to send children to school, so when schools open, they quickly fill up. But there is a glitch: In very poor circumstances, children fail to learn. A World Bank study estimated in 2012 that only 67% of students in Subsaharan Africa finish primary school and of those who finish, 25% are illiterate.
The ‘learning crisis’, as it is called, has multiple causes. My partner and I spent about 12 years teaching for a charity organization, and we witnessed them first hand. Urban classes have 60-120 students with children seated on the floor. Teachers are often absent, may not know how to teach, and they are never supervised. Corrupt officials often demand bribes, and textbooks are stolen before they get to schools. Children are malnourished and hungry. Not much is taught under these conditions.
Donors such as the World Bank ought to have a good handle on this reality. But their staff hardly visit classrooms. They prefer the company of high officials who send their children to private schools and have private agendas. Most world bankers are economists, so they love the virtual reality of datasets and glossy publications. Incredibly, the donors’ response to scant instruction is not better teaching but better testing. Governments are encouraged to develop learning benchmarks, test students against them, and then figure out how to teach children to achieve the benchmarks.
The triumph of testing over teaching was definitively proclaimed through the “Learning Metrics Task Force” deliberations. The prestigious Brookings Institute conducted a large-scale consultation that involved 1700 staff members of 30 organizations. They were asked to define what children should learn in school and how the learning should be measured. Dozens of organizer staff flew to exotic destinations like Dubai and Bellagio, Italy to deliberate on the findings.
The report was formally launched on September 24, 2013 at the 68th session of the United Nations General Assembly. [click here http://globaleducationfirst.org/2996.htm see entire report here http://www.brookings.edu/research/reports/2013/09/learning-metrics-task-force-universal-learning
The report affirms the need to take immediate action to ensure children’s right to quality education. Nevertheless, it says nothing about the practical obstacles to learning such as corruption, harassment, book thefts, and failure to teach. In fact the word “teach” is not mentioned even once. Students are somehow expected to learn through “opportunities to develop competencies across seven domains of learning, starting in early childhood through adolescence.” To achieve this, a small set of key learning indicators will be tracked globally, such as literacy and numeracy. Countries will obtain technical help to diagnose the quality of their assessment systems, convene stakeholders to determine priorities, identify inequities, and make the appropriate policy changes.
To justify this view, the task force introduces the concept that assessment is a Public Good (pp. 12, 32). No country should be denied the opportunity to test students just because they cannot afford to. Parents and other stakeholders should become advocates of testing (p. 15) and for increased funding for testing (p. 17).
To help children quickly there is not a moment to lose. The task force will meet in November 2013 and develop a plan for moving forward. Launch events will be held in at least 15 cities around the world from September through November 2013, to make stakeholders aware of the test benefits.
The “learning metrics” task force seems so out of touch with reality that its main recommendation is a “Global Paradigm Shift” – from mere investment in access to “access plus learning”. Really, in 2013? Over the last 20 years piles of studies have documented learning failures, while numerous UNESCO workshops have taken place on quality improvement. With the same surreal touch, the document omits references to the large-scale testing that has already taken place. Since the 1990s the kids have been fed alphabet-soup tests such as PASEC, SACMEC, TIMSS, EGRA, EGMA, ASER, Uwezo, and other tests (see http://www.eddataglobal.org). And practically no cases are known of governments that put test results to good uses and improved outcomes.
So why did the Brookings Institution compromise its standards for this initiative? Why not form a teaching-for-poverty task force? Cynics point to money, but experience with poor schools leads to some sobering decisions. Donors mainly want to see activities and feel optimistic for the future. The most productive activity is to help schools teach students, but it is time-consuming, sometimes dangerous and often frustrating. Donors may become disappointed and pull out. By contrast, testing is a winner. Field work takes just a few weeks, and analyses can be done from the comfort of air conditioned offices. The staff involved get invited to international conferences, pad their resumes, get promoted. It’s up to the host governments to use test results for policy improvement.
As the task force rushes into implementation, the only certain outcome is consultant welfare. Testing companies are asked to donate time (p. 35), but seven domains in all countries of the world amount to huge numbers of tests. USAID and other donors have spent millions on testing in the past, so consultant companies and associated nonprofits are preparing for a windfall. Our boss is also optimistic.
The smell of money may be one reason why no one has criticized the report publicly. The people who are building careers and retirement funds from money destined to educate poor kids will strongly argue that they are doing the very best they can for them. Anyway many countries are slowly rising out of poverty, and eventually the poor will turn up educated. It may not be exactly ‘Education for All’, but ‘Testing for All” is considered acceptable progress.
Testing, testing, testing. Metrics mania. How much of an industry it is. And in developing countries who reaps the benefit of this — the testing companies have a bonanza of millions and billions to find out “Is Johnny readin’.” RTI is a prime example. Follow the money. Just look at the fiasco that is the World Bank Trust fund calling itself the Global Partnership Education (GPE). Big money shoved out the door, the same big contractor(s) slavering at the prospect of big bucks. What about teacher education, decent salariies, and dare we say it, “quality” learning? The same old teach for the test mentality like we have in the US. Who actually DOES the testing and establishes the so-called ?metrics”? Why, the OECD companies and “think tanks” which are paid by the foundations and companies……Depressing.
Pingback : The Countdown to Defining What Counts: Measurement and Education Post-2015 | NORRAG NEWSBite
The final learning metrics report came out in September 2-2013. There is no mention of teaching, just testing! Countries are somehow supposed to modify instruction from tests, but everyone in the international education business knows that this has never happened. The ‘learning metrics’ is quite obviously a honeypot for companies like RTI that are looking for new testing business. (Coincidentally, Luis Crouch who is an RTI vice president was a founding member of learning metrics in his second stint at the World Bank.)
Diane Ravitch has a blog article on the learning metrics: Global Scandal: If You Can’t Teach Them, Test Them, October 23, 2013. It is hard to locate on her website, so it is copied below.
———-
A reader who works for an international agency sent me this essay about a pressing problem. For obvious reasons, he will remain anonymous, but his sources are cited.
Learning Metrics Taskforce: If you can’t teach the students of poor countries, just test them!
Much has been written about testing problems and corporate interests in the US. Could similar forces be operating outside the US? Here is a story that few readers probably know.
In poor countries education is mainly for the middle class. Most citizens of countries such as Rwanda, Congo, or Papua New Guinea have traditionally remained illiterate. In 1990 a worldwide initiative was launched, called “Education for All”. It was led by the World Bank and has evolved into a multi-billion dollar fund. About 55 low-income governments have received grants to build schools, buy books, and recruit teachers. Parents desperately want to send children to school, so when schools open, they quickly fill up. But there is a glitch: In very poor circumstances, children fail to learn. A World Bank study estimated in 2012 that only 67% of students in Subsaharan Africa finish primary school and of those who finish, 25% are illiterate.
The ‘learning crisis’, as it is called, has multiple causes. My partner and I spent about 12 years teaching for a charity organization, and we witnessed them first hand. Urban classes have 60-120 students with children seated on the floor. Teachers are often absent, may not know how to teach, and they are never supervised. Corrupt officials often demand bribes, and textbooks are stolen before they get to schools. Children are malnourished and hungry. Not much is taught under these conditions.
Donors such as the World Bank ought to have a good handle on this reality. But their staff hardly visit classrooms. They prefer the company of high officials who send their children to private schools and have private agendas. Most world bankers are economists, so they love the virtual reality of datasets and glossy publications. Incredibly, the donors’ response to scant instruction is not better teaching but better testing. Governments are encouraged to develop learning benchmarks, test students against them, and then figure out how to teach children to achieve the benchmarks.
The triumph of testing over teaching was definitively proclaimed through the “Learning Metrics Task Force” deliberations. The prestigious Brookings Institute conducted a large-scale consultation that involved 1700 staff members of 30 organizations. They were asked to define what children should learn in school and how the learning should be measured. Dozens of organizer staff flew to exotic destinations like Dubai and Bellagio, Italy to deliberate on the findings.
The report was formally launched on September 24, 2013 at the 68th session of the United Nations General Assembly. [click here http://globaleducationfirst.org/2996.htm see entire report here http://www.brookings.edu/research/reports/2013/09/learning-metrics-task-force-universal-learning
The report affirms the need to take immediate action to ensure children’s right to quality education. Nevertheless, it says nothing about the practical obstacles to learning such as corruption, harassment, book thefts, and failure to teach. In fact the word “teach” is not mentioned even once. Students are somehow expected to learn through “opportunities to develop competencies across seven domains of learning, starting in early childhood through adolescence.” To achieve this, a small set of key learning indicators will be tracked globally, such as literacy and numeracy. Countries will obtain technical help to diagnose the quality of their assessment systems, convene stakeholders to determine priorities, identify inequities, and make the appropriate policy changes.
To justify this view, the task force introduces the concept that assessment is a Public Good (pp. 12, 32). No country should be denied the opportunity to test students just because they cannot afford to. Parents and other stakeholders should become advocates of testing (p. 15) and for increased funding for testing (p. 17).
To help children quickly there is not a moment to lose. The task force will meet in November 2013 and develop a plan for moving forward. Launch events will be held in at least 15 cities around the world from September through November 2013, to make stakeholders aware of the test benefits.
The “learning metrics” task force seems so out of touch with reality that its main recommendation is a “Global Paradigm Shift” – from mere investment in access to “access plus learning”. Really, in 2013? Over the last 20 years piles of studies have documented learning failures, while numerous UNESCO workshops have taken place on quality improvement. With the same surreal touch, the document omits references to the large-scale testing that has already taken place. Since the 1990s the kids have been fed alphabet-soup tests such as PASEC, SACMEC, TIMSS, EGRA, EGMA, ASER, Uwezo, and other tests (see http://www.eddataglobal.org). And practically no cases are known of governments that put test results to good uses and improved outcomes.
So why did the Brookings Institution compromise its standards for this initiative? Why not form a teaching-for-poverty task force? Cynics point to money, but experience with poor schools leads to some sobering decisions. Donors mainly want to see activities and feel optimistic for the future. The most productive activity is to help schools teach students, but it is time-consuming, sometimes dangerous and often frustrating. Donors may become disappointed and pull out. By contrast, testing is a winner. Field work takes just a few weeks, and analyses can be done from the comfort of air conditioned offices. The staff involved get invited to international conferences, pad their resumes, get promoted. It’s up to the host governments to use test results for policy improvement.
As the task force rushes into implementation, the only certain outcome is consultant welfare. Testing companies are asked to donate time (p. 35), but seven domains in all countries of the world amount to huge numbers of tests. USAID and other donors have spent millions on testing in the past, so consultant companies and associated nonprofits are preparing for a windfall. Our boss is also optimistic.
The smell of money may be one reason why no one has criticized the report publicly. The people who are building careers and retirement funds from money destined to educate poor kids will strongly argue that they are doing the very best they can for them. Anyway many countries are slowly rising out of poverty, and eventually the poor will turn up educated. It may not be exactly ‘Education for All’, but ‘Testing for All” is considered acceptable progress.
Testing, testing, testing. Metrics mania. How much of an industry it is. And in developing countries who reaps the benefit of this — the testing companies have a bonanza of millions and billions to find out “Is Johnny readin’.” RTI is a prime example. Follow the money. Just look at the fiasco that is the World Bank Trust fund calling itself the Global Partnership Education (GPE). Big money shoved out the door, the same big contractor(s) slavering at the prospect of big bucks. What about teacher education, decent salariies, and dare we say it, “quality” learning? The same old teach for the test mentality like we have in the US. Who actually DOES the testing and establishes the so-called ?metrics”? Why, the OECD companies and “think tanks” which are paid by the foundations and companies……Depressing.