Metrics on Policies and Learning Outcomes for Post-2015: Some Words of Caution By Alexandra Draxler
By Alexandra Draxler, Independent Consultant.
The UN Post-2015 High Level Panel’s call for a “data revolution” and for a new global partnership embracing the private sector is being echoed by most of the classic development institutions. Global measurement of learning and of education systems is on the agendas of Brookings’ Learning Metrics Task Force, the OECD’s PISA for Development, the World Bank’s SABER and the Global Partnership for Education. At a recent NORRAG scoping meeting on Global Governance of Education & Training and the Politics of Data, this author added her voice to those who feel some caution is needed. Here is a brief checklist of points of scrutiny
- Recognize that metrics and testing are ideologically charged. We are living in a time dominated by productivist views of society with the assumption that everything can be commodified and that streamlining, harmonization, and lowered transaction costs (including in personal interactions) represent progress, including in education. This favours the acquisition of measurable skills as the main objective of learning. It is legitimate to look at such assumptions with a healthy dose of scepticism.
- Remember Campbell’s law that the more a quantitative social indicator (e.g. a learning achievement test) is used for decision-making the more apt it will be to distort and corrupt the social processes it is intended to monitor.
- Refuse the tyranny of ranking. Ranking people, or nations, or groups, is a pernicious and counter-productive process that pits them against each other instead of featuring uniqueness and cooperation. Data and statistics have essential functions. None of them should be to produce winners and losers.
- Keep the vendors out of the voting booth. Among the primary beneficiaries of large-scale standardized data collection and testing are the institutions (whether not-for-profit such as the World Bank, OECD or Brookings or for-profit such as Pearson, McGraw Hill and ETS) that develop and administer the instruments and collection techniques. They are also the principal advocates and lobbyists for more and bigger data collection and testing programmes. Driving out conflict of interest from international development initiatives has to be among the priorities.
- Analyse opportunity costs. Every action has an opportunity cost, and the bigger that action the bigger the opportunity cost. So big actions should be subject to analysis not only on their own merits or lack of them but on what they are costing in terms of lost alternatives. In this case, will the data revolution as it is currently taking shape focus on building local capacity and meeting local needs or will we have to wait for a “trickle down” effect?
- Probe the objective of product uniformity. The product here is the learner and her or his outcomes. In the search for efficiency, the productivist model is based on harmonizing outputs to make them reliable and uniform at the lowest possible cost. Since this can only take place through standardization of processes it cannot be what is most desirable for schools or children. Education should celebrate and reward individual differences, creativity, and the discovering of the treasure within each learner.
- Insist on democratic legitimacy for policies. New governance mechanisms and partnerships are being put into place that operate at one or several stages removed from democratic safeguards, imperfect as the latter can be. Public-private partnerships, private philanthropists, independent think tanks, or new global partnerships can bring significant creativity and energy to development initiatives. Governments and groups that accept their intervention need to insist on the application of hard-won regulatory and governance mechanisms that can ensure participation and transparency.
Alexandra Draxler is a former UNESCO education specialist, and now an independent consultant and member of NORRAG’s team. Email: a.draxler@gmail.com