International Benchmarking and Measuring the Quality of Learning By NORRAG

By NORRAG.

Current post-2015 proposals for universal goals, targets and indicators, and the mushrooming of global initiatives, meetings and reports, suggest a shift of focus away from developing country contexts and towards a global framework of development. One of the key elements of this framework seems to be a strong push for internationally comparable data on learning outcomes, notably through a “data revolution” called for by the UN High Level Panel in 2013.

Between 8-13 March 2015, the 59th Annual Conference of the Comparative and International Education Society (CIES) is due to take place in Washington, DC. NORRAG, along with AIR, Education International, GLOBED Master Consortium and the Open Society Foundations, is planning three panels at the 2015 CIES Conference. In the spirit of Ubuntu as described for the theme of CIES, NORRAG is looking at pluralistic approaches to benchmarking and measuring learning, which envision education as a cornerstone for the development of the whole individual who would become an active and thoughtful citizen in both social and economic spheres.

Big data, big questions

The collection of massive amounts of data on systems and participants in education, for example, could benefit by being examined in the light of potential unintended consequences, national and local needs, corporate vested interests and a more holistic approach to education only partially captured through standardized testing metrics. Moreover, funding is not unlimited, and international calls for comparable data on learning will likely strengthen global and regional institutions in contrast to national or local ones.

The first panel, on ‘Big data, big questions’, will explore questions concerning the political economy of big data collection efforts and administration of standardized tests designed to evaluate and rank student and teacher performance. There is a need to interrogate key assumptions about the need for, and characteristics of, large-scale data collection and testing. Who defines quality, and how? Is comparability truly possible and when is it necessary? Once needs have been defined, how can projects and interventions be taken to an appropriate scale? Who are and who should be the actors in deciding the what, how and why of measurement? Who are the custodians and purveyors of the results? Who has access and who should have access? What obligations do private sector actors have with respect to the collection, storage, ownership and use of tools and results, and with what controls? How will it be possible to determine whether principles of social justice and quality are being followed or compromised? And, finally, what mechanisms and conditions need to be established to make sure decisions and implementation are both fair and open to those who will be most affected by the use of data?

The presentations will include:

• Is a ‘data revolution’ on learning needed? Critical views and alternative approaches – Aaron Benavot, Director, Education for All Global Monitoring Report.
• Is it possible to capture quality and equity in education? Challenges and opportunities in the context of post-2015 debate – Antoni Verger, Xavier Bonal and Adrián Zancajo, Department of Sociology of the Universitat Autònoma de Barcelona.
• From measuring the world to measuring education. Departures from Daniel Kehlmann’s novel to a consideration of efficiency, privacy, transparency and control – Hugh McLean, Open Society Foundations.
• Private sector participation in delivery of public goods: whose needs whose benefits? – Alexandra Draxler, NORRAG. 

Alternative and complementary methodologies

The second panel, on ‘Alternative and complementary methodologies’, will look at initiatives that focus on quality, citizen participation (including education personnel), examples of broad measures of quality, and evaluating the evaluators. The panel will be organized around three complementary perspectives to current approaches to the post-2015 “data revolution”:

1. A focus on longitudinal studies and effects: many desirable outcomes of education are medium and long-term e.g., as productive workers, active citizens, discerning consumers, problem solvers, and family formers. Most benchmarking of “learning outcomes” is necessarily a snapshot at a particular time point, which thus provides limited evidence about longer-term impacts of education. It is entirely possible that existing data can be better utilized to tease out longer-term impacts of learning on individuals, families, communities and countries to support more longitudinal approaches to data collection and analysis.
2. Impact assessment: there is an urgent need to understand the impact of assessment practices. How do we develop, test and promote ways to more closely scrutinize reforms in learning assessment and related processes, notably in light of both expected desirable and unintended consequences?
3. Capacity building and empowerment for national and local data gathering: What kinds of assessment practices and information are more likely to result in real change in schools and classrooms? What should be the roles of teachers, citizens, school leaders, academics, and assessment specialists in improving and conducting learning assessments? How can different actors be empowered to increase their voice and participation in existing assessment techniques?

The presentations under this panel will include:

• On the tension between quantity, quality and equality in education: Reflections on demographic interferences to education targets – Parfait M. Eloundou-Enyegue, Associate Professor of Development Sociology, Cornell University.
• The ‘Young Lives’ longitudinal study – Angela W Little, Professor Emerita, Institute of Education, University of London.
• Improvement versus control: A view from the classroom on authentic assessment and professional judgment – James Tweyo, President of the Ugandan Teachers Union.
• Exceptionality: A case study of South African participation in internationally bench-marked standardising processes in education – Crain Soudien, Deputy Vice-Chancellor, University of Cape Town.

Addressing urban violence though education

The recognition that urban violence is a considerable threat to the stability and development of states and the wellbeing of their citizens has led national and international researchers to examine the role and effectiveness of violence prevention and reduction programs or citizen security programs in urban settings. Consequently, there has been a number of mapping exercises and analyses involving these initiatives. Recent examination of such initiatives (especially in Latin America) has shown a shift from ‘heavy-handed’ approaches (which emphasize the use of police force for dealing with violence), towards ‘softer handed’ civil society methods (which aim to create order through prevention or addressing conflict drivers). Often, education and training is at the heart of these methods.

Given the changing nature of conflict and violence globally – one that sees both armed conflict and so-called non-conflict increasingly taking place in urban settings – the third panel, on ‘Addressing urban violence though education’, will explore a simple question: what formal and nonformal education strategies are being implemented to address urban violence?

Brazil and South Africa are two examples of countries which have shown high levels of socio-economic development, but also inequality and interpersonal and public violence. Both countries have addressed violence, but have approached it from different contextual perspectives. In Brazil, armed violence prevention and reduction programs and citizen security programs seek to address conflict in urban settings. In South Africa, civil society has taken the lead on violence prevention and reduction, less through a security lens, and more through a socio-economic/poverty reduction lens. In both cases, initiatives that focus on formal and nonformal education (broadly conceived) can offer insights into good practices and lessons learned. Indeed, the case studies of Brazil and South Africa illustrate the relationship between conflict, violence, and education programming opportunities.

The presentations under this panel will include:

• Research design – Amy West, AIR and Joost Monks, NORRAG
• South Africa case study – Amy West and Claire Nowlin, AIR
• Brazil case study – Joost Monks, NORRAG

>> Read existing NORRAG logs on Data and Learning Assessments

NORRAG (Network for International Policies and Cooperation in Education and Training) is an internationally recognised, multi-stakeholder network which has been seeking to inform, challenge and influence international education and training policies and cooperation for almost 30 years. NORRAG has more than 4,200 registered members worldwide and is free to join. Not a member? Join free here.