Reflections on What we Have Learned Since Completing the UNESCO Study on How to Improve the Use of Evidence in Education
NORRAG was commissioned by the UNESCO Education 2030 Section to provide research and recommendations on how to improve the use of evidence in education policy, practice and planning to support the newly established SDG4-Education 2030 High-Level Steering Committee (HLSC).
In this blogpost three authors reflect on what they have learned in the intervening time since completing the study.
Moira V. Faul: In the UNESCO study, we adopted the language of “global” and “local”. Global was used to denote evidence products that are promoted as “global public goods” published by international organizations, universities and think tanks (or other knowledge producers) based in the Global North. Local was used to describe evidence that was produced in the Global South. We did specify that Global South knowledge was useful and relevant in the localities where it was produced and potentially also globally, in the same way that knowledge produced in the Global North might have global utility as well as local. So we made the argument strongly that Global South and North knowledges can be locally and globally relevant. Nevertheless, using the language of global vs. local gives the false impression that what is produced in and by the Global North (denoted as global) is placeless and universal, in contrast to what we presented as contextually grounded, locally relevant evidence from the Global South (local). Going forward, I will not use language that discriminates between “global and local” (see Kothari et al., 2019, for example). Rather, I will use language that underlines (rather than elides) the equal “localness”—and equal potential global usefulness—of evidence produced in both Global South and North.
Another learning point for me was to acknowledge the importance of politics in decision making. Within this study, we moved the debate forward from purely rational accounts of evidence production and use to also examine relational and systemic ways to improve evidence use (Hopkins et al., 2021). However, evidence is not the only factor in policy making; democratic deliberation in decision making—in parliaments and with people—is another critical aspect of policy making that cannot and should not be ignored.
Gita Steiner-Khamsi: I see the silhouettes of the Tower of Babel when I visualize what has changed over the past two years since we first delved into the question of when, how, and to what extent research evidence is used for policy and planning. The miscommunication is particularly concerning when the purpose of research evidence is being addressed. As a result, international donors only hear their own echo when they promote a greater use of research evidence in countries of the Global South. They speak about scaling innovations but what they really mean is institutionalization of donor-funded pilot projects, they speak about using research evidence but what they really mean is the use of technical reports produced by international consultants, and they speak about data but what they really mean is statistical information. The type of evidence, and as a corollary the type of knowledge mobilized, varies greatly depending on what is actually meant. Given the proliferation of talk on what works and what doesn’t, the paucity of discussion of how institutionalization works is remarkable. Implementation science is needed to guide us on what works, but it is institutionalization science that one would need to consult for making informed decisions on how to make an innovation “stick,” that is, on how to increase the survival rate and, even better, how to extend the lifespan of effective policies. Asked rhetorically, do governments in the Global South really need to be told what works in their country or is there rather a demand from their end to understand how existing, positive reform features may be sustained given their financial and capacity constraints?
The scholarship on institutionalization suffers of course from the same ailments as the communication between international donors and recipient governments: a knowledge inequity reflected in the surplus of international advice and a shortage of useful, locally produced research evidence. Most of the scholarship on institutional change deals with public administration in countries of the Global North. The five main types of institutional change (layering, conversation, drift, displacement and exhaustion), explained a while back by Mahoney and Thelen (2009), still hold. Nevertheless, the question becomes: Does this fivefold typology of institutional change also apply to how reforms in aid-dependent countries are sustained? In aid-dependent countries, layering or putting an innovation on top of something old, oftentimes only for the duration of external funding, seems to be the norm and not the exception. There is an urgent need to support local policy expertise that help all of us understand, including experts based in countries of the Global North, how ephemeral layering as the preferred mode of incremental change may be replaced with long-lasting, “deep” institutional change. Phasing out or sunsetting outdated and dysfunctional policies is a science too. Perhaps the old could make place for the new if only more local experts are given the opportunity to share their experience and knowledge of why layering alone does not lead to sustained change and, on the contrary, absorbs the already limited capacity and resources available in their country.
Chanwoong Baek: Understanding the intricate global landscape of research evidence utilization for policy, planning, and implementation is a challenging and complex task. In this study, we endeavored to uncover the current state of affairs by focusing on the evidence sources that the policy actors draw upon. Despite the abundance of sources identified by our study participants, there was a noticeable inequality in attention across SDG4 targets, themes, and topics. Additionally, the primary research and evidence sources that our study participants frequently consulted were predominantly international, particularly those associated with international organizations such as UNESCO, OECD, World Bank, and UNICEF, with loose connections between national and international sources. Participants in the study noted a lack of regional initiatives, networks, platforms, and organizations that could facilitate the flow of knowledge between international and national sources. Regarding strategies to enhance the uptake and utility of research evidence, survey respondents and interview informants stressed the importance of national relevance and participation.
What is both intriguing and disappointing is the apparent lack of significant change since the study was conducted in 2021. While many actors and organizations continue to produce evidence, there remains a gap in its utilization by policy actors. Evidence produced in and by the Global North continues to receive more prominence and attention. There is a disproportionate focus on a few SDG4 targets (falsely) perceived to be of shared interest across different regional and national contexts. To address these issues, the need for locally relevant evidence and the involvement and support of national expertise is widely acknowledged and emphasized (Persson et al., 2018; Stewart, 2021). However, this need is often challenged and diluted by various organizational interests, outdated norms and practices, and (inter)national politics. This suggests that while our focus may have predominantly been on the sources and content of evidence and expertise, there needs to be a greater exploration on how, when, by whom, and for what purposes evidence and expertise are selected, mediated, and utilized.
Thinking together and moving forward: Thinking together—as we had the privilege to do in this study—helped us to produce a report that we believe can move forward the understanding and mobilization of evidence for policy in ways that contribute to the achievement of SDG4. As is often the case, researching and writing moves forward authors’ thinking in ways that are not always reflected in the final document produced. This blogpost has allowed us to reflect on how our thinking has developed—as a result of this study and other research we have undertaken. In order for our knowledge and thinking on this topic to evolve, we would greatly welcome responses to and reflections on our report and on this blogpost.
References
Hopkins, A., Oliver, K., Boaz, A., Guillot-Wright, S., & Cairney, P. (2021). Are research-policy engagement activities informed by policy theory and evidence? 7 challenges to the UK impact agenda. Policy Design and Practice, 4(3), 341-356.
Kothari, A., Salleh, A., Escobar, A., Demaria, F., & Acosta, A. (2019). Crisis as opportunity. In Postdevelopment in practice: Alternatives, economies, ontologies. pp.100-116.
Mahoney, J. and Thelen, K., eds. (2009). Explaining Institutional Change. Ambiguity, Agency, and Power. Cambridge University Press.
Persson, J., Johansson, E., & Olsson, L. (2018). Harnessing local knowledge for scientific knowledge production: challenges and pitfalls within evidence-based sustainability studies. Ecology and Society, 23(4).
Stewart, R. (2021). Turning ‘evidence for development’ on its head: A view from Africa. Research for All, 5(1).
About the Authors:
Moira V. Faul, Executive Director, NORRAG and Senior Lecturer, Interdisciplinary Programmes, Geneva Graduate Institute.
Gita Steiner-Khamsi, Professor, Teachers College, Columbia University and Honorary UNESCO Chair of Comparative Education Policy of the Geneva Graduate Institute.
Chanwoong Baek, Assistant Professor, International Relations and Political Science Department and Academic Director, NORRAG, Geneva Graduate Institute, UNESCO Co-Chair in Comparative Education Policy.