Data Collection and Evidence Building to Support Education in Emergencies
This NORRAG Highlights published by Mary Mendenhall, Associate Professor of Practice and the Director of the International and Comparative Education program at Teachers College, Columbia University, and Arianna Pacifico, a doctoral student in the International and Comparative Education program at Teachers College, Columbia University, shows why quality data on Education in Emergencies (EiE) are critical by highlighting the contribution that NORRAG Special Issue 02 (NSI 02), launched in June 2019, makes to this field of research. The authors discuss the challenges concerning data collection in EiE and promising practices that can help to overcome them. Mendenhall and Pacifico shed light on the progress made in building a strong evidence base for the EiE field.
Quality data about the field of education in emergencies (EiE) are critical for making evidence-informed policy and programming decisions to better meet the education rights and needs of learners in conflict and emergency situations. Access to and availability of reliable data for humanitarian responses are even more critical amidst the financial constraints that confront the education sector in these contexts and can help inform decisions about what type of education programming is needed, where it is needed, and which communities affected by crisis need it most. NORRAG Special Issue 02: Data Collection and Evidence Building to Support Education in Emergencies, which we had the pleasure of compiling, pulls together examples of interesting research and evidence-generating approaches currently being used to ensure the provision of quality education in emergencies. The English version of NSI 02 entails 32 articles written by 61 authors about projects in multiple country contexts and captures the challenges we face in the education sector as well as the promising practices that can help us overcome them. Selected articles have been translated in Arabic, Mandarin Chinese and Spanish (stay tuned for forthcoming translations in French and Russian).
In Part 1 of NORRAG Special Issue 02, which provides an overview of the state of research in EiE, co-authors Buckner, Smiley and Cremin provide a great discussion about how humanitarian and development organizations collect and use data differently and how there are logical ways to bridge the gap through data sharing and harmonization. Drawing on their work in the Middle East, they show that data collected in EiE contexts are primarily used to inform sector coordination, program design, program effectiveness, policy and decision-making, and advocacy. They argue that we need to “foreground data usage in conversations about educational data in conflict-affected areas, moving away from talking about data in EiE in the abstract to specificifying data for what, where, and for what purpose” (p. 21, emphasis in original).
In Part 2, which highlights promising methodologies for understanding “what works” in EiE, Brown and Ngoga address the risks associated with collecting poor quality data and how the use of a “field-feasible observation protocol [that] focuses on core competencies of enumerator quality” can go a long way in mitigating related challenges. Reflecting on their work in Niger, they share their approach to providing structured training, quality supervision of data collection staff (i.e. the enumerators), and direct observation of enumerators to help them implement the research protocol as intended. The experiences and related results of these approaches serve as critical feedback loops to strengthen ongoing capacity development activities for enumerators. This article provides rich detail about their approach and useful lessons that will be helpful for other EiE projects that involve training and supporting enumerators.
In Part 3, which shares promising practices for data generation and application in complex contexts of forced displacement, Davis and Payan illustrate how baseline data from a USAID-funded project, Asegurando la Educación, that was conducted in 66 schools in Honduras has the potential to help school and community stakeholders develop rapid responses to important issues emerging in the data, including school safety. Often, data analysis takes a long time, which prevents the more immediate use of research findings to inform program design and effective responses. In this project in Honduras, project staff shared both national-level and school-level findings indicating that significant percentages of students experienced physical and/or emotional aggression, and felt sad and hopeless about their lives. When teachers and parents learned of these findings from the baseline data, some of the ways in which they responded across different schools included: offering anti-bullying workshops and campaigns; infusing anti-bullying messages into existing school activities; inviting external organizations and speakers to address issues such as gender-based violence and discrimination; increased monitoring of bathrooms and policies for students to go in pairs given students’ vulnerabilities in this location in some schools; and teacher training on how to conduct lessons on substance abuse without aggravating drug dealers working in the local community. These examples show how local school and community members can use good data (already at the baseline phase) to develop cost-effective and locally-generated practices and policies for school safety.
In Part 4, which highlights data needs related to critical issues and marginalized groups in emergencies, Symonds recounts the Aga Khan Foundation’s work through a multi-partner consortium that supports a girls’ and community-based education program called Education Success/STAGES in Afghanistan. Community-based education approaches have proven to be highly effective in Afghanistan and also necessary due to protracted crises that have affected the education system and children’s access to safe schools. The project is administering a quasi-experimental evaluation design conducted by an external evaluator to measure project results. In its focus on girls’ education, the project tracks girls over a long-term period, from baseline to endline, and looks at girls’ attendance, supportive community attitudes towards girls’ education, teaching quality, school management practices, and life skills. Symonds explains the methodology used, the significant challenges the project faces in conducting this work in Afghanistan due to mobility, attrition, and security issues that prevent access to research participants. Symonds also shows how the research findings from a project conducted in remote, rural, and controlled areas can contribute to national and local policies and practices, particularly related to community-based education and due to the absence of a national assessment system. The findings also feed into the ongoing development and improvement of the STAGES intervention.
Finally, in Part 5, several authors across different articles offer sound advice and cautionary tales about the ethical issues and related professional development needs that are paramount to our work in EiE contexts. Maglio and Pherali capture the array of ethical issues, including exposing research participants to ‘re-traumatisation’ through our questioning, especially with children and young people living in states of crisis and displacement, and continually asking the same participants or communities to participate in different research studies, among others. They provide examples of how humanitarian organizations, which are increasingly engaged in research activities, often hire consultants without the requisite training to conduct ethical research. Equally concerning, they shed light on the lack of personal security that consultants and other researchers face in many EiE contexts. Adelman and Chopra pick up this thread in their article, which reflects on the type of preparation that is needed for graduate students conducting research in EiE contexts. They highlight the potentially limited support that might be provided by both their academic institutions and the agencies with which they might affiliate, to support not only their research, but also their personal security and well-being during their field studies. Steele also addresses the issue of personal security for both enumerators and research participants in a study her organization conducted in Syria. She provides clear details about the steps the research team took to ensure quality data collection while also upholding methodological rigor, ethics, and concern for everyone’s personal safety. This section of NSI 02 will hopefully serve as an important reminder to researchers (and NORRAG readers alike!) that we must uphold high ethical standards of research and document what this entails in EiE settings. Another article in Part 5 by Martin and Umubyeyi also calls for meaningful participation of research participants as co-researchers, not only in the research process, but also in writing, presenting, and co-authoring of findings and recommendations. These are all issues that need to be addressed more consistently when conducting EiE research.
There is still a long way to go to build a strong evidence base for the EiE field, but promising efforts are underway, and of course the NORRAG Special Issue 02 is a great example. We can also point to NORRAG and INEE’s partnership, along with the USAID Middle East Education, Research, and Training Support (MEERS) Program, to further move these efforts forward with the upcoming EiE Data Summit, scheduled for June 20-21, 2019 in Geneva. We also take notice of the promising developments within the donor community to require, support, and invest in research as a significant part of their aid packages (see Dubai Cares Evidence for Education in Emergencies research envelope; the European Commission’s DEVCO’s Building Resilience in Education through Crisis initiative; and Dfid’s forthcoming Education Research in Conflict and Protracted Crisis request for proposals, to name several key examples).
The collective learning from these efforts can be further showcased in INEE’s Journal on Education in Emergencies through open-source research articles, field notes, and book reviews. In NSI 02, Dana Burde (editor-in-chief), and her colleagues Heddy Lahmann and Nathan Thompson, reflect on the journal’s efforts to create a research community to “promote the production, sharing, and use of rigorous evidence in EiE.” We would also highlight the FreshEd Podcast with Will Brehm, which aims to make “complex ideas in educational research easily understood” (for more highlights about Special Issue 02, check out a recent FreshEd Podcast with Mary Mendenhall). We look forward to reflecting a few years down the road on the contributions of these developments to helping build a stronger and more meaningful evidence base that supports our collective efforts to improve the provision of education in emergencies for all children, adolescents, and youth affected by crisis.
About the authors:
Mary Mendenhall is an Associate Professor of Practice and the Director of the International and Comparative Education program at Teachers College, Columbia University. She served as the Guest Editor of NORRAG Special Issue 02. Twitter: @MaryMendenhall1
Arianna Pacifico is a doctoral student in the International and Comparative Education program at Teachers College, Columbia University and the former Coordinator for INEE’s Standards and Practice Working Group. Twitter: @sloatari
Contribute: The NORRAG Blog provides a platform for debate and ideas exchange for education stakeholders. Therefore if you would like to contribute to the discussion by writing your own blog post please visit our dedicated contribute page for detailed instructions on how to submit.
Disclaimer: NORRAG’s blog offers a space for dialogue about issues, research and opinion on education and development. The views and factual claims made in NORRAG posts are the responsibility of their authors and are not necessarily representative of NORRAG’s opinion, policy or activities.