Defining Educational Quality(ies): Can the Right to Quality Education be Fulfilled through Market Incentives?
This post is published as part of the NORRAG Debates stream on the Right to Education. The authors take a quantitative approach to the classic economist argument that market competition incentivizes quality, questioning whether voucher-based competitive funding programs, and the associated use of standardized test scores as a measure of quality, should be the foundation to education policy initiatives. This has implication for right to education movements. Voucher programs reflect a vision that aims to guarantee access to schools where quality standards are defined by performance at standardized tests scores. Yet by doing so they undermine a more comprehensive vision of student development and educational quality therefore not fulfilling, by these other quality standards, the right to a quality education. The post builds on a paper recently published in the International Journal of Educational Development.
Nicolás Grau is an Assistant Professor at the Department of Economics at the Universidad de Chile, where he focuses on empirical microeconomics and economics of education. Felipe Gajardo is also at the Universidad de Chile.
What does it mean to receive a quality education? How much can market-driven initiatives incentivize quality education, in schools and in policy? Milton Friedman’s seminal work, published in 1962, and its promoters have argued that competition for enrollment among schools, and for subsidies when there is a voucher system, would put pressure on schools to improve their educational quality – but how do we define quality?
The Chilean educational system has been a paradigmatic case of a market-oriented educational system since the 1980s, leading to it receiving particular attention in the economic debate about the role of market incentives in education. In (one, possible) theory, quality would be ensured by the demand-based funding system known as vouchers, to the extent that parents decide their children’s schools based on comparative quality, and therefore schools perceived as providing higher quality education receive greater enrollment and more funding. However, this market-based mechanism alone is not enough, and a set of institutions and public policies have been developed in Chile to ensure that minimum quality standards are met as a form of public accountability beyond the market-driven quality incentives.
Due to the ease of using standardized tests, the lack of uniform and systematic data on other potential indicators of school quality, and the ideological bias in our profession, much of the focus of economists has been limited to a narrow definition of school quality, namely, standardized test results. This restricted definition of school quality has been intensely disputed by other disciplines, particularly by those who focus on pedagogical dimensions of the debate. However, economists have been reluctant to change their approach and, given their influence, a narrow definition of school quality based on standardized test scores has been the hegemonic approach in public discourse and as such has gained widespread influence on parental decisions and the design of public policies.
In allowing our attention as researchers to be limited by this definition of school quality, we are not just failing to study the effect of market dynamics on other aspects of educational quality, we are also missing the dynamics and possible tensions between other dimensions of school quality and standardized tests. Empirically documenting this tension could be vital to reimagining and improving educational policy, and ensuring that these policies work so that children have access to an education which strives to provide quality in all its aspects.
To contribute to this debate, in a recent paper accepted for publication at the International Journal of Educational Development, we address the effect of competition among schools – measured as is usual in the academic literature, by the percentage of schools in each district that are subsidized private schools – on a wide range of educational quality indicators beyond standardized test scores: academic self-esteem and motivation, the climate within the school, civic participation, and healthy lifestyle habits — all dimensions defined by the Education Quality Agency of Chile. As in the literature that focuses on the impact of competition on standardized tests, our empirical strategy addresses the potential bias in the estimates due to the endogeneity of the level of competition among schools, using a set of instrumental variables that are related to the size of the potential demand that the schools face in each district.
Following this methodology and using census data for Chilean fourth grade students in 2013, we document a clear difference between the impact of competition on standardized tests and the impact of competition on the other measurements of educational quality. In the case of our most reliable estimates (the instrumental variable approach), an increase in competition by one standard deviation could increase standardized test scores by 0.06 of a standard deviation, which is consistent with previous studies. However, our results also show that competition reduces all other quality indicators: the academic self-esteem and school motivation indicators decrease by 0.02 of a standard deviation, the school climate indicator by 0.1 of a standard deviation, the civic participation indicators by between 0.06 and 0.09 of a standard deviation, and the healthy lifestyle habit indicator by between 0.08 and 0.16 of a standard deviation.
Thus, we quantitatively document the impact of competition among schools on aspects of educational quality outside of standardized tests and, by doing so, we call attention to the tension between standardized tests and other measures of school quality. These results are consistent with the qualitative evidence available for Chile, which also find a tension between improving standardized tests and promoting more comprehensive student development (see for instance, Falabella and Opazo, 2014).
What could explain our results? Paradoxically, the answer could be in economic theory (and common sense), but not in the kind of theory that economists commonly invoke when they intervene in education policy debates, which is simplistic and not realistic. From the perspective of microeconomics, the model of problems of agency with multiple tasks (Dewatripont et al., 2000) allows one to interpret the conflict of incentives that a school faces between only improving performance on standardized tests and putting also some effort in some other areas. If the government and parents (i.e., the principal) are interested in promoting quality by only considering standardized test results when allocating resources or choosing the school for their kids (which determines the resources allocation in a voucher system), the schools (i.e., the agent) will confront explicit incentives that will make it prefer to act negligently, by dedicating the largest number of hours to preparing students for a standardized test rather than spending that time on improving other aspects of educational quality. In sum, faced with the power of the incentives that the principal grants, the agent will leave aside tasks that provide less compensation, which in this case are the other aspects of educational quality.
Our research adds to the literature on negative effects of the widespread voucher system implemented in Chile since the 1980s. This literature has found solid evidence of the impact of competition on school segregation (see Hsieh and Urquiola 2006); and it has found an impact of market dynamics, which implies significant numbers of schools closing every year as well as on grade retention and students dropping out (see Grau et al., Forthcoming).
Should competition and a system of incentives based on standardized test scores be avoided altogether? Even though the results of our paper show negative aspects to the Chilean voucher system, a proper answer to that question requires a more comprehensive evaluation of the system. The contribution of our paper is more modest, namely, we present quantitative evidence of a drawback of the Chilean educational system that have not been addressed. By doing so we contribute a simple message which should be useful beyond the Chilean experience: if all the incentives that schools face are related to standardized test scores, schools will naturally neglect other aspects of school quality to the detriment of the school and the students.
Felipe Gajardo. Universidad de Chile
Nicolás Grau. Economics Department, Universidad de Chile and COES – Centre for Social Conflict and Cohesion Studies.
References:
Dewatripont M., Jewitt I., and Tirole J. (2000). Multitask agency problems: Focus and task clustering. European Economic Review. 44, 869-877.
Falabella and Opazo (2014). Sistema de Aseguramiento de la Calidad y procesos de mejoramiento: una mirada desde la gestión educativa. Centro de Investigación y Desarrollo de la Educación. Universidad Alberto Hurtado.
Grau N., Hojman, D., and Mizala, A. School Closure and Educational Attainment: Evidence from a Market-based System. Economics of Education Review (Forthcoming).
Hsieh C., and Urquiola M., (2006). The effects of generalized school choice on achievement and stratification: Evidence from Chile’s voucher program. Journal of Public Economics. 90, 1477-1503.
Contribute: The NORRAG Blog provides a platform for debate and ideas exchange for education stakeholders. Therefore if you would like to contribute to the discussion by writing your own blog post please visit our dedicated contribute page for detailed instructions on how to submit.
Disclaimer: NORRAG’s blog offers a space for dialogue about issues, research and opinion on education and development. The views and factual claims made in NORRAG posts are the responsibility of their authors and are not necessarily representative of NORRAG’s opinion, policy or activities.
This is a very important finding and it raises questions far beyond the impact of voucher schemes. ALL major recommendations on education interventions, policies, and reforms are based on research that looks ONLY at impact on standardized tests, almost exclusively language and math. Look, for example, at the recommendations in the World Bank’s 2018 World Development Report or on its SABER platform — or those of the Education Commission. However, schools are intended to do a lot more than teach reading and math. Gajardo and Grau’s research shows what I have argued for a long time — that a focus on narrow cognitive skills may come at the expense of other things we want schools to teach. This is an empirical question. In economics language, many outcomes are jointly produced by schooling and it is sub-optimal to make policy without taking that into account. That is, basing educational recommendations on research that only studies reading and math impacts makes no sense and could well be counterproductive.
That tests tend to focus upon reading and math (sic) is not the problem. The problem is that testing itself, and recommendations based upon the findings of tests, damages education. A decade or so from now, people will wonder, with horror and amazement, at the way in which early-21st century children and young adults were subjected to examinations and assessment, along with their teachers being prevented from teaching by a never ending requirement to mark, categorise and report. As the universal lifelong Global School experience eventuates, odious trans-country comparisons by such as PISA, along with national league tables, will thankfully become redundant. The Global School consigns compulsive testing to the asylum of educational psychoses. Test-obsessed, performance-comparison-driven schooling must and may now be relegated to the dark (i.e. pre-digital) ages.