PISA-b on Proxima Centauri: A speculative story about the costs and benefits of ILSAs
In this NORRAG Highlights contributed by Sam Sellar (Manchester Metropolitan University) and Camilla Addey (Teachers College, Columbia University), the authors imagine life on Proxima Centauri b, the nearest habitable exoplanet to our solar system, where debate also rages on the benefits and pitfalls of comparing education systems using intra-planetary large-scale assessments of schooling (ILSAs) and the Programme for International Student Assessment on Proxima Centauri b (PISA-b).
Many thousands of years after the extinction event on Earth, the lands on Earth’s nearest habitable neighbour, Proxima Centauri b, were also divided into nations after thousands of years of conflict over territory. These nations had many differences, but they also shared many similarities. The peoples of all nations passed on knowledge to the next generation in a place called ‘school’, and all nations had governments that oversaw a system of these schools. At a young age, each child would begin their schooling and in most nations young people were required to continue their schooling until approximately 497 years of age. (Proxima Centauri has an orbital period of 11 days, so this is roughly equivalent to 15 Earth years).
But everyone was not equal on Proxima Centauri. Some nations had more wealth than others, and some people within nations had better lives than their fellow citizens. Some children attended school for as many years as their peers, but left school with relatively underdeveloped capacities for gaining employment and making a good life. In contrast, other young people acquired deep and diverse sets of capacities from school and this opened up many paths toward the lives they desired. Curiously, in some nations schools seemed to distribute life opportunities more equally than in others. What were these schools and systems doing differently? The representatives of many nations gathered under the Grand Dome in the city of Unity, protected from the stellar winds in the border zone between the light and dark sides of the planet. This gathering of politicians began devising a plan for comparing schools in all nations of Proxima Centauri and this initial meeting gave rise to the phenomenon of intra-planetary large-scale assessments of schooling (ILSAs).
Over time, a range of different ILSAs were developed to measure how well students learned the curriculum in their schools or what young people could do after living for 497 years in their societies. These tests focused on reading, mathematics and science, but also other traits like creativity and collaboration. The most influential of all the tests was the Programme for International Student Assessment on Proxima Centauri b, or PISA-b for short.
The results from these international tests were often shocking. In some nations, the schools that many had assumed to be very good performed poorly on the test. Although each national learning context was different, experts started to identify common factors related to curriculum, pedagogy and assessment and proposed many ‘nickel bullet’ solutions that would improve student outcomes in all schools. (The rocky surface of Proxima Centauri b contained almost no deposits of the element, Ag). Tests were carried out every few years, data were analysed, proposals were made, and schools were reformed.
But after one thousand years or so, complaints about the tests grew in volume and ferocity. Why had the nickel bullet solutions failed to make a difference? Why were more and more nations taking the tests when there seemed to be few tangible benefits? Were the tests improving education or the economy on Proxima Centauri b? Was this even possible using top-down approaches? In one of the largest nations, a hyperabacus entrepreneur spent billions on education reform, but was forced to admit defeat after his search for nickel bullet solutions failed.
Scholars from universities around the planet began exploring these questions. Some scholars sought to understand how comparisons between nations were being used and how ideas about schools travelled from nation to nation, while other scholars asked whether such tests could support nickel bullet solutions given the complexity of learning and the limited nature of what was being tested. In 20,018AD (AD stood for ‘after the Dome’, which was the time when the first dome community was established), two researchers raised an important question about the costs and benefits of ILSAs. This contribution marked a turning point in the planetary debate about testing.
Around this time, a paper was commissioned by the Unity Education Network (U-EN) that brought nations together to discuss educational matters. Two researchers were invited to table a paper on the question of why nations participated, or chose not to participate, in ILSAs. The researchers reviewed scholarly literature and empirical data from their previous studies. What they found was indeed surprising. It was commonly believed that ILSAs were designed to help identify how well schools and societies prepared people to become valuable workers. However, the paper tabled by the two researchers showed that governments in different nations were participating for many different reasons. Motivations for participating often changed, and most surprisingly, these motivations were not necessarily related to the purposes for which the tests were designed or even educational matters! Participation could be motivated by political, economic, technical, socio-cultural rationales, and often by a profane mix of different rationales.
In some nations, ILSAs were used to attract intra- and inter-planetary investment, or to access political and trade unions, or even to put new and contested nations on the map. The question of whether ILSAs were helping to improve schooling and the life chances of young people in different nations was too big to answer at this time? But the two researchers did suggest that all nations could conduct analyses of the costs and benefits of participation, and that these analyses should take into account the different motivations that each nation had for participating.
The two researchers found that some nations were already embarking on the types of cost-benefit analyses that they supported. A poor nation on the dark side of the planet participated in an ILSA that did not immediately improve schooling, but it did provide access to resources that could make a difference. A wealthy nation on the light side of the borderzone conducted a rare evaluation of PISA-b and showed that the results were not being properly interpreted by policymakers in their school system. Why spend money on a test that did not provide valid information? Some nations found that poor performance on the tests caused embarrassment in the eyes of Proxima Centauri b’s planetary community. Other nations found that the tests neglected the most important values underpinning education in their schools. In some nations, the tests proved useful because they enabled comparison of different sub-national dome communities, not because they enabled comparisons with domes on the other side of the planet.
The two researchers were also asked by the U-EN to make predictions about the future of ILSAs. Would nations continue to use the tests, or would they find new ways to produce information about schools and how well these strange places prepared people for work and to live good lives? The researchers explained that contextual factors such as the Proxima Centauri b’s planetary education agenda, changing views about the importance of testing, and the development of new and broader ILSAs could have a positive impact on the future of these tests. But expansion of the ILSA phenomenon could also be met with resistance and alternative methods for generating information about schools might emerge. Importantly, Proxima Centauri b was undergoing dramatic technological changes that were set to profoundly change its societies and these would inevitably impact on ILSAs.
The paper tabled by the two researchers did not resolve questions about the costs and benefits of participating in ILSAs for any particular nation. The value of participating in ILSAs depended on the different rationales that nations had for participating. However, debate about purposes, costs and benefits did enable clearer and more informed debate about ILSAs, and this played a very small part in paving way for the Disillusio Summit, where educators from around Proxima Centauri b came together to question the significance and promised benefits of nickel bullet approaches to improving the schools on their planet.
This blog entry is based on a paper published by UNESCO: Addey, C., and Sellar. S. E. 2019. Is it worth it? Rationales for (Non)participation in international large-scale learning assessments. Education Research and Foresight Working Papers Series, No. 24. Paris, UNESCO. https://en.unesco.org/node/268820
About the authors:
Sam Sellar is Reader in Education Studies at Manchester Metropolitan University. Sam’s research focuses on education policy, large-scale assessments and digital data. He is currently finalising an international comparative study of data infrastructure in schools and school systems in Australia, the USA, Canada and Japan, and he has recently begun a new project examining the impact of AI in education policy making. Sam’s most recent book is the World Yearbook of Education 2019: Comparative Methodology in an Era of Big Data and Global Networks (2019, Routledge), co-edited with Radhika Gorur and Gita Steiner-Khamsi.
Camilla Addey is a lecturer in Comparative and International Education at Teachers College, Columbia University (USA). Previously, Camilla worked as a researcher at Humboldt University in Berlin, and between 2009 and 2010, she worked at UNESCO in the Literacy and Non-Formal Education section. Camilla researches international large-scale assessments (ILSAs), global educational policy, and education privatizations. In autumn 2019, Camilla will join the Globalisation, Education and Social Policies (GEPS) research centre at the Universitat Autonoma de Barcelona as a Marie Curie Fellow to research the ILSA industry. Camilla is one of the directors of the Laboratory of International Assessment Studies.
Contribute: The NORRAG Blog provides a platform for debate and ideas exchange for education stakeholders. Therefore if you would like to contribute to the discussion by writing your own blog post please visit our dedicated contribute page for detailed instructions on how to submit.
Disclaimer: NORRAG’s blog offers a space for dialogue about issues, research and opinion on education and development. The views and factual claims made in NORRAG posts are the responsibility of their authors and are not necessarily representative of NORRAG’s opinion, policy or activities.