Inclusive Standardised Large-Scale Assessments: Prospects and Promises
Tuesday 19 March 2024
12:00pm (noon) to 1:00 pm CET
Online
Inclusive Standardised Large-Scale Assessments: Prospects and Promises
A key limitation of standardised large-scale assessments has been that it excludes certain populations – those who might be ‘non-standard’ or ‘outliers’ in one way or another. This is at odds with the philosophy of inclusivity which is now more and more widely demanded. The needs and rights of neurodiverse students and those with special needs or learning or other disabilities are now acknowledged as being important to recognise. This exciting seminar, jointly organised by the Laboratory of International Assessment Studies, NORRAG, and REDI (Deakin University), explored cutting-edge research that seeks to enable international large-scale assessments, including PISA, to become more inclusive, without compromising the rigour and quality of the assessment tools. Bryan Maddox (Digital Education Futures Initiative, University of Cambridge) discussed his work with Bruno Zumbo (University of British Columbia) on the impacts of improved accessibility in international large-scale assessments, including digital accessibility technologies and universal design principles. They argue that these new developments afford an opportunity to learn more about the diverse range of test-takers as well as their test-taking experiences. Ava Guez (OECD) and Elodie Persem (French Ministry of Education) reported on the current status of research on how PISA can be made more inclusive, detailing a proposed pilot study of accessibility options for students with special educational needs. Discussant Moira Faul (NORRAG) provided her expert commentary on the presentations. This was followed by an engaged Q&A session.
Abstract 1
By Ava Guez, Analyst, OECD, France and Elodie Persem, Head of Accessibility, Innovation and Research Unit, Department of Evaluation, Forecasting and Performance, French Ministry of National Education and Youth, France
Improving accessibility in PISA for students with special education needs In many countries, children with special education needs represent a significant proportion of students in school. Yet, due to various technical and methodological challenges, PISA has to date offered only limited accommodations for them. As a result, some students are currently excluded from the PISA target population at the sampling stage, and in some countries, exclusion rates are growing as more and more students are recognized as having disabilities that require testing accommodations. This practice in PISA contrasts with testing standards in many countries which call for the inclusion of students with disabilities in order to give every student the right to demonstrate their skills and to generate information that represents all students. In order to tackle these concerns and move PISA forward, the OECD has launched a research project aimed at improving accessibility in the assessment. This presentation will present the context, objectives and current research outcomes of this project, including results from a country survey providing an overview of accessibility practices in national assessments in PISA participating countries, as well as the design of a pilot study aiming to assess the feasibility of offering targeted accommodations for students with special education needs
Abstract 2
By Bryan Maddox, Director for Digital Assessment Futures, Digital Education Futures Initiative, Hughes Hall, University of Cambridge, United Kingdom, and Bruno Zumbo, Distinguished University Scholar and Tier I Canada Research Chair (CRC) in Psychometrics and Measurement, University of British Columbia, Canada Towards Digital Inclusivity and an Unforeseen Opportunity of International Large-Scale Assessments
International Large-Scale Assessments (ILSAs) are increasingly able to improve test inclusivity by adopting universal design principles and incorporating an expanding set of accessibility tools. Those features of inclusive digital assessment let test takers adapt and personalised their test-taking experience to meet their accessibility needs. Those tools are already included in many digital interfaces as ‘universal design’ becomes an established norm in assessment and more comprehensive software/web applications. In this presentation, we discuss the benefits of improved accessibility in ILSAs and point toward related metrics and validity tests that can be used to monitor and assess their consequences. We anticipate that accessibility tools and features included as standard in ILSA designs should lead to improved test-taker experience for students with neurodiversity and disabilities – including those with undiagnosed conditions. Those equityrelated benefits should also improve test quality and student performance. We may also hope that improved accessibility could increase the number of students with neurodiversity and disabilities who are included in ILSAs. Both benefits should lead to a more reliable assessment and claims from test performance that reflect a more representative sample of test takers and performances. As we move towards improved inclusivity in ILSAs, we also have an excellent opportunity to evaluate and understand the impacts and efficacy of digital accessibility tools and designs in large-scale international studies. To conclude our presentation, we suggest suitable evidence, measures and validity practices that can be adopted. Moreover, we urge re-imagining and re-focusing digital ILSA studies and the rich multi-contextual array of data they provide as ideally suited to critically unpack the uniquenesses and commonalities of the test-taking experiences of diverse populations across educational settings as a window into the (mostly hidden) multiple diversities and intersectionalities of their target test taker populations.
Speakers:
- Moira Faul, Executive Director, NORRAG, Switzerland – Discussant
- Radhika Gorur, Director, Laboratory of International Assessment Studies, Deakin University, Australia – Chair
- Ava Guez, Analyst, OECD, France
- Bryan Maddox, Director for Digital Assessment Futures, Digital Education Futures Initiative, Hughes Hall, University of Cambridge, UK
- Elodie Persem, Head of Accessibility, Innovation and Research Unit, Department of Evaluation, Forecasting and Performance, French Ministry of National Education and Youth, France
Partners:
Laboratory of International Assessment Studies (lab) – Website
The Laboratory of International Assessment Studies is an inter-disciplinary network that brings together academic researchers, testing agencies, policy makers and the end users of international educational assessment data.
Deakin University’s Strategic Research and Innovation Centre, Centre for Research for Educational Impact (REDI) – Website
Deakin University’s Strategic Research Centre in Education, Research for Educational Impact (REDI) is a connected, collaborative, and vibrant research centre that delivers and translates high quality research into outcomes that are relevant and meaningful to their communities.