For Global Learning Metrics, Ask Cognitive Scientists by Helen Abadzi

By Helen Abadzi, College of Education, University of Texas at Arlington.

The December of 2016 brought worrisome messages to many Ministries of Education.  The 2015 PISA and TIMSS[1] scores were publicized.  Despite efforts, Latin American countries have low scores, as do the wealthy Gulf States.  Policymakers are unsure how to improve instruction for better outcomes.

The results’ release coincided with several regional workshops to contextualize the Sustainable Development Goal 4 (SDG 4).  The targets were developed after consulting with thousands of people who were related to education in some ways.  Agreement could only be reached in the broadest terms: e.g. “Percentage of children and young people in grades 2/3 achieving at least a minimum proficiency level in reading and mathematics, by sex.” [Global indicator 4.1.1]

Now comes the hard part. How to set valid benchmarks of “minimum proficiency” in reading and math for Yoruba, Arab or Lao children in grades 2/3 or justify comparisons with the Finns and Chinese?

Apparently no one knows. Thus far, the only emerging benchmarks are for more monitoring and consultations. One proposal is for subject experts to meet regionally, discuss how learning progress in each subject is understood in different contexts, and which assessment questions best capture learning progression.  Organization and funding will be needed, and any answers may be years and millions of dollars away.

In the meantime, policymakers despair over test scores. Some wish that this enthusiasm for testing spread into curricula, teacher training, textbooks, and instructional time. But the SDG consultations focus on lofty, big-picture strategies. The “translation” job is left for local actors to figure out.

Is it possible to shortcut the looming years of uncertainty and teach students efficiently?  Most certainly, if someone asks cognitive scientists. But readers must be warned that some answers run counter to popular beliefs.

The unifying principle for international comparisons is the DNA of homo sapiens.  Our brains think in similar ways, say the writings of Dr. William Huitt, Professor Emeritus of Valdosta State University in Georgia, USA.  Despite individual differences, humans process information similarly across countries and cultures. This bedrock of human commonalities could be used to build SDG benchmarks.

An obvious aid for benchmarking is “working memory,” would say Dr. William Klemm, senior professor of neuroscience at Texas A & M in the US and a prolific author on memory topics. Working memory contains what is in your mind right now and accurately maintains information that is necessary to work through problems.  But its capacity is very limited; it holds 4-7 items for maybe 12 seconds. This gives us just seconds in which to comprehend, calculate, make decisions.  We must be able to act rapidly and effortlessly.

Thus the holy grail for comparing performance is execution speed and automaticity. Global metrics could be developed for fluency in reading, math, science, various other skills.

Dr. Pierluigi Zoccolotti, professor at University of Rome psychology department, finds this a promising approach.  Words per minute are a rough but easy way to gauge whether students retain enough text in working memory in order to understand it. For that, students must know the language and recall meanings instantly. If they must exert mental effort, they use up extra milliseconds, lose their trail of thought and may answer incorrectly. Students taking TIMSS essentially engage in the battle of the milliseconds.

But how to speed up processing to the point of instant, effortless, automatic performance?  The nervous system demands practice. Classroom time and homework must offer sufficient practice opportunities, according to Dr. Craig Speelman, of Edith Cowan University in Australia. Improvement is rapid at first, then slows down. Learning curves have mathematical trends.  They could help predict the amounts of time children would attain various objectives and the number of instructional hours.

How else to speed up reading?  The size and spacing of letters read are critical, says Dr. Marialuisa Martelli, University of Rome.  Our visual system slows down if the letters are too small or too dense.  Optimal spacing and size will shave seconds off the reading time giving our working memory more time to consider a message before it disappears. Younger students are affected. This matters a lot in the younger ages, but adults often write texts for their own expert reading processes.  In principle, 4^th grade TIMSS scores in visually challenging scripts like Arabic or Thai could improve a bit by optimizing size and spacing.

Which mental functions would improve math performance in TIMSS and other tests?  Accurate estimations are important, says Dr. Daniel Ansari of U. Western Ontario.  Our brains have an exact and an approximate number system.  Again, practice is needed to automatize the components of the exact system, and also to fine-tune the approximate system.  This way, a student may get a fuzzy estimate and see which response alternatives fit.

But practice, memorization, automaticity are not on the menu of modern schools.  They sound like narrow, sterile, traditional, 19^th century education! Governments nowadays aim for creativity, discovery learning, and critical thinking.  Some fun projects are certainly important, but many activities gobble up teaching time to the detriment of practice. And neurocognitive research suggests that is putting the cart before the horse. Complex thinking is possible only when lots of prerequisite knowledge arrives instantly into working memory.  Students who were merely “exposed” to certain concepts may forget them entirely or retrieve them after a delay. Life events require instant retrieval.  To spend our money wisely, for example, we must calculate unit prices quickly, not three hours after leaving a store.

Why is speed a hard concept to digest? Perhaps because of incongruity. Memory functions are largely unconscious, so we are unaware of what we do. Please pay some attention and see how many seconds you took to read and understand this paragraph.  If you used more than about 10, you would get tired and give up.

The primacy of processing speed has big testing implications.  Most tests do not exactly measure knowledge; they only measure the knowledge that can be retrieved and applied within a few seconds. To perform, students should spend significant time memorizing and practicing sequences, which then pass as a single item into working memory. Top-scoring countries in TIMSS or PISA have historically emphasized practice.  Their students may not necessarily know more; instead, they may be able to retrieve more in fewer milliseconds.  Emphasis on fun activities worldwide may reduce practice time and eventually test scores. And governments that invested millions in modernizing methods may be at a loss to explain why.

Below are some of the ways to specify terms like “minimum proficiency.”  If cognitive scientists had been consulted about the indicators for SDG 4, they might have recommended some items like the ones below:

• Reading speed for the primary school grades to monitor the attainment of automaticity
• Calculating speed, magnitude processing tests, speed of single-digit operations
• Learning rates for students in key tasks

Neuroscientists may have much more to add.  For example, research to model structural changes in developing brains may help estimating learning rates for various subjects beyond basic skills. By 2030, neuroimaging devices may assess knowledge through learners’ brain states and thus bypass some problems of international comparability. This is already feasible for some tasks. Experimentally, reading automaticity is often assessed using neuroimaging methods in ways that bypass the complexities of different languages and scripts. Thus, comparative education may morph into comparative neuroscience.

To achieve the 2030 targets, governments and international organizations must engage cognitive scientists and neuroscientists. Right now neither side knows that the other exists. Education colleges keep their distance from psychology, and their degrees rarely include courses on memory. During the 2015 consultations about the SDGs, the input of the few learning specialists was averaged with that of generalists. The consequences are now obvious.

If we continue to disregard the DNA guidance, test scores will continue to be modest. By 2030 surely more policymakers will demand help from memory functions. But students, particularly low-income students, should not have to wait until then.

[1] Programme for International Student Assessment; Trends in Mathematics and Science Study.

Helen Abadzi is a researcher at the University of Texas at Arlington.  To improve the outcomes of education investments she regularly monitors research in cognitive psychology and neuroscience.  Her publications can be found at: uta.academia.edu/HelenAbadzi.

>> View all NORRAG Blogs on Learning Assessments

NORRAG (Network for International Policies and Cooperation in Education and Training) is an internationally recognised, multi-stakeholder network which has been seeking to inform, challenge and influence international education and training policies and cooperation for almost 30 years. NORRAG has more than 4,700 registered members worldwide and is free to join. Not a member? Join free here.