If Data is Not Wisdom, then Non-Data Certainly is Not
Karina Veal, Asian Development Bank.
Where hide the wise answers to questions vexing TVET policy makers and practitioners in developing countries today? They ask, for example: will benefits outweigh costs of building a National Qualifications Framework. What works best to engage the most marginalized youth in skills training? How can one measure learning outcomes from skills? Plus the perennial, plaintive query, can anyone really say (beyond that it is a nice idea) more about how to actually, practically, efficiently introduce a broad based system of recognition of skill for the informal sector?
If your curiosity is restricted to the development of national frameworks for TVET qualifications you are in luck, thanks to the ILO’s 16 country study (ILO, 2010) of their establishment and effectiveness published in 2010 (1). It is a good study, but a lonely study. Where are the reports to help provide answers to other common issues of interest? An authoritative international comparative study on approaches to recognition of skill, or a meta-analysis on successful strategies for inclusive TVET, perhaps?
One may have hoped for some answers from the suite of major reports on skills released in 2012. Certainly there is some very useful teasing out of terminology (2), some good overall conceptualization and some comprehensive understanding of the prevalence of particular types of skills across the globe. But a focus on the issues and, importantly, on perspectives from the developing world is slender, and there is insufficient transformation of knowledge into wisdom, stemming from an insufficient base of data and information.
Yet wisdom is urgently needed. For example the shift in interest towards quality learning outcomes in TVET will be notoriously difficult to measure. Does one take employment (or wage) outcomes as the proxy for relevance and quality? Does one measure (somehow) quality of teaching and curriculum inputs and make assumptions on quality learning outcomes? Is there even any definitive research on the factors that lead to learning achievement in TVET? For learning achievement in schools there is John Hattie’s meta-analysis of over 800 meta-analyses of student learning (Hattie, 2008). He finds that two-way student teacher feedback repeatedly emerges as the top influencer of learner achievement in schools. How similar might be the findings in the context of the type of learning undertaken by young adults in vocational courses? The idea of doing a similar meta- meta-analyses in TVET, with its dearth of basic research studies, would not be possible. A simpler meta-analysis on the topic, anyone?
Taking ‘easier’ questions of access to, and participation in TVET, do we have a sufficient base of analysis (beyond straight numbers) even there? Remember the fantastically useful ‘Education Inequality Tree’ in the 2010 UNESCO EFA Global Monitoring Report (GMR)? It took school participation data and showed that the average 8 years of education in Turkey comprised nearly 10 years for rich boys and only 3 years for poor Kurdish girls. For Nigeria the range was 10.3 years for rich rural boys down to a startling 0.3 years (i.e., one term of school) for a poor Hausa girl. This kind of information can be extremely valuable for government policy makers and the development partners who give them support. Do we have a similar ‘TVET Inequality Tree’ in the 2012 UNESCO EFA GMR with its focus on skills? No, we don’t. Not, presumably because the authors didn’t care but because sufficient base data for that level of analysis is not available.
Taken together, the reports reflect the still emerging maturity of TVET research. There are still many gaps in the foundations of good basic research (e.g. lack of theoretical agreement on even core terms, lack of solid base of international data) and large gaps in applied research (e.g. large scale studies to find answers to pressing problems).
If you had hoped, like me, that the swathe of new global reports on skills released in 2012 (see NORRAG NEWS 48) might provide well researched knowledge on key issues of strong interest to developing countries, you will remain disappointed.
I will finish by referring to the beginning of this short reflection. It’s surely difficult to write wise, and useful, reports without a foundation of adequate evidence based data, information or knowledge.
End notes:
[1] The answer is: perhaps, but you may find other aspects make a bigger positive impact first.
[2] World Bank’s terrific trio of ‘jobs need skills, jobs pull skills, and jobs build skills’ for example and ILOs ‘how skills are developed, activated and utilized
Karina Veal is a Senior Social Sector Specialist at the Asian Development Bank, Manila. Email: kveal@adb.org
This blog post also appears in NORRAG NEWS 48, 2012: The Year of Global Reports on TVET, Skills & Jobs Consensus or diversity? (April 2013), available free online at www.norrag.org