OECD Education Working Papers

This series is designed to make available to a wider readership selected studies drawing on the work of the OECD Directorate for Education. Authorship is usually collective, but principal writers are named. The papers are generally available only in their original language (English or French) with a short summary available in the other.

English, French

Data comparability in the teaching and learning international survey (TALIS) 2008 and 2013

This report focuses on data comparability of scale scores in the Teaching and Learning International Survey (TALIS). Valid cross-cultural comparisons of TALIS data are vital in providing input for evidence-based policy making and in promoting the equity and effectiveness of teacher policies. For this purpose, an investigation of data comparability is a prerequisite for any meaningful cross-cultural comparison. TALIS involves a large number of countries and economies, and has used rather strict conventional statistical methods to test comparability. Thus, many scales in TALIS do not reach the level of comparability that allows direct comparisons of scale scores. To facilitate the effective data analysis of TALIS and maximise its policy implications, this project: (1) uses a more flexible statistical method to test comparability, and (2) investigates the level and sources of scale data incomparability. With teacher and principal self-report data from the two rounds of TALIS (2008 and 2013), three studies are carried out to address these issues. Study 1 compares the conventional statistical method with more flexible Bayesian approximate invariance testing in scale data comparability testing. Study 2 investigates whether scale characteristics (e.g. scale length, item length, number of response options, and self-evaluative components) are associated with data comparability in principal and teacher scales. Finally, Study 3 examines the specific cultural variations that contribute to the lack of comparability. It tests the comparability of the Satisfaction with Current Work Environment scale (a key outcome construct in TALIS) between each participating country or economy with a pooled international average reference group. The paper concludes with a discussion of the implications for large-scale survey design and data analyses such as using more flexible psychometric method to test comparability and using fewer response options in items forming scales.


This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error