2. Studying teaching as a knowledge profession across education systems

A better understanding of teacher knowledge, in particular general pedagogical knowledge, and effective means of promoting it is of value for both education systems aiming at improving teacher quality and learning outcomes and for practitioners striving continuously to improve their teaching and the support they receive for their work. An international study of teacher knowledge can play a key role in sparking greater attention for the topic and making education systems, schools and teachers part of a global education community determined to strengthen knowledge-based and evidence-informed practices in schools.

In the 2024 cycle of the OECD Teaching and Learning International Survey (TALIS), a new optional TALIS module will explore teaching as a knowledge profession across education systems. The Teacher Knowledge Survey (TKS) assessment module will complement the TALIS self-report measures (e.g. teachers’ feeling of preparedness for different teaching tasks) with an objective assessment of the strength and weaknesses of teachers’ knowledge across countries, delivering a new, innovative theme and set of indicators for TALIS 2024.

The chapter provides an overview of the Teacher Knowledge Survey (TKS) assessment module. It describes the aims and design of the module as well as its conceptual underpinning and instruments.

The TALIS Teacher Knowledge Survey assessment module (TKS) aims to shed light on this “black box” of teaching: teachers’ professionalism and their knowledge base. More specifically, the module informs teacher policies and practice on:

  • whether teachers are sufficiently prepared for 21st century teaching, for example teaching diverse classrooms or using modern pedagogical approaches

  • the strengths and weaknesses of teachers’ current pedagogical knowledge base

  • the ways in which teacher education (including initial education, induction and continuing professional development) can be improved to ensure a robust knowledge base in the profession in line with national policy priorities.

Box 2.1 outlines further the value of the TKS assessment module for policy-making, including examples of key policy questions addressed by the survey.

The TKS assessment module is designed to extend and complement the existing OECD TALIS and Programme for International Student Assessment (PISA) surveys. By providing an assessment of teachers’ knowledge, it goes beyond self-reports. At the same time, it does not report the results of individual teachers; it aims to report results for all teachers or specific groups of teachers in a country. The TKS therefore positions itself as a useful research and policy tool for system level analysis, which helps identify pertinent challenges to teacher professionalism. It is important to emphasise that the solutions to identified challenges do not necessarily come in a top-down fashion. Rather, the intention is to highlight the importance of empowering teachers to take charge of their own professional knowledge base. Box 2.2 explains in detail the value of the TKS assessment for the profession.

The module focuses on teachers’ general pedagogical knowledge. A recent systematic review of the international research evidence has shown that general pedagogical knowledge matters for the quality of teaching and student outcomes, indicating that more knowledgeable teachers achieve a three-month additional progress for students (Ulferts, 2019[1]). The module is designed to better understand this specialised knowledge of teachers, and to explore whether this knowledge is up to date with current research as well as adapted to 21st century societal needs. It also looks at how this relates to learning opportunities in teacher education and, equally, to professional competence.

Target participants of the TALIS Teacher Knowledge Survey assessment module are teachers in ISCED 2 (lower-secondary) level varying in teaching experience. This will provide valuable insights into the evolution of teachers’ knowledge with growing teaching experience. The optional module will sample teachers from the same school, aiming for representative samples that allow studying the role of the school context for teachers’ knowledge.

Originally, the TKS was developed as a stand-alone study in the OECD’s Centre for Education Research in Innovation (CERI). The frameworks and instruments designed for the study were piloted in 2016 in five countries (Estonia, Greece, Hungary, Israel and the Slovak Republic) (Sonmark et al., 2017[2]). To integrate the survey optimally into TALIS, frameworks and instruments will be amended, capitalising on existing synergies and capturing more of the knowledge and skills that teachers need for mastering challenges in today’s and future classrooms.

The TKS assessment module aims to contribute to a better understanding of a main pillar of teachers’ professionalism: teachers’ pedagogical knowledge (OECD, 2019[3]). As explained in Chapter 1, the development of the module’s conceptual underpinning can build on the extensive conceptual and empirical work in the area of teacher knowledge conducted in CERI.

With its publication on “Pedagogical Knowledge and the Changing Nature of the Teaching Profession”, CERI provided a strong conceptual basis for an international assessment of teacher knowledge (Guerriero, 2017[4]). The book – with contributions of several leading experts in the field – provides an extensive summary of the existing conceptual and empirical work on teachers’ pedagogical knowledge. The overview of empirical studies included in the book showed the value of a knowledge assessment for improving teacher policy and practice (as summarised in Box 2.1 and Box 2.2).

Drawing on the conceptual and empirical work, Guerrieiro developed a conceptual framework of teachers’ professional competences where competence is defined as “the ability to meet complex demands in a given context by mobilising various psychosocial (cognitive, functional, personal and ethical) resources” (2017, p. 261[4]). As such, teachers’ knowledge is part of their competence (see Figure 2.1).

Teachers create and design learning environments by drawing on their knowledge on effective teaching and learning, as well as teaching-related beliefs, and other motivational and affective competences. They base instructional decisions and their professional judgement in the classroom on such knowledge and competences. Professional judgement guides the subsequent teaching approaches, which include curriculum and lesson planning, selecting and applying sets of teaching methods, ways of classroom management, student assessment etc. This then influences both cognitive and socio-emotional aspects of student learning (Ulferts, 2019[1]).

In light of rapidly changing technology and societies, as well as the expanding research on teaching and learning, teachers must constantly update their knowledge and teaching skills. The nature of teachers’ pedagogical knowledge is shaped by the educational context and the scope and quality of their learning opportunities. Research and experience continuously feed into the knowledge base that is transferred to and co-constructed by teachers through individual and collective learning. Teachers’ learning opportunities shape not only their knowledge of the subject(s) they teach and pedagogy in general, but also other competences. Though displayed as a series of linear relationships between the elements of the model, the teaching and learning process is highly complex and the different pieces are in continuous and dynamic interaction with one another. For example, during instruction teachers observe and reflect on student learning, which is part of their informal learning.

The TALIS 2024 conceptual framework will retain essential themes and a blend of indicators from previous TALIS cycles to allow for trend analysis, as done for TALIS 2018, but will be expanded to cover issues relevant to the study of teaching as a knowledge profession (Ainley and Carstens, 2018[5]).

In light of the discrepancy observed for self-rated and assessed knowledge in previous research (Baier and Kunter, 2020[6]; Drummond and Sweeney, 2017[7]; Maderick et al., 2016[8]; König, Kaiser and Felbrich, 2012[9]), it seems crucial to include an assessment of teacher knowledge. This helps gaining a more sophisticated understanding of teaching as a knowledge profession as obtained in previous cycles of TALIS through self-reports of teachers. Previous studies such as the Teacher Education and Development Study in Mathematics (TEDS-M) (Tatto, 2013[10]), the Service Delivery Indicators (SDI) (Bold et al., 2017[11]) and the TKS pilot study (Sonmark et al., 2017[2]) have shown the feasibility of using an assessment to obtain international comparable data on teacher knowledge (see Chapter 1 for a discussion of these studies).

To be valid and informative, an assessment of teacher knowledge needs to cover the content areas and types of teacher knowledge relevant for effective teaching and learning. Drawing on an extensive review of available assessment of teacher knowledge, a framework for assessing teacher knowledge across countries was developed in CERI (the CERI TKS assessment framework, see Figure 2.2).

The framework postulates three core dimensions of general pedagogical knowledge – instruction, learning and assessment –, each of which is further specified into two sub-dimensions (Sonmark et al., 2017[2]):

  1. 1. Teaching methods and lesson planning (sub-dimension of instruction): Knowing how to productively use instructional time and teaching methods (e.g. direct instruction, discovery learning), when and how to apply methods to foster students’ conceptual understanding and learning, how to structure learning objectives, lessons, curricular units and assessment.

  2. 2. Classroom management (sub-dimension of instruction): Knowing how to maximise instructional time through classroom monitoring, simultaneous handling of multiple classroom events, and appropriate pacing of lessons, knowledge of how to maintain student attention through clear directions and transparent rules.

  3. 3. Sciences of learning and development (sub-dimension of learning): Knowledge of learning processes and individual differences (including learning strategies, the impact of prior knowledge, memory and information processing), adaptive teaching strategies and the dynamics of individual and group learning.

  4. 4. Affective-motivational dispositions (sub-dimension of learning): Knowledge of emotional and motivational processes in learning (e.g. achievement motivation) and strategies to motivate and engage individual students and the whole group.

  5. 5. Evaluation and diagnostic procedures (sub-dimension of assessment): Knowledge of different forms and purposes of formative and summative classroom assessments, and their impact on student motivation and learning, understanding what matters for the quality of evaluations.

  6. 6. Data use and research literacy (sub-dimension of assessment): Knowledge of how to use research and data to inform the teaching and learning process, understanding how to interpret and assess the quality as well as generalisability of research.

The three knowledge areas of instruction, learning and assessment are all relevant to high quality teaching and student outcomes, as shown in an international review and meta-analysis (Ulferts, 2019[1]).

As discussed in Chapter 1, teachers need general pedagogical knowledge to tackle emerging challenges in today’s classrooms. More than ever, teachers need to know how to foster 21st century skills (e.g. critical thinking and collaboration), how to meet the diverse needs and backgrounds of their students and how to design effective online teaching and learning environments (König et al., 2017[12]; Wasonga, 2005[13]; Valanidou and Jones, 2012[14]; Schleicher, 2014[15]; Mishra and Koehler, 2006[16]). Tackling such challenges requires specific knowledge in instruction, assessment and learning. For example, managing diversity requires knowledge about how to tailor pedagogies and instructional methods to various student needs and backgrounds (instruction). It also requires knowledge about the sources and implications of individual differences in learning, motivation, behaviours and thinking (learning) as well as diagnostic procedures and tools suitable for different student groups and criteria for evaluating their suitability (e.g. fairness and sensitivity of different evaluation tools) (assessment). The framework therefore considers teachers’ knowledge for fostering 21st century skills, managing diversity in classrooms and using digital  technology for teaching as transversal knowledge areas across the three main areas.

Teachers also need to be able to apply their knowledge in the context of their classrooms (Ulferts, 2019[1]). Accordingly, the framework considers both types of knowledge: theoretical/scientific knowledge (e.g.  Formal knowledge of concepts and theories), as well as practice-based knowledge and knowledge-based skills (e.g. the ability to apply professional judgement in a given classroom situation and knowledge-based decision making).

An in-depth, updated review of the research literature on teacher professional competence, GPK assessment and national teaching frameworks will determine the cultural validity and coverage of the framework, and identify key areas for further possible development. The refinement and validation will also include a critical review by cross-country and country experts on general pedagogical knowledge and teacher assessment.

This publication aims to contribute to this challenging endeavour by summarising the scientific literature on a particular key topic relating to the study of teacher knowledge. The following chapters also entail specific suggestions for surveys that study teacher knowledge across education systems:

  • Chapters 3, 4 and 6 of this publication, for example, provide ideas for capturing teacher knowledge about the effective use of technology in teaching and inclusive teaching in diverse classrooms and for obtaining information on teachers’ opportunities to learn about general pedagogy.

  • Chapter 5 includes ideas for improving the measurement of practice-based knowledge and knowledge-based skills through revising the item and response format as well as the approach to scoring items.

  • Chapter 7 proposes an innovative testing design for studying teacher knowledge across countries (Multidimensional Adaptive Testing, MAT). These designs help increase the assessment precision while reducing the survey length and maximising the test-taking motivation of participating teachers.

Developing an assessment that delivers rich and comparable information for understanding teaching as a knowledge profession is as challenging as it is important. It is important to cover all important areas and types of knowledge for 21st century teaching outlined above. Sufficient attention must be paid to the most pressing challenges in today’s classrooms, such as diversity and technology and a use of knowledge that fits the pedagogical context. At the same time, it is vital to limit the length of the survey to avoid survey fatigue and teachers’ dropping out of the survey. The development of the assessment for the TKS assessment module can draw on the item bank created for the TKS in CERI (see Table 2.1 for sample items).

The item bank covers the areas specified in the framework (see Figure 2.2) with more than 200 items, which were validated from experts and country representatives from across the OECD. Over 50 items of the bank were additionally piloted in five countries from April to June 2016 (Sonmark et al., 2017[2]). The item bank consists of binary items: either simple multiple choice (MC; e.g. a question with four response options where one is correct and three incorrect) or complex multiple choice items (CMC; e.g. a question with four or more response options, where each can be answered with “right” vs. “wrong” or “suited” vs. “not suited”).

The development of the assessment for the TALIS TKS assessment module will draw on the items developed in CERI. Yet, significant changes are planned regarding content and also the item and response format as well as the approach used for scoring items. For example, items will be modified and newly developed to allow for an optimal coverage across all knowledge areas and types relevant for 21st century teaching. This means including more items on the transversal knowledge areas (i.e. digital technology, 21st century skills and diversity) and items assessing teachers’ ability to apply knowledge in the context of particular teaching situations and contexts (i.e. practice-based knowledge and situation-based skills). For expanding on the transversal knowledge areas, a review of existing assessments in these areas is helpful, for example a review of instruments that assess diversity-related pedagogical knowledge (see Table 2.2).

The refinement of the assessment can draw on these instruments as well as on the few objective assessments of teachers’ technology-related pedagogical knowledge that have been developed [e.g. (Baier and Kunter, 2020[6]; Drummond and Sweeney, 2017[7]; Maderick et al., 2016[8])]. Most studies use, however, self-rating via qustestionnaires. Table 2.2 also shows that most objective assessments focus on diversity-related knowledge in certain knowledge areas or cover knowledge about specific student characteristics such as attention-deficit/hyperactivity disorder (ADHD) and external behaviour problems.

To assess teachers’ ability to apply their knowledge in context, so-called contextualised or situation-based items will be used, that confront teachers with typical and challenging classroom situations, for example short written descriptions of classroom situations (“text vignettes”). Chapter 5 provides a detailed description of this approach and provides examples of contextualised items for the assessment of teacher knowledge.

When measuring teachers’ application of knowledge, it is also important to acknowledge that teaching in real classroom settings is often not about making the right choice but the most adequate one for a given situation. Teaching approaches, for instance, may not be correct or incorrect but vary in effectiveness, depending on the situational context. It is, therefore, necessary to also innovate the response format and approach to scaling of the assessment developed in CERI. As displayed in Table 2.1, the original assessment used multiple choice items that require one correct answer that is clearly justifiable based on research evidence. While this can be suited to assess theoretical/scientific knowledge, the revised assessment will include items with Likert scales (e.g. asking teachers to judge the effectiveness or utility of certain teaching approaches on a continuum from “not at all effective” to “very effective” etc.). Such methods allow for mapping the spectrum of options for teaching in the classroom and measuring teachers’ practice-based knowledge and knowledge-based skills. In addition, it seems promising to consider an approach to scaling such items that compares teachers’ answers to the answers of a designated board of experts. Chapter 5 explains the use of Likert scales and expert ratings as a scoring approach for an assessment of teacher knowledge.

As displayed in Figure 2.1, the conceptual and empirical work conducted in CERI underlines that teacher knowledge needs to be understood in the context of national policies and the education system, as well as the broader context of teacher professionalism, such as:

  • teachers’ opportunities to learn (OTL) pedagogy (in initial teacher education, induction and continuous professional development)

  • teachers’ approaches to teaching and their instruction in the classroom

  • motivational-affective characteristics of teachers (e.g. self-efficacy, self-responsibility).

In line with the approach of the original Teacher Knowledge Survey, the TKS assessment module will include indicators and constructs that are necessary to provide the context for the assessment results. Since the module will be embedded within the 2024 cycle of TALIS, indicators and constructs from TALIS can be used to provide context. To the extent necessary, other indicators and constructs necessary for providing context information will be added to the module.

For designing the questionnaire, the module can draw on the battery of questionnaire instruments. Table 2.3 provides an overview of the instruments. A detailed description of the instruments including results of the pilot study can be found in Sonmark et al. (2017[2]). Additionally, the expert chapters make suggestions for additional scales to obtain context information and adjusting the TALIS 2018 questionnaire to the new theme on teacher knowledge, for example items for obtaining more detailed insights into teachers’ opportunities to learn about general pedagogy (see Chapter 6).

Teaching involves the design of effective online, offline and hybrid learning environments in increasingly diverse classrooms. Teachers also have an important role in guiding and shaping students’ use of digital tools and optimising the educational benefits of their digital experiences. They are agents of inclusive, equitable education and ambassadors of embracing diversity as an enriching element of our societies.

To fulfil this education mission teachers need to be experts of teaching and learning, who base their practice on a specialised, integrated and updated body of knowledge. However, there is a great need for a better understanding of the specialised knowledge and skills that teaching in the 21st century requires. This is the ambition for the next cycle of the OECD Teaching and Learning International Survey (TALIS) and its new Teacher Knowledge Survey (TKS) assessment module. The module will explore teaching as a knowledge profession across education systems and provide international comparable data on teachers’ general pedagogical knowledge.

References

[5] Ainley, J. and R. Carstens (2018), “Teaching and Learning International Survey (TALIS) 2018 conceptual framework”, OECD Education Working Papers, Vol. 187, pp. 1-108, https://doi.org/10.1787/799337c2-en.

[6] Baier, F. and M. Kunter (2020), “Construction and validation of a test to assess (pre-service) teachers’ technological pedagogical knowledge (TPK)”, Studies in Educational Evaluation, Vol. 67, p. 100936, http://dx.doi.org/10.1016/j.stueduc.2020.100936.

[11] Bold, T. et al. (2017), “What do teachers know and do? Does it matter? Evidence from primary schools in Africa”, Policy Research Working Paper, Vol. 7956, pp. 1-35, http://hdl.handle.net/10986/25964 (accessed on 20 July 2021).

[17] Brühwiler, C. and F. Vogt (2020), “Adaptive teaching competency. Effects on quality of instruction and learning outcomes”, Journal for educational research, Vol. 12/1, pp. 119-142, http://dx.doi.org/10.25656/01:19121.

[24] Cowan, J. and D. Goldhaber (2014), Assessing the Relationship between Teacher Performance on Washington State’s ProTeach Portfolio and Student Test Performance, University of Washington, Seattle, WA.

[7] Drummond, A. and T. Sweeney (2017), “Can an objective measure of technological pedagogical content knowledge (TPACK) supplement existing TPACK measures?”, British Journal of Educational Technology, Vol. 48/4, pp. 928-939, http://dx.doi.org/10.1111/bjet.12473.

[23] ETS (2013), Study Guide for Principles of Learning and Teaching: Grades 5-9 and 7-12., Princeton, New Jersey: Educational Testing Service.

[20] Gottfried, K. et al. (2021), “Adaptiver Umgang mit externalisierenden Verhaltensproblemen (Adaptive dealing with external behavior problems)”, Swiss Journal of Educational Research, Vol. 43/2, pp. 260-272, http://dx.doi.org/10.24452/SJER.43.2.6.

[4] Guerriero, S. (2017), Pedagogical Knowledge and the Changing Nature of the Teaching Profession, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264270695-de.

[19] König, J. et al. (2011), “General Pedagogical Knowledge of Future Middle School Teachers: On the Complex Ecology of Teacher Education in the United States, Germany, and Taiwan:”, Journal of Teacher Education, Vol. 62/2, pp. 188-201, http://dx.doi.org/10.1177/0022487110388664.

[22] König, J. et al. (2020), “General pedagogical knowledge, pedagogical adaptivity in written lesson plans, and instructional practice among preservice teachers”, Journal of Curriculum Studies, Vol. 52/6, pp. 800-822, http://dx.doi.org/10.1080/00220272.2020.1752804.

[12] König, J. et al. (2017), “Erfassung von pädagogischem Wissen für inklusiven Unterricht bei angehenden Lehrkräften (Measurement of pedagogical knowledge for inclusive teaching of future teachers)”, Unterrichtswissenschaft, Vol. 45/4, pp. 223-242.

[9] König, J., G. Kaiser and A. Felbrich (2012), “Spiegelt sich pädagogisches Wissen in den Kompetenzselbsteinschätzungen angehender Lehrkräfte? (Is pedagogical knowledge reflected in the competence-related self-assessments of future teachers?)”, Zeitschrift für Pädagogik, Vol. 58, pp. 476-491.

[8] Maderick, J. et al. (2016), “Preservice teachers and self-assessing digital competence.”, Journal of Educational Computing Research, Vol. 54/3, pp. 326-351, http://dx.doi.org/10.1177/0735633115620432.

[16] Mishra, P. and M. Koehler (2006), “Technological pedagogical content knowledge: A framework for teacher knowledge”, Teachers College Record, Vol. 108/6, pp. 1017-1054, http://dx.doi.org/10.1111/j.1467-9620.2006.00684.x.

[3] OECD (2019), TALIS 2018 Results (Volume I): Teachers and School Leaders as Lifelong Learners, TALIS, OECD Publishing, Paris, https://doi.org/10.1787/1d0bc92a-en.

[15] Schleicher, A. (2014), Equity, Excellence and Inclusiveness in Education: Policy Lessons from Around the World, International Summit on the Teaching Profession, International Summit on the Teaching Profession, OECD Publishing, Paris, https://doi.org/10.1787/9789264214033-en (accessed on 25 April 2021).

[21] Sciutto, M., M. Terjesen and A. Bender Frank (2000), “Teachers’ knowledge and misperceptions of Attention-Deficit/hyperactivity disorder”, Psychology in the Schools, Vol. 37/2, pp. 115-122, http://dx.doi.org/10.1002/(SICI)1520-6807(200003)37:2<115::AID-PITS3>3.0.CO;2-5.

[2] Sonmark, K. et al. (2017), “Understanding teachers’ pedagogical knowledge: Report on an international pilot study”, OECD Education Working Papers, Vol. 159, pp. 1-150, http://dx.doi.org/10.1787/43332ebd-en.

[10] Tatto, M. (ed.) (2013), The Teacher Education and Development Study in Mathematics (TEDS-M) - Policy, Practice, and Readiness to Teach Primary and Secondary Mathematics in 17 Countries. Technical report, IEA , Amsterdam, https://www.iea.nl/publications/technical-reports/teacher-education-and-development-study-mathematics-teds-m-technical.

[1] Ulferts, H. (2019), “The relevance of general pedagogical knowledge for successful teaching: Systematic review and meta-analysis of the international evidence from primary to tertiary education”, OECD Education Working Papers, No. 212, OECD Publishing, Paris, https://dx.doi.org/10.1787/ede8feb6-en.

[14] Valanidou, A. and J. Jones (2012), “Teaching greek in multicultural, primary classrooms: Teachers’ perceptions of the challenges in four greek-cypriot primary schools”, Cyprus Review, Vol. 24/1, pp. 119-145.

[18] Voss, T., M. Kunter and J. Baumert (2011), “Assessing Teacher Candidates’ General Pedagogical/Psychological Knowledge: Test Construction and Validation”, Journal of Educational Psychology, Vol. 103/4, pp. 952-969, http://dx.doi.org/10.1037/A0025125.

[13] Wasonga, T. (2005), “Multicultural education knowledgebase, attitudes and preparedness for diversity”, International Journal of Educational Management, Vol. 19/1, pp. 67-74.

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

© OECD 2021

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.