copy the linklink copied!Chapter 3. Creating a highly qualified teaching workforce

This chapter looks at how Georgia’s teacher appraisal system evaluates the country’s teachers and supports them to develop professionally. Several factors are preventing Georgia from creating a more modern and professional teacher workforce. Its teacher professional development scheme is based upon acquiring credits, the accumulation of which do not necessarily signal good teaching practice. Furthermore, most teachers still remain at the lowest level of the scheme, reflecting a lack of support to improve themselves. Finally, over a quarter of Georgia’s teachers are past retirement age, which contributes to a less motivated teaching population overall. To address this situation, Georgia should revise its teacher professional development scheme to focus more on demonstrating effective teaching practices and then use the scheme to support teachers to reach basic minimum standards of competence. To improve incoming teachers’ skills, initial teacher preparation standards should be strengthened, and older teachers will have to be supported in exiting the profession in order to make space for incoming teachers.

    

The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.

copy the linklink copied!Introduction

Teacher appraisal can be a strong lever for modernising and improving teaching and learning. By providing teachers with regular feedback and setting high standards for teaching quality, appraisal encourages teachers to continually adapt and improve their practice. Georgia clearly recognises the importance of appraisal for updating and strengthening teaching, as reflected in the recently implemented performance career system. The adoption of the teacher professional development scheme coincides with efforts to shift instruction towards a more student-centred approach that is focused on the development of complex competencies.

However, the current system has so far had little impact in terms of professionalising teaching or encouraging teachers to adopt newer, more effective teaching techniques. This reflects the system’s design, in which moving up the teacher career path is contingent on form-filling and acquiring credits. Accumulating credits does not necessarily reward the most effective educators and distracts teachers from their central focus on student learning. While recognising the positive features of the system, this chapter recommends how its design can be improved. Notably, by basing promotion decisions on teaching quality.

The chapter also addresses the need for teachers to receive more regular guidance to develop professionally. Currently, undergoing professional development is not mandatory and many teachers have gone years without receiving formal support to improve their teaching. Georgia needs to systematise its provision of teacher support by designing a regular appraisal system that is linked to professional development opportunities. This is especially vital in Georgia, where many teachers are at the lowest level of the professional development scheme.

Along with improving in-service professional development, Georgia can improve the quality of incoming teachers by strengthening its initial teacher education programmes. Historically, entrance requirements into these programmes have been low, which affected the calibre of persons who became teachers. Raising entrance requirements into teacher education programmes and increasing the level of support that beginning teachers receive would help improve the overall quality of incoming teachers.

Finally, Georgia will need to take measures to attract talented new teachers into the workforce. Georgia’s teaching population is the oldest out of any country that participated in the 2018 Teacher and Learning International Survey (TALIS). This review recommends that Georgia establish a retirement age for teachers, require existing teachers to become certified and support those who cannot, or who do not wish to, as they exit the workforce. The ministry can then take measures to improve the attractiveness of teaching to entice younger, capable persons to fill open positions and help modernise the teaching profession in Georgia.

copy the linklink copied!Key features of an effective appraisal system

Teacher appraisal refers to how teachers are assessed and given feedback on their performance and competencies (see Figure 3.1). An effective appraisal system focuses on how well teachers are supporting the learning of all students. It provides teachers with incentives and support to continually develop their teaching competencies and assume roles that contribute to the development of the teaching profession overall. When used in this way, appraisal can positively influence teachers’ attitudes, motivation and classroom practices and, through this, help to improve students’ learning outcomes (OECD, 2013[1]). Countries combine different types of appraisal at different moments of a teacher’s career to inform on-going learning, professional development and career progression (see Figure 3.1).

copy the linklink copied!
Figure 3.1. Types of teacher appraisal
Figure 3.1. Types of teacher appraisal

Teacher standards

Standards provide a common reference point for teacher policies, including appraisal

A growing number of OECD countries have developed teaching standards to inform teacher policy and practices. Teaching standards describe what “good” teaching is and how it is demonstrated. They are used to align key teacher policies such as initial teacher training, certification and re-certification, career progression, professional development and teacher appraisal. Teacher standards are an essential part of an effective teacher appraisal system as they provide a common reference point for both teachers and evaluators that establish clear expectations, encourage consistent judgement and focus appraisal on the key aspects of teaching that matter for learning (Santiago et al., 2013[2]).

Teaching standards typically include a general profile setting out expected teacher competencies. Some also include specialised profiles for particular types of teachers, such as more experienced teachers as part of a differentiated career path, or teachers of different educational levels or subjects (Santiago et al., 2013[2]). Effective teaching standards are aligned with national education priorities, learning standards and curricula to ensure that teachers develop teaching competencies that will support national learning goals (Louden, 2000[3]). They are also grounded in national and international evidence of the types of teaching approaches that have been shown to have the greatest impact on student learning.

Initial teacher preparation

Select candidates with strong academic skills and motivation to teach

Selecting teacher candidates with strong academic skills and the motivation to teach is key to instilling high quality learning and teaching in schools. Selection refers to how teachers are recruited both into initial teacher education programmes and into the teaching profession. A recognised feature of the world’s highest-performing education systems is setting a high bar for entry into initial teacher education, with places accorded only to the most able school graduates (Barber and Mourshed, 2007[4]). A common method of setting entrance requirements is by establishing a minimum threshold that candidates must achieve on upper secondary completion or tertiary entrance examinations.

Set a rigorous certification process at the end of teacher education to make sure to select qualified new teachers

Initial certification at the end of teacher education serves as a gatekeeper to ensure that those who enter the profession have acquired the basic competencies required for good teaching. In most OECD countries, initial certification requires successful completion of teacher education programmes, which provide at least a bachelor’s level qualification, and increasingly a qualification at master’s level. However, many OECD countries also require that prospective teachers pass an external licensing examination, which can help maintain fairness and consistency in selection and guarantee that basic minimum standards are met (OECD, 2014[5]). Licensing is particularly important in countries where teaching is a “career-based” public service, lifetime employment is largely guaranteed and where quality assurance in the tertiary sector is weak. Since an examination cannot recognise all the attributes that are important for teaching, countries with examinations often complement them with other forms of assessment such as interviews, which can capture motivation and socio-emotional skills. Finally, in most countries full certification as a teacher is dependent on successfully passing a probation appraisal, during which teachers are able to demonstrate their teaching skills in the classroom.

Types of teacher appraisal

A probation period and appraisal provides new teachers with essential support in their first year(s) on the job

The first years of teaching are critical to building the foundations of good teaching practices. Most OECD countries set probation periods that combine mentorship, classroom observations and formative feedback to provide new teachers with support to develop their teaching practice (OECD, 2014[5]). Regular appraisal and feedback to teachers are key components of the probation period. In countries where the latter are not part of the probation period, retention rates of new teachers are often lower (OECD, 2017[6]).

In about half of OECD countries, successfully passing an appraisal at the end of the probation period is a requirement to become a fully certified teacher (see Figure 3.2). Probation appraisals help ground decisions about full certification on an evaluation of all the key competencies for teaching. Appraisal by the school leadership team, the school board or the teacher’s mentor are the most common approaches to awarding full certification. These in-school actors have the opportunity to observe a trainee teacher’s practice throughout the year, which provides them with a fuller picture of a trainee’s readiness to enter the profession. In some countries, the probation appraisal also includes an external evaluator (OECD, 2013[1]). An external dimension for the probation appraisal is particularly important in education systems where the school leadership might lack capacity to make a valid and objective judgement about a teacher’s competencies.

Regularly appraising teachers provides meaningful feedback and informs classroom practices

Regularly appraising teachers to provide feedback on their professional practices is a common component of teacher appraisal in the majority of OECD countries (see Figure 3.2). Regular appraisal is primarily developmental and identifies a teacher’s strengths and learning needs. It draws on information from classroom observations to provide specific feedback to support teachers’ continued professional growth (OECD, 2013[1]). Some OECD countries also use teachers’ self-evaluation and their teaching portfolio as part of regular appraisal, as they encourage self-reflection and provide a range of evidence on a teacher’s practices and needs for professional development (OECD, 2015[7]).

In most OECD countries, the regular appraisal of teachers is led by the school leadership team because they can develop a more accurate understanding of a teacher’s practice based on multiple observations throughout the year. Since the leadership team is familiar to the teacher, this is also likely to create a more informal setting for appraisal to encourage open and honest feedback (OECD, 2013[1]).

The formative value of regular appraisal is strengthened when the findings are used to inform decisions on teachers’ professional development. In many countries, the school leader or leadership team is expected to work with teachers to establish individualised development plans, which define the type of activities a teacher will undertake in order to improve specific areas of practice. Such plans are most effective when they connect individual goals with school priorities for teacher development, as this helps to encourage teacher collaboration and peer learning (Goe, Biggers and Croft, 2012[8]).

Appraisal for promotion informs teachers’ career progression and rewards performance

An increasing number of OECD countries are setting merit-based career structures to encourage teachers to develop higher levels of competence and take on differentiated teaching roles. External appraisal is often used in countries that introduce a merit-based career structure to inform teacher career advancement. This appraisal is often voluntary, at the request of a teacher, and is led by an evaluator that is external to the teacher’s school to maintain integrity and transparency. This type of appraisal evaluates teachers’ capacity to take on further responsibilities and rewards effective teaching (OECD, 2013[1]). Recognising and rewarding good teaching is important to motivating a teaching workforce. It also helps to make the best use of teachers’ talent, by providing opportunities for career growth and retaining talented teachers.

Some education systems require teachers to go through an appraisal process to be re-certified periodically. Re-certification ensures that teachers are periodically appraised by an external body even if they are not applying for promotion, which helps a country uphold minimum teaching standards (Kitchen et al., 2017[9]).

copy the linklink copied!
Figure 3.2. Types of teacher appraisals in OECD countries (2015)
Figure 3.2. Types of teacher appraisals in OECD countries (2015)

Source: (OECD, 2015[7]), Education at a Glance 2015: OECD Indicators, OECD Publishing, Paris, https://dx.doi.org/10.1787/eag-2015-en.

copy the linklink copied!The teaching profession in Georgia

Since 2007, Georgia has implemented successive changes to teacher policy to professionalise and modernise the teaching workforce (Figure 3.3). These changes have included increasing the qualification requirements to become a teacher and introducing a merit-based career structure. Many of these changes bring Georgia more into line with practices in OECD countries. However, in many cases the impact of the reforms has been diminished by a lack of follow-through or policy reversals. For example, the requirement that new teachers complete a one-year induction has not been implemented. Similarly, the requirement that all teachers reach senior status by 2014 never came into force.

copy the linklink copied!
Figure 3.3. Timeline of teacher policy reforms
Figure 3.3. Timeline of teacher policy reforms

Sources: World Bank (2014[10]), SABER Country Report Georgia – Teachers, World Bank Group, http://documents.worldbank.org/curated/en/788461468198234275/pdf/105634-WP-ADD-SERIES-PUBLIC-SABER-Teachers-Georgia-CR-Final-2014.pdf (accessed on 9 January 2019); World Bank (2014[11]), Georgia - Technical assistance to support preparation of education sector strategy : education sector policy review – strategic issues and reform agenda, http://documents.worldbank.org/curated/en/505151488895322292/Georgia-Technical-assistance-to-support-preparation-of-education-sector-strategy-education-sector-policy-review-strategic-issues-and-reform-agenda (accessed on 21 June 2018); World Bank (n.d.[12]), A Review of Teacher Policy Reforms in Georgia - A Case Study, World Bank.

Teaching workforce

One in four teachers is over 60

A history of low teacher salaries, low pension payments and the absence of a retirement age mean that many teachers in Georgia continue to teach long after they start to receive their pension. Georgia teachers were the oldest on average out of all teachers from TALIS countries. Around 26% of Georgia’s lower secondary teachers are over 60 (up from roughly one-fifth five years ago), compared with less than 8% across TALIS-participating countries (OECD, 2019[13]).

The high share of older teachers is a sensitive political issue. As teachers’ salaries and pensions are low, there is a perception that it is fair for older teachers to benefit from their salaries as they collect their pensions. However, the presence of a large share of older teachers limits the availability of full-time teaching posts and also deflates average teacher salaries nationally. There are also concerns that older teachers less motivated to engage with professional development or upgrade their skills, making modernising teaching difficult (World Bank, n.d.[12]). Fewer older teachers, for example, have passed the new certification examinations to reach higher levels on the new teaching career path (see Table 3.1).

copy the linklink copied!
Table 3.1. Georgia’s teachers by age and status

Under 25 years

25-29 years

30-39 years

40-49 years

50-59 years

60 years or more

Status level as % of all teachers

(% of teachers at each status level within each age group)

Practitioner

9%

44%

56%

50%

56%

77%

59%

Senior

4%

23%

34%

41%

41%

22%

35%

Lead

0%

0%

1%

1%

1%

0%

0%

Mentor

0%

0%

0%

0%

0%

0%

0%

No status

87%

9%

2%

3%

3%

1%

6%

Age group as % of all teachers

1%

4%

4%

27%

25%

26%

100%

Source: Author’s calculations based on data provided from EMIS in 2018.

The teacher population has not declined in proportion to the fall in student numbers

While the student population has declined by around 21% over the past decade, the teacher population has fallen by less than 1%. One reason that teacher numbers have not fallen in line with the decline in students is a political decision to keep open many small schools in rural areas. In these areas, the average student to teacher ratio is as low as 3.5:1 (World Bank, n.d.[12]). This leaves many teachers in part-time positions, even though they would like to work full-time. Overall in Georgia, 48% of teachers work part-time, in contrast to 23% on average across TALIS-participating countries (OECD, 2019[13]). The current oversupply of teachers is also reflected in principals’ responses to a survey conducted for this review. When asked what their schools most important goals are, retaining staff or hiring more staff was ranked as the least important (improving teachers’ pedagogical practices, however, was ranked as the most important).

There are teacher shortages in some subjects, and in rural, mountainous areas

Despite the general oversupply of teachers, there are shortages in some subjects such as sciences, mathematics, and Georgian as a second language. There is also a shortage of qualified teachers in rural, mountainous areas. In response, in 2009 the Teacher Professional Development Centre (TPDC) in the ministry launched Teach for Georgia. This programme provides young, motivated teachers with financial incentives, including a small salary bonus, to work in rural, mountainous areas. However, the scale of Teach for Georgia remains limited – with only 200 teachers participating in the programme in 2017.

Some teachers work across a broad range of grades

Among the schools that the OECD team visited, a number of teachers taught across grades 1-12. This practice was not just limited to small rural schools, but was also common in large urban areas. While necessary in some cases, asking teachers to teach across several levels creates some risk because the teachers might not be equally knowledgeable about student learning at different stages of their development. In most OECD countries, teachers focus on one or two grades within the same level of education because students’ learning and developmental needs vary vastly across different age groups. This configuration enables teachers to develop specialised knowledge in effective teaching strategies for the age group that they work with.

Teacher salaries and career progression

Most teachers remain on the entry level of the career path

Two new teacher statuses – uncertified and certified - were introduced in 2010. Teachers were required to become certified, which entailed passing examinations in pedagogical skills and subject knowledge. However, few teachers actually became certified and an evaluation of the policy determined that a more effective scheme was needed. Established in 2015, the new scheme (called the teacher professional development scheme) sets out four categories of teachers: practitioner, senior, lead and mentor teacher. Each step is associated with a significant salary increase and teachers are expected to take on new roles and responsibilities (see Table 3.2). A new appraisal for promotion was introduced at the same time, according to which teachers must accumulate credits through various professional development activities (see Teacher appraisal in Georgia) in order to advance in the scheme. Nevertheless, despite the introduction of career pathways, the vast majority of Georgian teachers are still at the practitioner level (see Table 3.1).

While several factors, including the complexity of the evaluation procedures, have inhibited teachers’ progression, the fact that the majority of teachers work part-time and do not stand to benefit fully from the salary increase also affects their motivation to move along the pathway. Instead of relying on the professional development scheme to raise their incomes, many teachers also supplement their low earnings through private tutoring (World Bank, n.d.[12]); around half of the teachers surveyed for this review indicated that they offer private tutoring.

copy the linklink copied!
Table 3.2. Georgia’s teacher career structure

Roles and responsibilities

Teacher steps

Annual salaries (2015)

Preparation and instruction

Professional development

Supporting the teaching community

Practitioner

GEL 4 860 (Georgian lari)

USD 5 758

60.4% per capita GDP

Practitioners plan and undertake teaching in line with the national curriculum and use assessments developed by the school.

Teachers determine their professional needs and pursue professional development

-

Senior

GEL 8 700 (Georgian lari)

USD 10 308

108.2% per capita GDP

Senior teachers develop innovative teaching strategies.

They create their own assessments and analyse results to improve the learning.

Teachers determine their professional needs and pursue professional development

Teachers cooperate with colleagues on research and to improve professional practices

Lead

GEL 10 860 (Georgian lari)

USD 12 867

135% per capita GDP

Lead teachers adapt teaching in response to students’ characteristics. They undertake research on effective learning strategies.

Teachers determine their professional needs and pursue professional development

Lead teachers support their colleagues to identify their professional needs and plan professional development.

Mentor

GEL 13 260 (Georgian lari)

USD 15 711

164.8% per capita GDP

Mentor teachers undertake research and use the results to develop recommendations for the school to ensure a positive school environment and improve learning.

Teachers determine their professional needs and pursue professional development

Mentor teachers develop learning resources for colleagues and plan activities for professional development within the school

Sources: World Bank (n.d.[12]), A Review of Teacher Policy Reforms in Georgia - A Case Study, World Bank; Ministry of Education/UNICEF (2015[14]), Country Background Report: Georgia.

Entry level salaries are low, but now increase rapidly with promotion

Full-time teachers in Georgia with minimum training and 15 years of experience earn less than 66% per capita GDP. In contrast, the average teacher in an OECD country with the same training and years of experience earns 10% more than the per capita GDP in their country (OECD, 2016[15]). However, reforms to Georgia’s teacher career structure and salary scale in 2015 mean that once a teacher is promoted to the second step on the teacher career path (i.e. to “senior” teacher), their salary increases to 108% of per capita GDP. A teacher’s salary also increases substantially with each subsequent step, so that a teacher at the top step – mentor – earns 2.7 times more than a teacher at the lowest step - practitioner. This is a bigger increase than virtually all other Programme for International Student Assessment (PISA) participating countries (OECD, 2016[15]). At the time of this review, the Georgian government was planning to increase the salaries of teachers, however the details of this reform (e.g. scale of increase, criteria for allocation) had not yet been determined.

Schools have considerable autonomy for hiring and firing teachers

According to PISA 2015, almost all students in Georgia (99%) attend schools where the principal is responsible for hiring teachers, in contrast to 70% of students on average across OECD countries (OECD, 2016[15]). Principals are not expected to follow any central guidelines or requirements when recruiting teachers (World Bank, 2014[10]). Also, unlike in most OECD countries, teachers are not public servants. They are only employees of the school where they work with contracts signed by the school principal. Overall, this means that principals have considerably more autonomy and responsibility for managing the teacher workforce than in many other countries. While this level of autonomy can help principals align teacher recruitment with the school’s needs, it also raises questions of fairness and transparency, and complicates management of the workforce.

Principals in Georgia also have far more autonomy over teacher dismissal than in many OECD countries (OECD, 2016[15]). Making principals solely responsible for dismissals can put them in a difficult position. A principal works with the teachers in their school every day, and in rural areas may have close relationships with a teacher outside the school. This is one of the reasons why, in OECD countries, around half of students attend schools in which regional or national education authorities are responsible for teacher dismissal (OECD, 2016[15]).

Initial teacher education and continuous professional development

There are currently two routes into teaching – a four-year (240 credits) concurrent programme and a one-year (60 credits) consecutive programme. Historically, most teachers were educated through dedicated teaching programmes provided mainly by specialised pedagogical institutes. However, changes in the structure of higher education combined with declining interest in teaching as a first choice of profession mean that, today, most beginner teachers graduate from a consecutive programme (World Bank, n.d.[12]). At the time of the OECD review, Georgia was also considering introducing a two-year work experience period as an alternative pathway into teaching for mid-career professionals from other fields.

Some practising teachers received no or low quality preparation

Data from PISA 2015 indicates that roughly 30% teachers have not received any initial teacher preparation in Georgia. In contrast, virtually all practising teachers in OECD countries received initial preparation (OECD, 2016[15]).

Among those teachers with initial preparation in Georgia, there are concerns about the quality of their content knowledge. This is a particular concern regarding teachers who entered the profession in the past 10 to 15 years, as the academic aspect of initial teacher education programmes has reportedly declined in recent years. In a study of 17 countries’ initial teacher education, future primary teachers in Georgia had the lowest scores for mathematics content knowledge and mathematics pedagogical content knowledge. The mathematical content knowledge of future secondary teachers was also the lowest of the 17 countries (Ingvarson et al., 2013[16]).

The threshold for entry to concurrent programmes is low

The concurrent programme educates prospective teachers for all levels of schooling. In September 2018, an additional year was added to extend the programme to five years (300 credits). The new programme devotes 60 credits to school practice and practice research. Starting in 2018, prospective primary teachers will receive a bachelor’s degree from this programme and prospective upper secondary teachers a master’s degree.

Entrance to the new concurrent programme, like for other tertiary programmes, is based on a student’s score on the United Entrance Examination (UEE) at the end of upper secondary education. However, since demand to enter teacher education is low, entry is not selective. In 2014, entrants to the four-year teacher education programme had the lowest average scores in the UEE of all tertiary entrants (World Bank, n.d.[12]).

Recently introduced consecutive programmes are more selective and higher quality

In 2010, a one-year (60 credits) programme in pedagogy for holders of a bachelor’s or master’s degree in a national curriculum subject was introduced. The course prepares persons to teach Grades 7-12. Since the programme focuses on developing teaching skills, candidates must pass a subject knowledge examination for entry. This programme is perceived to be better designed than the existing four-year programme. Since entrants have to meet a minimum threshold in subject knowledge before they enter, it is more selective. The review team was told that graduates from this programme tend to be more successful in finding a teaching post than those from the four-year programme.

Initial teacher education providers have significant autonomy over content

Initial teacher education is provided by all nine public universities in Georgia. In the past, while providers were expected to incorporate the national teacher standards into their programmes, they developed their own curricula and were not expected to follow a standard course structure and content. The absence of robust and consistent requirements contributed to the low quality of initial teacher education. However, from 2018 providers will be expected to follow standard benchmarks, based upon national teacher standards, for the new 300 credits programme (National Center for Educational Quality Enhancement, 2017[17]).

Accreditation requirements for teacher education programmes are also not very robust or specific, leaving providers with significant flexibility. For example, while all programmes are expected to provide a teacher practicum, in recent years it has fallen out of use in many programmes (Ingvarson et al., 2013[16]). In contrast, a teaching practicum is mandatory in virtually all OECD and many non-member countries (OECD, 2016[15]).

Teachers participate less in professional development than in OECD countries

While the country’s teacher standards create the expectation that teachers participate in professional development, participation is not compulsory. Just one-fifth of teachers in Georgia attended a professional development course in the three months preceding PISA 2015, compared to half of teachers on average in OECD countries (OECD, 2016[15]). For teachers not seeking promotion, and therefore not needing to accumulate credits, there is little incentive to participate in professional development. Another explanation for the low rate of participation is the cost. Although courses are free for teachers, there are still transportation and accommodation costs for many.

The vast majority of teacher training is provided by the Teacher Professional Development Centre (TPDC) through regional training centres. However, following policy reversals in recent years to decentralise and then re-centralise training, courses provided by other institutions are once again starting to be accredited. The professional development on offer has also recently been strengthened by support from international donors. The United States Agency for International Development (USAID)’s Georgia Primary Education Project (G-PriEd), the Millennium Challenge Corporation (MCC), the United Nations Children's Fund (UNICEF) and other development partners have been have been working with TPDC to provide subject-specific professional development activities embedded in teachers’ daily practice at the school-level (see Box 3.1). These projects reflect a major investment in the quality and availability of professional development in Georgia.

There is little tradition of school-based groups for professional development

Schools have established bodies – the pedagogical board and subject chairs – that encourage teacher collaboration. It is common in Georgia for teachers to come together in subject or grade groups to plan their lessons and determine how the curriculum will be delivered. However, there has historically been little culture of collaborative learning or deliberate focus on how to improve instructional practice. Since 2011, the G-PriEd programme has been implementing a school-based professional development model in around a third of Georgia’s schools at the primary level. Since 2016, the MCC has been doing the same at the secondary level. These initiatives have sought to make teacher collaboration at the school-level more meaningful by training teachers within schools to lead peer learning and implement activities for professional learning, like observing teaching and providing feedback.

At the end of 2018, the ministry announced a “New School Model”, which builds on the G-PriEd and MCC work. Under this new programme, schools will be supported by coaches to develop teachers’ confidence and capacity to adapt instruction to students’ needs and interests. This supports the country’s on-going curriculum reform that is focused on competency development, and provides teachers with greater flexibility since the curriculum will now be organised by key learning stages rather than individual grades (see chapter 2).

copy the linklink copied!Teacher appraisal in Georgia

With the introduction of the new teacher professional development scheme in 2015, Georgia has made a significant effort to use appraisal to encourage teachers’ professional growth and base career progress on merit. However, many teachers are not engaging with this programme and few have been promoted. The scheme’s current design, which requires teachers to participate in a broad range of activities in order to accumulate credits for promotion, is distracting teachers from their core function. This situation is exacerbated by the absence of a regular, formative appraisal that would support and feedback and motivate changes in teaching and learning. Finally, Georgia does not require new teachers to undergo a dedicated appraisal for probation to confirm their competence to teach after acquiring some classroom experience. This is a notable gap given concerns about the quality of initial teacher education programmes.

copy the linklink copied!
Table 3.3. Teacher appraisal in Georgia

Type of appraisal

Reference standards

Evaluator

Frequency

Procedure

Use of Results

Initial certification

Teacher standards

Initial teacher education providers

National Assessment and Examination Centre (NAEC)

Complete initial teacher education programme and possess a bachelor’s degree

Teachers apply directly to school vacancies

Probation

Does not exist at present, but plans to introduce a two-year probation as alternative pathway into teaching

Regular appraisal

Does not exist at present, but plans to be introduced

Promotion

Teacher standards

Teacher Assessment Group (composed of school principal, deputy principal, facilitator, relevant subject chair and representative of local Education Resource Centre (ERC)

Optional

1. Teacher’s self-evaluation of professional activities in previous year

2. Classroom observations

3. Teachers acquire credits through a combination of different activities

4. Teachers upload documents certifying activities to an electronic portfolio

5. The TPDC validates the activities and awards credits

Teachers must accumulate a specified number to be promoted.

Re-certification

Does not exist

Reward

Does not exist

Teaching standards are based on evidence of effective teaching, but are not well-integrated with teaching policy and practice

When Georgia introduced its new teacher professional development scheme in 2015, it also developed new teaching standards for the four steps in the career path. The standards are based on international good practice and cover a number of the aspects that research suggests are important for effective teaching (see Table 3.2). While this was a positive step, the standards do not seem to be the central reference for teaching policy and practice in Georgia. For example, because the system for teacher’s professional development focuses on the acquisition of credits, evaluators and teachers concentrate on meeting the requirements for credits, rather than meeting the standards that set out higher levels of teaching competence in line with the teacher career path.

Requirements for initial certification have been strengthened in recent years

Beginner teachers are required to pass an examination in order to take up a teaching post

Prospective teachers in Georgia are required to have successfully completed either a concurrent or consecutive initial teacher education programme to be eligible to teach in a public school. Assessment of teachers during initial teacher education is continuous, based on a teacher’s portfolio, which is reviewed according to standardised instruments. Since 2010, new teachers have also been required to pass subject examinations (secondary teachers) or an integrated test of different subjects (primary teachers), as well as teaching methodology examinations in order to take up a teaching post. The examinations are developed and administered by the National Assessment and Examinations Centre (NAEC), following guidance from the ministry. Teachers that are successful enter the profession at the second step on the career path, as senior teachers.

Only around 40% of in-service teachers have passed the initial certification examinations

The certification examinations that new teachers are required to pass were initially introduced for the whole profession with the intention of incentivising all teachers to meet minimum standards in terms of teaching competence and subject knowledge. When the scheme was introduced in 2010, in-service teachers automatically became practitioner teachers with the goal that, by 2014, all teachers would be certified. However, low certification rates compelled a policy reversal, making it optional for in-service teachers to take and pass the examinations. As a result, nearly 60% of in-service teachers remain at the practitioner level as of 2018, according to data provided by EMIS. In PISA 2015, only 34% of science teachers were reported to be certified, compared to 84% on average across OECD countries (OECD, 2016[15]).

Georgia does not have an induction programme or a probation period with appraisal

As part of the General Education Law of 2007, Georgia planned to introduce the requirement that all new teachers complete a one-year induction programme in schools (World Bank, 2014[10]). However, the policy was never implemented, leaving Georgia without an induction period or associated appraisal for probation. This leaves new teachers without any support in their first years on the job, which are recognised to be challenging. It also means that there is no rigorous confirmation of core teaching skills before a new teacher becomes fully registered for the rest of their career.

There are plans to introduce a regular appraisal

One of the conclusions from the evaluation of the teacher professional development scheme in 2016 was the need to introduce a regular appraisal. This will be important in order for Georgia’s appraisal system to more effectively support an improvement in teaching practice. At the time of drafting this review, it was not yet clear how Georgia planned to implement the new regular appraisal. Key questions that remain to be addressed include the evidence it will be based on, who will be responsible for evaluating teachers and how the results will be used.

Teachers need to accumulate credits in order to be promoted

The teacher professional development scheme requires teachers to acquire credits to move up the four step career path. Teachers can acquire credits by completing a range of specified mandatory and voluntary activities like attending training, speaking at conferences, developing educational materials, and preparing students for academic competitions. Teachers also undertake a self-evaluation and receive a number of classroom observations, depending on the step that they are seeking to be promoted to (see Table 3.4).

copy the linklink copied!
Table 3.4. Credits required for promotion

Practitioner to senior teacher

Senior to lead teacher

Lead to mentor teacher

Number of credits required

19

19

25

Mandatory activities

Two internal classroom observations (4 credits)

Subject knowledge exam (max. 10 credits) or

Pedagogy exam (max. 10 credits)

External classroom observation (4 credits)

Pedagogical research (2 credits)

Two model lessons evaluated internally (2 credits)

Four model lessons evaluated internally (4 credits)

Conducting assessment of an educational institution (3 credits)

Developing educational resources or peer-reviewed professional literature (2 credits)

Optional activities

Meetings with colleagues (0.5 credit)

Preparing students for competitions (0.5 credit)

School group activities (0.5 credit)

Using ICT in teaching (0.5 credit))

Member of the Teacher Assessment Group (1 credit)

In-service training, 25 hours (1 credit)

Club activities (1 credit)

Working with students with special education needs (1-2 credits)

Social projects (0.5 credit)

Official language course (2 credits)

Professional development e.g. accredited training (1 credit), official language course (2 credits), or creating (1 credit) or providing (0.5 credit) an accredited training course

Extracurricular activities e.g. running a school club (1 credit), or organising a summer school (1 credit)

Teaching and learning e.g. working with students with special educational (1-2 credits), teaching projects (0.5 credit), or preparing students for competitions (0.5 credit).

Supporting school development e.g. meetings with colleagues (0.5-1 credit); supervising other teachers (3 credits); membership of the Teacher Assessment Group (1 credit);

External school activities e.g. speaking at a conference (1 credit); creating teaching resources (1 credit); producing peer-reviewed professional literature (1 credit); d); education blog (1 credit);

Any other additional activity initiated by the teacher (number of credits defined by the Teacher Assessment Group).

Credits required to maintain status

13 in 3 years

17 in 4 years

Status maintained permanently

Sources: Ministry of Education/UNICEF (2015[14])Country Background Report: Georgia; World Bank (n.d.[12])A Review of Teacher Policy Reforms in Georgia – A Case Study, World Bank; MoESCS (n.d.[18]), Forms for teacher appraisal, MoESCS.

Each school has a Teacher Assessment Group that is responsible for undertaking classroom observations and awarding credits. The group is led by the school principal and includes the deputy principal (if applicable), a facilitator and a subject chair. More recently, a representative from the local Education Resource Centre (ERC) has been added. The ERC is a small team of four-five staff in each municipality that visits schools to check compliance with regulations and shares the information with the ministry (see chapter 1 and chapter 4). After having awarded credits to a teacher, the Teacher Assessment Group makes a recommendation to TPDC on whether the teacher should be promoted to the next step or retain their current status. TPDC reviews each teacher’s file and takes the final decision. One positive aspect of the system is that it provides teachers with multiple ways to demonstrate their competencies. In practice, however, the process often requires lots of form filling without necessarily evaluating teaching quality (World Bank, n.d.[12]).

The ministry has recently decentralised the management of underperformance to schools

In cases when a teacher applies for promotion and the Teacher Assessment Group determines that they have not satisfied the credit requirements for maintaining their position, the teacher can be demoted (except for practitioner teachers). However, if a teacher does not apply for promotion there is no central mechanism to assess or address their underperformance. The ultimate sanction – teacher dismissal – is the responsibility of school principals.

copy the linklink copied!Policy issues

Georgia’s most pressing priority is to develop existing teachers so they have the knowledge and pedagogical skills to deliver the changes to instruction envisaged by the country’s new curriculum. Achieving this goal will mean revising the teacher professional development scheme so that it is more effective in driving modernisation and improvement in teacher practice. Teachers also need to receive regular support and feedback – through regular appraisal and by continuing the recent work to strengthen teachers’ professional development – so that they are motivated to continue developing professionally throughout their career. Over time, requiring that new entrants to the profession have strong academic skills and are well supported in their early years will raise the overall quality of teaching and learning.

copy the linklink copied!Policy issue 3.1. Applying minimum standards for teaching and encouraging the development of higher teaching competencies

Since 2004, Georgia has attempted to shift teaching towards competency development and a more student-focused approach. This implies a major change in a teacher’s role – from lecturing and giving instructions to guiding and supporting students in response to their individual needs. It also implies an important shift in pedagogical resources and practices – away from tests where students recall facts and towards tasks that require them to critically use information to solve problems and produce a compelling argument. Many countries have implemented a similar change in recent years and have found that teachers need to be equipped with the necessary pedagogical skills in order for this new approach to reach all classrooms. Addressing this concern in Georgia will be critical as teachers already engage in less professional development than most other countries.

Reforms to the teacher professional development scheme since 2010 reflect an important effort by the ministry to professionalise and motivate the country’s teaching body. However, nine years after the scheme was first introduced, it has had little impact on improving teaching quality. The majority of teachers remain at the entry stage, suggesting that they are not motivated to progress up the career path or lack the skills and knowledge to do so.

Georgia’s priority should be to revise the teacher professional development scheme to better achieve its original aims. First, it should require all teachers to demonstrate minimum teaching competencies, broadly indicated by reaching senior teacher status, and support them in their efforts to reach that level. Second, instead of focusing on credit accumulation, which is distracting teachers from focusing on individual student learning, the scheme should recognise and reward effective instructional practice. This emphasis will help motivate teachers to develop higher levels of teaching competence as set out in the country’s teacher standards.

Recommendation 3.1.1. Support all teachers to meet minimum standards

The teacher examinations that were introduced in 2010 were originally intended to ensure that all teachers possess basic subject and pedagogical knowledge. While written examinations cannot evaluate all the attributes of effective teaching (D’Agostino and Powers, 2009[19]), they can reveal whether teachers have acquired a basic knowledge. For beginner teachers, an examination can help to make sure that new graduates have acquired the foundations for teaching. This is particularly important in Georgia, where quality assurance in initial teacher education is weak and teaching tends to attract students with low levels of academic achievement (see Recommendation 3.3.1). For teachers already in the classroom, the requirement to pass an examination for certification can help establish clear standards for the profession and make sure that teachers have the knowledge and skills required to meet national learning goals. Requiring that teachers pass an examination can also help direct professional development to address important gaps in teacher competencies.

Clearly communicate the examinations’ role

One important message that the ministry needs to communicate about the examinations is that they are essential for professionalising teaching. One of the reasons some practitioner teachers currently appear unmotivated to take the exams are recent policy changes that have created ambiguity around whether teachers need to pass the exams or not. This has led some teachers to think that it is not worth investing in passing even one of the examinations since this requirement may be changed in the future (World Bank, n.d.[12]). Consequently, the ministry should clearly state that all teachers are required to reach senior teacher level within a reasonable period of time (e.g. three years).

The original policy that teachers had to meet a minimum threshold in both subject content and pedagogy examinations was positive because it validated that teachers are both knowledgeable in their domain and have the teaching skills to make content accessible for students. However, this policy has now been changed, and teachers are able to reach senior level by doing well in only one examination. Since both pedagogical and content knowledge are essential for teaching, the ministry should revert to requiring that teachers pass both examinations.

The examinations should also be presented as the first step to be confirmed in a profession that is setting higher standards for itself. The examinations are not independent, but rather are part of a wider policy to support teachers to move up the career path, alongside fuller, more authentic types of evaluations. This message is important to address teachers’ valid concerns that the examinations cannot accurately assess all the skills and qualities of an effective teacher (World Bank, n.d.[12]).

Another important message to communicate is that examinations are primarily developmental. This means that the results will be used to confirm that teachers have the knowledge to teach the curriculum and, where this is not the case, direct teachers to professional development to address their knowledge gaps.

A policy for managing those teachers who are unable to demonstrate minimum teaching competence also needs to be introduced. If a practitioner teacher is unable to reach senior teacher status within the determined period of time, they should be offered a non-teaching role or a position within the central administration.

Develop the certification examinations with the goal of assessing essential teaching knowledge and skills

Georgia should review the examinations with the goal of determining whether their content and question types effectively assess minimum standards for knowledge and skills. The review team’s interviews with practising teachers who had passed the examinations suggested that the content was generally relevant for the role of a teacher and accessible for well-prepared teachers. However, teacher focus groups also demonstrate that around half of teachers are sceptical that the examinations effectively assess their skills (World Bank, n.d.[12]).

While it is true that such examinations can never assess all the attributes of teaching knowledge and skills, some steps can be taken to strengthen the connection, such as:

  • Align the examinations with the curriculum. The examinations should assess if teachers have sufficient knowledge of the curriculum in order to deliver it.

  • Align the examinations with teacher standards, in particular the standard for new entrants (see Recommendation 3.3.1).

The ministry might also consider if the content for the pedagogy examination should be adapted for existing and graduate teachers. Since existing teachers have acquired many years of teaching experience, they can be expected show a deeper understanding of how to apply teaching concepts. Adapting the examination’s content would also provide the opportunity to reflect upon the different expectations of existing and graduate teachers, as set out in the revised teacher standards that this chapter recommends.

Support existing teachers to master essential knowledge and skills

The ministry is faced with two challenges to support all teachers to pass the examination. First, as of the end 2018, nearly 40% of teachers below 60 years of age had not taken even one examination. These teachers may not be motivated to take the examinations for a variety of reasons - it is no longer a compulsory requirement, the value of the examinations is questionable, or they work part-time and the potential pay increase is not attractive enough. These challenges are addressed elsewhere in this chapter (Recommendation 3.1.1 and Recommendation 3.4.1).

A barrier for minority language teachers is that the examinations and associated training are not currently available in minority languages. This is likely one reason why a far higher share of minority language teachers – over 75% of Armenian and Azerbaijani teachers - have not yet taken one examination. The ministry should make it a priority to address this gap immediately.

Second, at around 30% between 2010 and 2016, the pass rate for the examinations is very low. TPDC already provides training to help in-service practitioner teachers prepare for the examinations, which should be continued. There are a number of other steps that the ministry should consider in order to support teachers to gain the knowledge needed to teach the curriculum (which is also assessed in the examination):

  • Provide more support in areas or topics where examination results indicate teachers are struggling. The examination results vary widely across subjects. On average in 2017, the review team was told that teachers did less well in mathematics, civic education and Georgian language and literature. The ministry should conduct further analysis to identify the specific types of questions or topics that teachers find difficult so as to better orient training.

  • Provide greater support for groups of teachers who tend to do less well in the examinations. These teachers could be in rural areas and smaller schools who, on average, perform less well in the examinations (World Bank, n.d.[12]). In 2018, the share of practitioner teachers was more than 20 percentage points higher in villages than in cities.

  • Cover the transport and accommodation costs associated with participation in training for teachers.

Georgia will also need to develop a specific policy for older teachers who are not motivated or able to pass the examinations (see Recommendation 3.4.1).

Encourage each school to make it a priority that all teachers reach senior status

Principals need to recognise that passing the examinations is not just the goal and responsibility of an individual teacher, but also for themselves and their schools. Transforming their role in this manner would be line with existing national efforts (e.g. MCC’s leadership academies and the G-PriEd programmes) to help principals in Georgia become instructional leaders (see chapter 4).

A key responsibility of an instructional leader is supporting teachers’ professional development and improving the quality of teaching and learning in the school. This might mean that, in the school development plan, principals are encouraged to include a target for the share of practitioner teachers who will become senior teachers within the year and how the school intends to support them to do this. For example, principals should help teachers develop individual learning plans that set out how they will reach senior status and arrange for senior teachers to mentor practitioners. Principals might also organise teacher groups to discuss the kinds of content that the examinations assess.

Recommendation 3.1.2. Re-focus the teacher professional development scheme on demonstrating higher levels of teaching competencies

The design of the current teacher professional development scheme is not always rewarding good teaching practice nor motivating teachers to develop. A central issue is the requirement to accumulate credits for promotion. This involves burdensome reporting, encourages teachers to undertake activities that might contribute little to improving their teaching and pays little real attention to the quality of teaching practice.

Other issues include how the promotion decision is taken, by school-level committee, with limited external actors. In many countries with promotion appraisal, external evaluators are more involved in decision-making since promotion carries high stakes for a teacher’s career (OECD, 2015[7]). Teacher standards are also vague in terms of the specific differences between the roles of senior, lead or mentor teachers. This makes it difficult for the evaluation group to fairly and consistently evaluate whether a teacher has demonstrated the competencies required to be promoted to the next level. Finally, while promotion in theory provides a significant increase in a teacher’s salary (see Table 3.2), since more than half of teachers only work part-time they cannot fully benefit from the increase. Georgia should consider the following changes to the teacher professional development scheme as part of wider changes to management of the teaching profession and the planned pay increase (see Recommendation 3.4.2).

Make teacher standards the main reference for promotion

Teacher standards should be the main reference for promotion to focus the process more directly on demonstrating higher levels of teaching competence. For example, when a teacher is applying to be promoted from senior to lead teacher, an evaluator would expect that teacher to demonstrate competencies similar to those of a lead teacher (or have clear capacity to develop these competencies). Instead, the current scheme in Georgia focuses on completing forms to acquire credits for activities, such as attending a conference or organising extracurricular activities.

To address these inconsistencies, Georgia should first review and revise its current standards so that they become a more operational tool to determine appraisal judgements. Revisions include:

  • Clearly set out expectations for increasing teaching competence. Georgia’s current teaching standards do mention some key features of good teaching and how these evolve as teachers move up the career path. However, not all areas that are important for effective teaching are discussed in this manner. For example, areas like planning and preparation and managing the classroom environment are important and relatively underdeveloped in Georgia’s standards (Danielson, 2013[20]). Providing more clarity would help teachers be evaluated at all levels of the career ladder in relation to the types of practices that are most important for effective teaching.

  • Add specific examples to illustrate effective practice at each step on the career path to guide evaluators to make consistent promotion decisions. This would also provide teachers with a clear understanding of what they should be aiming towards. Georgia might consider using videos, like Australia’s teacher standards does, to illustrate examples of effective teaching practice at the different steps (AITSL, 2011[21]).

  • Demonstrate the new roles and responsibilities teachers are expected to take on as they move up the career path. For example, in Australia teachers who reach higher levels are expected to take on broader roles within the school that contribute to teacher and school development (AITSL, 2011[21]). In Georgia, this might include mentoring trainee teachers during their induction period (see Recommendation 3.3.2).

Focus promotion appraisal on authentic evidence of teaching practice

Georgia’s appraisal for promotion already includes many of the evidence sources that are common in OECD countries - classroom observation, an interview between the teacher and their evaluator(s), a teacher’s self-assessment and review of a teacher’s portfolio (OECD, 2015[7]). Classroom observations is one of the most accurate methods of assessing a teacher’s ability to teach because most of the key aspects of teaching are displayed when a teacher interacts with their students in the classroom (OECD, 2013[1]). However, alongside classroom observations, promotion in Georgia is also based on many other credit-awarding activities that provide less relevant information about a teacher’s impact on student learning. For example, teachers can receive credits for organising extracurricular activities, writing an education blog or speaking at a conference (see Table 3.4).

Georgia should revise its appraisal to focus more on demonstrating effective teaching. The practice of collecting credits should end and be replaced by appraisal based on classroom observations, a teacher’s self-evaluation, the review of a teacher’s portfolio and an interview with the teacher. Each of these sources of evidence is already part of appraisal for promotion in Georgia. A few changes could be introduced so that they are used more effectively:

  • Provide evaluators with guidance on the kinds of evidence of student learning that can be collected during classroom observations. Teachers should be evaluated in relation to how effectively they are engaging with the learning needs of every student in their class and helping them to realise their full potential. Evaluators might be provided with guiding questions to help them focus on the quality and frequency of teachers’ formative assessment and summative judgements (see chapter 2).

  • Restructure the teacher portfolio to document how teachers have demonstrated the knowledge and skills for the next level in the teaching career path. The current portfolio is primarily used for adding certificates of completed training. Instead, teachers would use the restructured portfolio to provide evidence of teaching practice like examples of lesson plans where they have made changes to accommodate a new teaching approach, or examples of student work where different types of assessments are used. The new portfolio can be used by teachers during their self-evaluations and interviews to demonstrate how they have understood and applied curriculum and other instructional changes to their practice.

  • Provide evaluators with guidance on how to structure the appraisal interview so that teachers can demonstrate increasing professional maturity and competence. For example, evaluators might be provided with prompts or questions to ask teachers about which professional development opportunities they have pursued, and how they have demonstrated higher levels of teaching competence.

Strengthen the role of independence and professional competence in promotion decisions

The current arrangement for promotion decisions – by the Teacher Assessment Group – provides important opportunities for school input. However, external actors can be more independent and impartial than school-level actors who are personally familiar to a teacher. They should also have professional competence, as a trained evaluator, to make such decisions.

In Georgia, while the Teacher Assessment Group might continue to provide their views on teacher promotions, the final decision should be undertaken by an external evaluator. The external evaluator might undertake classroom observations and the teacher interview to collect evidence to reach their decision. As part of its new professional development programme for teachers, Georgia has recently started to contract experts to undertake external classroom observations of teachers. These experts could also be given the responsibility for making decisions about promotion. Given that their responsibilities will affect the career trajectories of many teachers, the experts should be highly qualified to assume their responsibilities. For instance, they should have significant teaching experience. They should also receive training on how to undertake classroom observations that are focused on determining the quality of teaching and learning, and in particular how to assess the competencies required to move up the teaching career path.

copy the linklink copied!Policy issue 3.2. Supporting teachers to develop professionally throughout their career

With the introduction of the teacher professional development scheme, Georgia has recognised the importance of appraisal for professionalising the teaching workforce. However, an important lever to embed more student-centred teaching is also providing regular feedback and guidance to teachers. Regular appraisal can encourage and support teachers to adopt teaching methods with a proven impact on learning, such as formative assessment and student feedback. While Georgia does not have regular appraisal, the country has recently announced plans for its introduction. This section provides suggestions on how this might be designed and used most effectively.

Regular appraisal will need to be complemented by a significant expansion in high quality professional development opportunities. The latter has recently been boosted by donor-funded projects that are focused on school-based professional development programmes. This policy issue also provides recommendations for expanding and continuing these programmes to ensure their future sustainability.

Recommendation 3.2.1. Focus the new regular appraisal on student learning and providing feedback for teachers’ professional learning

Regular, formative appraisal encourages open discussion and critical self-reflection, activities which have long been recognised as important for professional growth and development (Dewey, 1938[22]). Regular appraisal tends to be relatively informal, since it is frequently led by a school principal or someone else who is familiar with a teacher’s work, and carries few stakes for a teacher’s career. Effective appraisal is action-oriented, giving teachers advice and support on how they can improve their daily classroom practice.

One of the challenges of developing an impactful regular appraisal in Georgia will be creating the kinds of informal, open conversations that are fundamental to this type of appraisal. The current appraisal process for promotion is formal and high stakes, and has not developed evaluators’ capacity to provide constructive, formative feedback. In particular, it has reinforced existing approaches where the measure of success is not learning in the classroom, but teacher activities largely outside the school. In this context, creating an effective, regular appraisal process in Georgia will be difficult and take time. This recommendation provides practical advice for developing an impactful and valuable process.

Introduce guidelines on a set of simple steps that schools can take to encourage regular appraisal practices

Regular appraisal guidelines should clearly state that the purpose of regular appraisal is formative and then provide simple pointers on how the school can execute the process. For example:

  • Evaluators start the year with a conversation with teachers on where they stand in terms of the teacher standards and the skills that they would like to develop over the coming year.

  • The evaluator and teacher develop a simple Individual Teacher Development Plan and agree how the teacher will be supported throughout the year.

  • During the year, the evaluator regularly undertakes short classroom observations and provides the teacher with feedback on their strengths and suggestions for improvement.

  • At the end of the year, the evaluator and teacher discuss how the teacher has performed that year, focusing on the teacher’s development in reference to the teacher standards.

Since regular appraisal will be new for the teachers that are evaluated, they will also need a clear explanation of how it will work and the purpose. Teachers will also need to feel ownership of regular appraisal, rather than perceiving it as an externally imposed process. These factors make it important to engage key school actors like the principal but also other experienced and respected teachers in each school. Teachers might also be asked to contribute to the development of regular appraisal tools, such as the forms that guide evaluator-teacher interviews.

Determine the evaluators

It will be important to select an evaluator(s) that is familiar with a teacher’s work, and has the opportunity to regularly observe their teaching. In order to establish an accurate view of a teacher’s daily practice, classroom observations do not need to be long but should be conducted frequently. To create an open discussion where teachers feel comfortable to discuss any difficulties they are experiencing, it is helpful for the evaluator to be known to the teacher. In most OECD countries, regular appraisal is led by an evaluator that is internal to the school (OECD, 2015[7]).

In Georgia, principals should start to conduct regular appraisals. However, they will need to be strongly supported as they take on this responsibility as many are not experienced with observing teachers and evaluating their teaching accordingly. According to a survey administered as part of this review, roughly one-third of teachers have not been observed by their principals within the past year, and 10% have never been. To support principals, deputy principals and/or experienced teachers can be asked to assist with performing observations and appraising teachers (also see Chapter 4 about developing the capacity of principals). In particular, undertaking regular appraisal might be a specific responsibility for lead and mentor teachers, especially when they do not have a full teaching load.

Involving the Teacher Assessment Group, which includes five individuals, in the direct evaluation of teachers for regular appraisal is likely to inhibit the kinds of open discussion and critical self-reflection that are essential for its efficacy. However, the Teacher Assessment Group could fulfil an important function by meeting once a year with the principal and any other evaluators. The purpose of this meeting would not be to discuss the appraisal of individual teachers but rather how the process has been all teachers across the school, identify what worked well and areas to be improved in the coming year. The discussion might also be used to identify professional development needs across the school, which would inform a school’s annual development plan (see chapter 4).

Develop national guidance for evaluators on how to collect and review evidence of teaching quality

Evidence of student progress and learning is central to evaluating the effectiveness of classroom teaching (OECD, 2013[1]). At present, a major gap in the appraisals for promotion that are undertaken as part of the teacher professional development scheme is the lack of a central focus on learning (World Bank, n.d.[12]). Evaluators for regular appraisal should be provided with guidance that encourages them to focus on the strategies that teachers are using to enable student learning. Examples of things that evaluators should be encouraged to look at include evidence of formative assessment, student feedback, creating an inclusive classroom that responds to different learning styles and monitoring students at risk of falling behind. Evaluators also need to collect evidence on whether teachers’ summative judgements are accurate and aligned with national learning expectations (see chapter 2).

Collecting evidence on and effectively evaluating the above will require significant expertise in teaching and learning and how to provide feedback. Evaluators for regular appraisals – school principals and other experienced teachers – will therefore need substantial support to undertake classroom observations. One form of support might be videos developed by the ministry to demonstrate good practices for observations like which materials to focus on, questions to prepare and how to engage with the teacher and students.

Support evaluators to provide useful developmental feedback

Following each classroom observation, teachers should receive feedback on their lesson, as well the opportunity for a broader discussion with their evaluator about their strengths, learning needs and plans for professional development over the coming year. The following tools can be developed for evaluators on how to provide feedback:

  • A form with guidance on how to conduct the evaluator-teacher interview. The form might include prompts for the evaluator to highlight the teacher’s strengths and how to provide constructive advice to address development areas, both in relation to national teacher standards.

  • Guidance and/or a template to systematically introduce teacher development plans. Research suggests that effective teacher development plans identify specific actionable growth objectives that are tied to the classroom, with realistic timelines and practical examples of activities that can lead to change (Cole, 2012[23]).

Recommendation 3.2.2. Give teachers access to high quality professional development

Countries that provide teachers with high quality, impactful professional development frequently combine two main types. One is in-service training, often organised at the national level outside a teacher’s school. This type of training can be helpful when introducing major policy changes (e.g. updating teachers on curricula changes) or advancing policy priorities (e.g. on formative assessment). The second is school-embedded professional development that takes place in a teacher’s school. This type of professional development often involves collaboration with other teachers and focuses on challenges or issues related to a teachers’ daily practice. In contexts where overall teaching capacity is relatively low, an external impetus is essential to make school-embedded professional development a genuine learning experience.

In Georgia, the G-PriEd project, MCC projects and the recently announced “New School Model” reflect many of the characteristics that are associated with the most effective types of professional development (Box 3.1). These include: providing professional development that is subject specific, providing opportunities for teachers to try out new teaching strategies through classroom observations and feedback, and creating school-based groups for teacher collaboration (Darling-Hammond, Hyler and Gardner, 2017[24]). Evaluation of the G-PriEd programme found that it has positively impacted students’ learning outcomes in mathematics and reading (USAID, 2016[25]). This reflects the OECD team’s interviews with stakeholders which indicated that the G-PriEd and MCC programmes are perceived to be having a major positive impact on support for teachers.

To continue this work, Georgia will need to address some of the structural issues that currently impede some teachers from engaging in professional development. It will also necessitate creating the necessary support to progressively mainstream this support and create a sustainable model to provide high quality professional development in the future.

copy the linklink copied!
Box 3.1. Support for teachers’ professional development as part of the Millennium Challenge Account and the Georgia Primary Education Project

USAID Georgia Primary Education Project (G-PriEd)

Launched in 2011 and ending in 2017, G-PriEd provided comprehensive assistance to around 28% of Georgia’s public schools to improve the reading and mathematics competencies of students in grades 1-6, and to introduce financial literacy. A major component of the project was supporting teachers to improve reading and mathematics instruction. The emphasis was on creating school-based professional development by:

  • Providing teacher training to primary teachers and national trainers. Training combined online training and face-to-face training. An online forum for teachers was created, with a series of webinars where trainers provided feedback to teachers and responded to their questions.

  • Developing an e-Portal with a variety of instructional resources including videos, electronic training courses, teacher resource books and tutorials.

  • Teacher learning circles for mathematics and reading were created for teachers to collectively discuss student achievement and ways to enhance instructional effectiveness. G-PriEd trained facilitators for these groups.

  • Classroom observations provided teachers with follow-up and feedback after training. National trainers provided teachers with descriptive feedback on how they implemented new teaching methods after training.

Millennium Challenge Corporation – Georgia, Training Educators for Excellence Project

The Training Educators for Excellence project will train 18 300 grades 7-12 teachers of science, mathematics, English and geography over 2016-19. The project aims to train teachers in modern teaching methods and strategies through:

  • three modules on student-centred learning approaches (36 hours in total)

  • six modules in active learning in subject-specific modules (144 hours in total).

The project also includes the training for teacher trainers and the development of training materials. Teachers who complete the training will be able to participate in study groups organised by the project. The study groups provide teachers with the opportunity to reflect on their training experience, develop their professional skills and plan their teaching practice with their colleagues.

Sources: USAID (2018[26]), Georgia Primary Education Project (G-PriEd), https://chemonics.com/wp-content/uploads/2018/07/Georgia-Primary-Education-Project-G-PriEd-Final-Report.pdf (accessed on 6 January 2019); MCC (2018[27]), Millennium Challenge Account-Georgia Monitoring and Evaluation Plan Compact II, https://assets.mcc.gov/content/uploads/georgia-compact-ii-me-plan.pdf (accessed on 22 January 2019).

Require that all teachers undertake professional development

All teachers need to understand that undertaking professional development is a part of their professional duties as a teacher. Requiring that teachers devote some time to school-based professional development is also important so that the work that has taken place as part of the G-PriEd and MCC projects continues.

Teachers in Georgia are expected to work 36 hours per week, but are only expected to teach for half of this time. While teachers are officially required to engage in non-teaching tasks, there are no mechanisms to verify that this takes place (World Bank, 2014[10]). This means that teachers have a significant amount of time that could be devoted to school-based professional development. At present however, while there is no official data, the review team’s interviews with stakeholders suggested that many teachers do not remain in school when they are not expected to teach. It is likely that many use this time for private tutoring. National research has found that 89% of private tutors in Georgia are school teachers (World Bank, n.d.[12]) and a survey conducted for this review revealed that roughly half of all teachers offer private-tutoring services.

To encourage teachers to collaborate with their colleagues on professional tasks for at least some of the time when they are not teaching, Georgia might consider introducing some of the following requirements:

  • Require that teachers spend at least a proportion of their non-teaching time in their school. This is a practice in many OECD countries (OECD, 2017[28]). Requiring that teachers remain in school makes communication and collaboration with colleagues more likely, and in Georgia would limit opportunities for teachers to engage in private tutoring.

  • Make collaboration with other teachers a mandatory, non-teaching task. In Georgia, required non-teaching tasks include collaboration on the school plan and designing the curriculum (World Bank, 2014[10]). However, collaboration for the purposes of instructional improvement is not a specified task. Across OECD countries, teamwork and dialogue with colleagues is mandatory for lower secondary teachers in around half of the countries with available data (OECD, 2017[28]).

  • Specify the number of hours or percentage of working time that teachers are expected to devote to non-teaching tasks. In Singapore, for example, 20 hours per week are built into teachers’ schedules for shared planning and classroom visits (Darling-Hammond and Rothman, 2011[29]). The specified time should include non-embedded professional development that takes place in training outside schools, as well as working collaboratively with other teachers in the same school.

Teacher’s participation in and contribution towards professional development should also play a much greater role in all types of appraisal. Teachers should be asked about how they have engaged with professional development in their school during the discussion with the evaluator, and to provide authentic examples of how they have integrated what they learnt during professional development into their teaching practice.

Sustain the programmes and capacity that has been built up through G-PriEd and MCC

G-PriEd has been provided to almost a third of primary schools, and MCC has covered well over 1 000 schools. While expanding these programs will be time consuming, Georgia should take steps in the immediate term to help sustain their impact. The first priority will be to provide sufficient financial resources for professional development. Georgia’s plans to increase the education budget (see Chapter 1) create the opportunity to increase resources for professional development. As well as increasing the national budget for professional development activities, the ministry should consider providing schools with their own funds that can be used flexibly for encouraging and developing school-embedded professional development activities. This would also support the country’s plans for a “New School Model” where schools have more confidence and capacity to adapt instruction to students’ individual needs and interests (see chapter 4).

One important use of the increased funds will be ensuring a sufficient number of national trainers across the country. The G-PriEd programme has already trained around 330 national trainers and MCC trained 446. The ministry’s “New School Model” promises to provide schools with their own coaches to develop teachers’ skills (see chapter 4). Since reaching all schools will be a major undertaking, an effective use of resources would be to prioritise the most disadvantaged schools.

Provide relevant and high quality training

As the G-PriEd and MCC programmes end, it will be important to establish other high quality training programmes. In line with the recent policy direction to start accrediting external providers of professional development again, TPDC should take an active role in facilitating the development of effective training offerings. One way to do this is by accrediting programmes based upon whether they exhibit many of the features that are associated with successful professional development. Such important features include:

  • Subject-specific professional development. This was a need echoed by teachers that the review team met. Teachers would like less generic training and more subject-specific courses that enable them to develop higher levels of teaching competence in their subjects.

  • Interactive professional development. Courses should also provide teachers with opportunities to design and try new teaching approaches through hands-on activities.

  • Follow-up. Require that teachers receive follow-up after participating in a course on how they have put what they have learnt into practice. For example, a requirement for completing a course might be that a teacher can demonstrate how they have applied new practices in their classroom.

As part of its role to provide professional development, TPDC should also consider how it can fully exploit opportunities for online learning. For example, an online repository of teaching materials like model lesson plans and videos of effective instruction can be made available so that they can be accessed by a wide range of teachers, especially those in remote areas (see chapter 2).

Use data to inform the design and supply of professional development

The TPDC should also use available information about teachers’ learning needs to design courses that are relevant for teachers. For example, if the results from the teacher examinations reveal that classroom management is a need, then resources should be directed towards developing or identifying professional development that strengthens teachers’ skills in this area. Equally, an annual summary from each school’s Teacher Assessment Group based on the learning needs of the teachers’ in their school could be shared with the ministry and analysed to make decisions regarding professional development opportunities. School external evaluation results can also be used to identify teachers’ development needs (see chapter 4).

copy the linklink copied!Policy issue 3.3. Setting high standards for entry to teaching and provide more structured support in the early years

Strong education systems have teachers with robust academic competencies, an aptitude for teaching and motivation to teach. Countries use a combination of different mechanisms to build this kind of teaching population, such as selecting candidates with strong academic skills, establishing high quality initial teacher education and requiring teacher candidates to pass a standardised examination and/or formal probation appraisal in order to become fully qualified. In the past, none of these mechanisms has been present in Georgia. A study of the initial preparation of mathematics teachers in 17 countries found that Georgia had among the least developed quality assurance systems for new entrants to the profession out of all the participating countries (Ingvarson et al., 2013[16]).

Over the past decade, Georgia has introduced wide-ranging reforms to address this concern and raise the bar for entry into teaching. These have included certification examinations at the end of initial teacher education, a new one year consecutive initial teacher education programme and, most recently, a new master’s degree in education. These measures improve the support for and expectations of new entrants. However, more could still be done to enhance the rigour of pathways into the profession and the process for full certification. Interviews with the OECD team revealed that one of the reasons why the latter has not been done so far is that low demand to enter the profession means policy-makers want to avoid further dis-incentivising an already small pool of candidates with additional quality controls for entry. However, this risks that new teachers will continue to lack essential academic and teaching competencies. It also creates the perception that teaching is not a demanding profession for talented school graduates, thus perpetuating the cycle of low quality teaching and contributing to low learning outcomes nationally. Finally, the high teacher numbers in relation to student numbers suggests that they is scope to be more selective about entry into the profession.

Recommendation 3.3.1. Establish more rigorous standards for entry and completion of initial teacher education

Initial teacher preparation needs to provide new teachers with the subject knowledge and the pedagogical skills they need to build students’ competencies. Beginner teachers in Georgia, however, have very low levels of both content knowledge and pedagogical content skills (Ingvarson et al., 2013[16]). Investing in stronger quality assurance mechanisms to improve both the standards of initial teacher programmes and the rigour of licensing requirements should be a priority.

Set a minimum threshold for teacher candidates’ academic knowledge and skills

High-performing education systems select new teachers from among students with the strongest academic performance (Barber and Mourshed, 2007[4]). In contrast, entrants to the four-year initial teacher education programme in Georgia obtain the lowest results in the Unified Entry Examination (UEE) of all tertiary entrants (World Bank, n.d.[12]). To improve this situation, Georgia should require that teacher candidates reach a certain threshold in their UEE subject tests in order to be admitted to a teacher preparation programme. Not only would this guarantee minimum qualifications of the entrants, but it would also help boost the perception of teaching as a rigorous and demanding profession, thus attracting higher quality candidates in the future.

It was repeatedly mentioned to the OECD review team that a key concern with increasing the requirements to enter the teaching profession was that this would disincentive a pool of already small candidates. The limited number of full-time teaching posts, however, indicates that there is currently a surplus of teachers in Georgia. This situation suggests that there is space to be more selective about new entrants to the profession.

Set clear standards for certification, and use these as the key reference point for the design and quality assurance of initial teacher preparation

A number of OECD countries use teacher standards to set out expectations for new teachers. Such standards inform the content and quality assurance mechanisms of initial teacher education programmes, including accreditation requirements and certification examinations for new graduates. The standards are also the main reference for the probation appraisal of new teachers. In Georgia there is ambiguity around the expectations for new teachers that will need to be addressed. At present, Georgian Teaching Standards do not set out the specific expectations for teacher graduates. Rather, new teachers who graduate from an initial teacher education programme and pass the certification examinations are automatically appointed as senior teachers. At this level, they are accorded the same status and pay as practising teachers who have gone through the full performance appraisal process. This situation raises a number of concerns, both in terms of the inconsistency in requirements to reach senior status, as well as the lack of clear expectations to guide initial teacher programme providers and aspirant teachers. By default, the certification examination becomes the main quality reference, though it only captures some of the competencies that a graduate teacher should be expected to master.

This review recommends that, as part of the proposed revisions to the country’s teacher standards (Recommendation 3.1.2), a specific standard be developed for “Graduate” teachers. This standard would set out and illustrate the expectations for teachers upon graduation from an initial teacher education programme and signal the standards expected of any new teacher taking up a teaching post in a Georgian school. In developing such a standard, Georgia could look to similar standards in other countries, such as the Standard for Registration in Scotland or the Graduate Standard in Australia (General Teaching Council for Scotland, 2018[30]; AITSL, 2011[21]). Once developed, the standard should be a key reference for quality assurance processes in higher education, such as accreditation and provider guidelines, as well as for the certification examination. Teachers who pass this exam would be eligible, with initial certification, to teach. They would be expected to gain full and permanent certification, and with this senior status, upon successful completion of a structured probation period and formal probation appraisal. A related salary that graduate teachers would receive during their induction period would also need to be set. The salary would need to be competitive, probably close to the salary for senior teachers, to attract talented graduates.

Establish an attractive and high quality 300-credit programme

Policy-makers in Georgia are concerned that the recent changes to its concurrent initial teacher education programme, in particular its longer length, will discourage potential teacher candidates. However, these changes are important and will provide more time to cover in-depth core knowledge domains and strengthen the teaching practicum. The latter are recognised internationally as being key aspects of effective initial teacher education (OECD, 2019[31]). Five years is also the most common duration of initial teacher education for lower and upper secondary teachers across OECD countries (OECD, 2016[15]). Georgia should continue to develop its new, longer programme and take steps to make it high quality and attractive.

First, the programme should provide new teachers with a strong foundation in all knowledge domains. Initial teacher education should equip new teachers with:

  • content knowledge (i.e. knowledge of specific subject content)

  • pedagogical content knowledge (i.e. knowledge of the teaching and learning processes particular to a subject)

  • general knowledge of pedagogy (i.e. knowledge of teaching and learning that is cross-curricular) (Shulman, 1987[32]).

In the past, initial teacher education in Georgia reportedly provided teachers with strong content knowledge. However, the low levels of new teachers’ content knowledge (Ingvarson et al., 2013[16]) and the pass rates in the certification examinations suggest that this is has become weaker.

Second, the teaching practicum needs to be well-integrated and provide teacher candidates with professional feedback from experienced teachers. The teaching practicum has also been a weak element of initial teacher education programmes in the past. Since it is left to universities’ discretion, it does not always occur (Ingvarson et al., 2013[16]). When the practicum does take place, universities and schools do not always work together closely to provide a meaningful experience for the teacher-students (World Bank, 2014[11]). The teaching practicum should become a mandatory part of the new teacher education programme, as it is in the majority of OECD countries (OECD, 2016[15]).

University-school partnerships should also be strengthened in which universities work closely with schools to explain how the practicum should be organised so teacher candidates benefit fully (e.g. by being paired with an experienced teacher for mentoring and receiving regular feedback following classroom observations). Mentoring a teacher-student might be an explicit responsibility for lead or mentor teachers. Schools that provide good practicum experiences might receive a reward or recognition, such as a training session from university staff.

Third, it is important to provide teacher educators with a good knowledge of modern teaching and learning. At present, Georgia’s teacher educators have not all been trained in modern teaching and learning methods, which affects how new teachers approach education (World Bank, 2014[11]). The country should require that teacher educators regularly update their knowledge, for example, by making this part of the accreditation requirements for teacher programmes. Teacher educators should also be encouraged to model new teaching practices to provide an authentic and coherent model of teaching for their teacher-students. Research suggests that this kind of “role-modelling” can be very effective in helping new teachers understand and apply new teaching techniques (OECD, 2019[31]). It is particularly important in Georgia, where few teacher-students will have experienced modern teaching techniques during their own schooling or their practicum.

Review the quality of the consecutive model

Since most new teachers enter through the consecutive initial teacher education programme, it is important the programme be effective. Given the programme’s shorter length, particular efforts need to be made to provide new teachers with a strong foundation in modern pedagogical skills. As with the concurrent programme, teacher educators should be required to keep their teaching knowledge up-to-date and be encouraged to model effective teaching techniques in their programme delivery. Again, as with the concurrent programme, Georgia needs to systematically include in all consecutive programmes a teacher practicum that gives quality feedback to teacher-students.

As the consecutive programme has now been in place for nearly ten years as the main entry point for new entrants, the ministry should consider evaluating the programme. The evaluation should focus on aspects to be improved and where there is a need to improve alignment with recent changes to the system, such as the new curriculum. The evaluation should also draw on evidence from the certification examinations, programme graduates’ entry into, and retention in the teaching profession, and the results of probation appraisals.

Ensure that the new alternative pathway for entrants is well-targeted and rigorous

Georgia is planning to introduce a two-year in-service programme as an alternative pathway into teaching for mid-career professionals. A number of OECD countries have similar alternative pathways that require entrants to have a number of years of work experience (OECD, 2014[5]). In Georgia, the country might consider focusing the programme on specific fields where there is an identified teaching shortage, such as sciences, foreign languages and mathematics (World Bank, 2014[10]).

It will be important that new entrants to teaching via this pathway still have sufficient opportunity to reflect on teaching practice, alongside time in the classroom. This is important so that new teachers can reflect on what they have seen and experienced in the classroom. Georgia might consider providing study time during or alongside practical teaching experience for this purpose. In all OECD countries with alternative pathways, participants have classroom time for learning and reflecting either before they begin teaching practice or alongside it (OECD, 2014[5]).

Entrants to teaching via the alternative pathway should be required to demonstrate the same levels of competence as all other new teachers. They should be required to pass the existing certification examinations and the probation appraisal recommended in this chapter in order to become fully certified, senior level teachers (see Recommendation 3.3.2). At the same time, Georgia will also need to consider their salaries. While mid-career professionals may be motivated to move into teaching because of intrinsic factors, they are likely to be discouraged if they will experience a significant reduction in salary. Georgia might consider providing financial benefits to recognise mid-career professionals’ previous experience in a different sector.

Recommendation 3.3.2. Introduce an induction period and probation appraisal for new teachers

When Georgia introduced changes for new entrants to teaching in 2007, they included a one-year induction programme. However, the programme was never implemented, in part because of a lack of consensus around the salary that a trainee teacher should receive, and a desire to avoid creating another hurdle to enter a profession where demand is already low. This report recommends that Georgia reconsider introducing an induction period, as well as a probation appraisal for all teachers from initial teacher education programmes. The country’s recent investment in a new professional development scheme also means that there are potential lead and mentor teachers who can take on mentorship functions as part of a new probation period and appraisal.

The support and accountability functions of an induction period and probation appraisal are particularly important in Georgia. The one-year education programme from which the majority of new entrants is drawn is generally regarded to be well-structured and provides graduates with a good foundation to begin their teaching careers. However, one year is a relatively short period of time in which to acquire pedagogical theory, and in particular to receive sufficient opportunities to practice teaching. In England, Scotland and Spain, which also have one-year consecutive programmes, trainee teachers are also required to successfully pass a probation period before they become fully qualified teachers. In England and Scotland the induction period is mandatory (OECD, 2014[5]). An induction period and probation appraisal are important to have in Georgia, given the acknowledged weakness of the concurrent teacher education programme (Recommendation 3.3.1), which will take time to address.

Create a mandatory induction period, with one year as the minimum duration

Formal induction periods are mandatory in around half of OECD countries. They include structured activities, such as mentoring from experienced teachers to help introduce new teachers to the profession (OECD, 2014[5]). The regular, professional advice and feedback that this kind of structured induction provides can help teachers manage the demands of teaching when they reach the reality of the classroom.

When defining its induction period, Georgia should consider one year as a minimum. The country might also consider a longer period, since this would allow trainees to receive additional support, mentoring and coaching. In some systems, such as Boston and Chicago in the United States, probation lasts three or even four years (OECD, 2013[1]). Introducing a probation period would be beneficial in Georgia where, at present, the majority of initial teacher education graduates has only received one year of preparation. A longer induction period can also enable a better decision to be made on a teacher’s potential to be a successful teacher, in particular if the probation appraisal is able to draw on reliable feedback of their daily practice from the trainee’s mentor and school principal.

Provide mentoring for new teachers during their induction period

All trainee teachers entering their first teaching post should receive a mentor. The mentors can be drawn from teachers who have reached lead or mentor level in the career path. Mentors would need to be provided with training on the purpose of their role, which should focus on acting as a “critical friend” who provides formative feedback to help trainee teachers grow professionally. Minimum expectations for a mentor should include:

  • Visiting a trainee teacher’s classroom at least once a month to observe their teaching and giving the trainee formative feedback on their strengths and learning areas.

  • Having at least one informal discussion with the trainee teacher each month. During this discussion, the mentor would ask the trainee to reflect on their progress, and identify any aspects of teaching that they find particularly challenging so that the mentor can work with them to address their learning needs.

The ministry should develop this guidance as well as providing mentors with pointers on what to look for when they conduct the classroom observations and what kinds of questions to ask during their informal conversations with new mentees. The majority of OECD countries (23) also provide new teachers with mentors in their first years of teaching (OECD, 2014[5]), and Georgia might try to build on their experiences (see Box 3.2).

copy the linklink copied!
Box 3.2. Mentor programmes in OECD countries

In Finland, a pilot induction programme called “Osaava Verme” (“Expert Peer Group Mentoring”) was launched in 2008. This programme consists of monthly meetings for teams of new teachers that are facilitated by experienced and trained teachers and supported by the expertise from eight teacher preparation institutions.

In Queensland (Australia), the Mentoring Beginning Teachers (MBT) programme aims to support beginning teachers with mentorship and their schools with increased funding. Beginning teachers are selected for the programme according to the following criteria:

  • be provisionally registered with the Queensland College of Teachers

  • have worked for less than 200 days

  • be employed permanently or on a term-long temporary contract in a Queensland state school.

Principals are given flexibility to decide the mentoring arrangements of beginning teachers according to their school contexts. Annual evaluations of the programme are conducted to ensure schools are properly supporting their beginning teachers.

In Ireland, mentoring is an important part of the National Induction Programme for Teachers. In the framework of this programme, trained Professional Support Teams (PST) and mentors provide personal, professional and pedagogical support to newly qualified teachers during their first year. PSTs are fully certified teachers with minimum of 5 years teaching experience that are nominated by the schools.

In New Zealand, mentoring is part of the induction programme for provisionally certified teachers and aims to provide them with the guidance of an experienced, fully certificated colleague who has received training to give constructive feedback. Although induction and mentoring programmes may be different from one setting to another, essential components must be developed and these are explained in a set of guidelines.

Sources: Driskell, N. (2015[33]), Global Perspectives: Mentoring and Support for New Teachers in Ontario and Finland, NCEE, http://ncee.org/2015/09/global-perspectives-mentoring-and-support-for-new-teachers-in-ontario-and-finland/ (accessed on 29 July 2019); Queensland government (2019[34]), Mentoring Beginning Teachers, https://education.qld.gov.au/about-us/budgets-funding-grants/grants/state-schools/core-funding/mentoring-beginning-teachers (accessed on 29 July 2019); NIPT (n.d.[35]), About NIPT, http://teacherinduction.ie/en/about/about-nipt (accessed on 29 July 2019); Teaching Council of Aotearoa New Zealand (2019[36]), Induction and mentoring, https://teachingcouncil.nz/content/induction-and-mentoring (accessed on 29 July 2019).

Give new teachers in small, rural schools access to mentorship

Introducing trained, experienced mentors for all new teachers in Georgia will take time. At the end of 2018, there were only 420 teachers who have reached lead status, and 26 who had reached mentor levels. In the country’s small, rural schools, there are even fewer teachers, who have reached these levels, with less than 0.1% of teachers in small towns or villages at either lead or mentor level.

Giving new teachers in the country’s small, rural schools guidance and feedback from a mentor is critical. The smaller teaching body in these schools means that new teachers have fewer opportunities to learn professionally from their peers. Also, their teachers are less able to access professional development opportunities than their colleagues in urban areas. At the same time, students from disadvantaged backgrounds and ethnic minority groups are more prevalent in these schools, both of which are associated with lower learning outcomes and higher drop-out rates in Georgia (see chapter 1). There is a critical need to provide new teachers in small rural schools with more support to meet these demands.

One way of meeting this need is to enable lead or mentor teachers to work across multiple schools. Another option is that the new school coaches from the “New School Model” (see chapter 4) could act in this capacity for new teachers in schools where there are not sufficient numbers of experienced teachers. The country’s good technological infrastructure might also provide opportunities for e-mentorship by using telecommunications software that enables video discussion and voice calls.

Introduce an external appraisal at the end of the probation period

By evaluating attributes that cannot be assessed in an examination and requiring classroom experience, a formal process for confirming teaching competence at the end of a probation period helps to ensure that new teachers demonstrate important practical and attitudinal qualities before they are fully confirmed in their post. This is particularly important in Georgia given the lack of rigorous quality assurance mechanisms for initial teacher education programmes, and in particular the absence of a strong practicum. The results from probation appraisal can also be used by the ministry to monitor the quality of initial teacher education programmes and help address areas of recognised weakness.

In most OECD countries with a formal probation appraisal, a combination of evaluators internal and external to the school are involved in taking the decision on whether the trainee teacher meets the requirements for full certification. An individual that is familiar with the trainee’s teaching practice contributes to the decision, such as a principal or mentor. There is also an external evaluator who tends to be drawn from the central, regional or local education authorities or from an externally accredited evaluation body (OECD, 2014[5]). Some externality for the probation decision is important to ensure independence and reliability across different schools and teachers given the high stakes that the decision carries for a new teacher’s career. In Georgia, this responsibility might be given to the external experts that have recently been contracted to undertake external classroom observations of teachers and that this review recommends take on the role of appraising teachers for promotion (Recommendation 3.1.2). As for the promotion appraisal, these external experts would need to receive training and support for their new roles and be made clearly aware of the different expectations for teachers at different levels of the career path.

The external evaluator would appraise the trainee teacher at the end of the probation period according to the standards for a senior teacher. The appraisals might include a classroom observation focusing on teaching practice and an interview with the trainee teacher about what they have learnt during their probation period and what they consider to be their strengths and learning needs. The school principal and the trainee teacher’s mentor would also be asked to complete a form detailing this information. The external evaluator would use this information to make the probation decision.

Develop a clear process for addressing underperformance

Where trainees do not successfully pass their probation on their first attempt, they should be provided with the opportunity to re-enter the probation period. This should be accompanied by specific support tailored to their particular needs. This might include developing a plan with their mentor focused on addressing areas where they have not met the required standards. If a trainee teacher is not successful after a second attempt, they will no longer be eligible to teach in a school.

copy the linklink copied!Policy issue 3.4. Attracting new teachers and motivating them to succeed

Georgia is looking to transform its teaching workforce to make it highly capable and motivated. While facilitating such broad change is beyond the scope of this review, this Policy issue discusses briefly several factors that currently hinder the development of a professional and qualified teaching workforce. These include a large number of older teachers who are relatively less interested in developing themselves but continue to teach, the perception of teaching as a less prestigious career and financial incentive schemes that are misaligned with the actual causes of lower teacher earnings. These concerns will need to be considered as part of overall strategic planning to improve the state of education in Georgia (see chapter 5).

Recommendation 3.4.1. Encourage renewal of the teaching profession

The presence of a large share of older teachers who are less motivated to engage in career development is impeding the success of the educational reforms introduced by the ministry. By remaining in their positions, these teachers also reduce opportunities for talented young graduates to enter the profession. Devising a resourcing strategy that considers the needs of established teachers and persons who wish to enter teaching will be necessary to effectively renew the profession.

Establish a mandatory retirement age

A challenge to modernising the teaching profession in Georgia is the fact that many teachers are over the retirement age of 60. A lot of these persons are motivated to keep working because their salaries were low for most of their career and they were allowed by the government to stay in their positions in order to continue earning their salary while also collecting pension payments, which are likewise low. Since certification is currently not mandatory, many of these older, uncertified teachers have little incentive to invest in the preparation needed to meet certification requirements in the future and engage with the new pedagogical approaches that the professional development scheme encourages teachers to adopt.

To address this situation, this review first recommends that Georgia introduce a mandatory retirement age for teachers. This will prevent the current circumstances from becoming worse in the future (the share of teachers over 60 grew from roughly one-fifth in 2013 to over one-fourth in 2016). This measure will have to be phased in so teachers have ample time to prepare and to avoid a sudden loss of a quarter of the profession. Teachers already over retirement age would leave the profession after the requirement is phased in. For teachers who are approaching retirement age (e.g. within four to five years), they would be given the choice of engaging with the professional development scheme so they can continue teaching until retirement age, or leaving the profession.

It will be important to consider the social impact of mandatory retirement for older teachers. These persons have worked for a long time under difficult financial circumstances and are still working in their positions with the expressed consent of the government. If they do not pass or engage with the certification requirement, they will need to be supported as they exit the teaching profession. Supporting measures could include a one-time financial bonus to off-set the potential loss of income that these individuals will face if they leave.

Attract talented graduates into the profession

Attracting the most able school graduates to enter teaching will take time and require a coordinated approach across a number of areas. Some of the actions detailed throughout this report, such as raising the threshold to enter teaching and ensuring that all teachers demonstrate minimum competencies (Recommendation 3.3.1 and Recommendation 3.3.2) will help recruit talented teaching candidates. In addition, Georgia might also consider introducing incentives to encourage high-performing high school graduates to apply to become teachers. For example, applicants with high marks in the UEE might receive a scholarship to enrol in initial teacher education. Any increase in entry level teacher salaries should also be well communicated to potential teaching candidates.

A communications campaign could help advertise new incentives and improve the overall prestige of teaching. Many countries have organised similar campaigns to address the low demand to enter the teaching workforce (see Box 3.3). In order to encourage individuals with strong intrinsic motivation to teach, such a campaign might focus on the essential role that teachers have in students’ lives and the development of Georgia. The campaign might combine national advertisements on television, in cinemas, the press and include a website and leaflets to provide further information.

copy the linklink copied!
Box 3.3. A national campaign for teaching from the United Kingdom

In the United Kingdom, a national marketing campaign called “Your Future | Their Future” aimed at teacher training recruitment was launched in 2014 by the Department for Education. The campaign included:

  • creating an official website, “Get Into Teaching”, which disseminated information and advice on teacher training and on the teaching profession

  • television advertising, social media channels and online videos

  • recruitment events where higher education institutions, subject associations (such as the Institute of Physics) and a network of 600 teaching schools provided guidance to prospective teachers.

The government has also been offering financial incentives to attract more of the best graduates to teach in-demand subjects.

Sources: GOV.UK (2014[37]), Your future their future: new teacher recruitment campaign, www.gov.uk/government/news/your-future-their-future-new-teacher-recruitment-campaign (accessed on 29 July 2019); (Lane et al., 2019[38]), Your Future, Their Future impact: initial findings - Main report, Department for Education, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file /768485/Teacher_marketing_evaluation_2019_-_initial_findings_Main_Report.pdf (accessed on 29 July 2019).

Purposefully allocate new teachers to high need subject areas and geographic locations

While there is currently a surplus of teachers overall in Georgia, ministry officials acknowledge there are shortages in some specific subjects and in the more remote areas of the country. Therefore, as Georgia becomes more selective about new entrants to the profession (see Recommendation 3.3.1) and actively recruits new teachers, it is recommended that these new, talented teachers should be purposefully allocated to specific subjects and to certain schools. It is important to consider this recommendation in light of the expected departure of older teachers in the short term. As well as thinking about the impact of teacher retirement on overall teaching numbers, Georgia should consider if the departure of many older teachers is likely to disproportionately affect demands in specific subjects or parts of the country. How new teachers are allocated can be continuously adjusted in response to these teacher retirement trends.

Recommendation 3.4.2. Review planned adjustments to teacher salaries to make them impactful and educationally valuable

While teachers in Georgia historically have had lower salaries than international benchmarks, the introduction of career pathways and associated salary scale revisions have helped bring Georgian teacher pay scales in line with OECD norms. In fact, teachers at all levels except the practitioner level currently earn more, in relative terms, than their peers internationally. The reasons that, despite these changes, Georgian teachers’ earnings are still considered low are because most teachers are at the lesser paid practitioner level and also most only work part-time.

The ministry has recently announced an increase to teacher salaries, but changing the salary scale will not necessarily increase the incomes of most teachers because of the aforementioned reasons. In fact, such measures could undermine concurrent efforts to professionalise the teaching workforce by, for example, removing incentives for teachers to progress through the pathways. Therefore, it is recommended that future adjustments to teacher salaries be carefully reviewed so they actually impact teachers’ earnings while also helping to improve the quality of education that students receive.

Limit teacher salary scale increases to practitioner teachers and consider alternative methods to raise teachers’ earnings

According to the current pay and career structure, only practitioner teachers have salaries levels that are low by national and international standards (see Table 3.2). Any increases to the formal teacher salary scale, therefore, should only affect the practitioner level. Beyond this, this review recommends that Georgia should expand professional support so more teachers can move along the career path and therefore benefit from higher salaries (see Recommendation 3.3.2). Another way to make effective use of additional funding for teacher pay, which would also help improve educational equity, is to provide financial incentives to teachers who work in hard-to-staff schools. Similarly, working in such environments could be considered positively in teachers’ appraisal for promotion.

Consider options to reduce the high share of part-time teachers

The large share of part-time teachers who do not earn a full salary contributes to the overall low level of teacher earnings. Over time, enforcing a retirement age and creating more rigorous standards for entry will reduce the overall teacher numbers. This will create more full-time teaching posts for in-service teachers and reduce the number of part-time teachers.

In the short term, the ministry should consider introducing opportunities for qualified part-time teachers to take on additional non-teaching activities and increase their working time. For example, part-time lead or mentor teachers could be given additional mentoring or professional development activities in their school or across other local schools (see Recommendation 3.2.2). These roles should be explicitly set out in the revised teacher standards (see Recommendation 3.1.2). These measures would not only increase teacher earnings, but their extra time spent in schools (along with the added income) would also discourage them from providing private tutoring (see Chapter 2).

copy the linklink copied!Recommendations

copy the linklink copied!

Policy issue

Recommendations

Actions

3.1. Apply minimum standards for teaching and encourage the development of higher teaching competencies

3.1.1. Support all teachers to meet minimum standards

Clearly communicate the examinations’ role

Ensure that the certification examinations effectively assess essential teaching knowledge and skills

Support existing teachers to master essential knowledge and skills

Encourage each school to make it a priority that all teachers reach senior status

3.1.2. Re-focus the teacher professional development scheme on demonstrating higher levels of teaching competencies

Make teacher standards the main reference for promotion

Focus promotion appraisal on authentic evidence of teaching practice

Strengthen the role of independence and professional competence in promotion decisions

3.2. Support teachers to develop professionally throughout their career

3.2.1. Focus the new regular appraisal on student learning and providing feedback for teachers’ professional learning

Introduce guidelines on a set of simple steps that schools can take to encourage regular appraisal practices

Determine the evaluators

Develop national guidance for evaluators on how to collect and review evidence of teaching quality

Support evaluators to provide useful developmental feedback

3.3.2. Ensure teachers have access to high quality professional development

Require that all teachers undertake professional development

Sustain the programmes and capacity that has been built up through G-PriEd and MCC

Ensure that training is relevant and high quality

Use data to inform the design and supply of professional development

3.3. Set high standards for entry to teaching and provide more structured support in the early years

3.3.1. Establish more rigorous standards for entry and completion of initial teacher education

Set a minimum threshold for teacher candidates’ academic knowledge and skills

Set clear standards for certification, and use these as the key reference point for the design and quality assurance of initial teacher preparation

Establish an attractive and high quality 300 credits programme

Review the quality of the consecutive model

Ensure that the new alternative pathway for entrants is well-targeted and rigorous

3.3.2. Introduce an induction period and probation appraisal for new teachers

Create a mandatory induction period, with one year as the minimum duration

Provide mentoring for new teachers during their induction period

Ensure that new teachers in small, rural schools receive adequate support

Introduce an external appraisal at the end of the probation period

Develop a clear process for underperformance

3.4. Attracting new teachers and motivating them to succeed

3.4.1. Encourage renewal of the teaching profession

Establish a mandatory retirement age

Attract talented graduates into the profession

Purposefully allocate teachers to high need subject areas and geographic locations

3.4.2. Review planned adjustments to teacher salaries to make them impactful and educationally valuable

Limit teacher salary scale increases to practitioner teachers and consider alternative methods to raise teachers’ earnings

Consider options to reduce the high share of part-time teachers

References

[21] AITSL (2011), “Australian Professional Standards for Teachers”, https://www.aitsl.edu.au/docs/default-source/apst-resources/australian_professional_standard_for_teachers_final.pdf (accessed on 2 March 2018).

[4] Barber, M. and M. Mourshed (2007), How the world’s best-performing school systems come out on top, McKinsey & Company, https://www.mckinsey.com/industries/social-sector/our-insights/how-the-worlds-best-performing-school-systems-come-out-on-top (accessed on 4 January 2019).

[23] Cole, P. (2012), Aligning professional learning, performance management and effective teaching Introduction 3, Centre for Strategic Education, Victoria, https://ptrconsulting.com.au/wp-content/uploads/2018/05/Aligning-professional-learning-performance-management-and-effective-teaching.pdf (accessed on 26 January 2019).

[19] D’Agostino, J. and S. Powers (2009), “Predicting Teacher Performance With Test Scores and Grade Point Average: A Meta-Analysis”, American Educational Research Journal, Vol. 46/1, pp. 146-182, http://dx.doi.org/10.3102/0002831208323280.

[20] Danielson, C. (2013), The Framework for Teaching: Evaluation Instrument, 2013 Edition, Danielson Group, http://www.loccsd.ca/~div15/wp-content/uploads/2015/09/2013-framework-for-teaching-evaluation-instrument.pdf (accessed on 10 December 2018).

[24] Darling-Hammond, L., M. Hyler and M. Gardner (2017), Effective Teacher Professional Development, Learning Policy Institute, Palo Alto, CA, https://learningpolicyinstitute.org/sites/default/files/product-files/Effective_Teacher_Professional_Development_REPORT.pdf (accessed on 10 January 2019).

[29] Darling-Hammond, L. and R. Rothman (eds.) (2011), Teacher and Leader Effectiveness in High-Performing Education Systems Stanford Center for Opportunity Policy in Education, Alliance for Excellent Education and Stanford Center for Opportunity Policy in Education, Washington DC, https://edpolicy.stanford.edu/sites/default/files/publications/teacher-and-leader-effectiveness-high-performing-education-systems.pdf (accessed on 16 April 2018).

[22] Dewey, J. (1938), Experience and Education, Macmillan, New York.

[33] Driskell, N. (2015), Global Perspectives: Mentoring and Support for New Teachers in Ontario and Finland, NCEE, http://ncee.org/2015/09/global-perspectives-mentoring-and-support-for-new-teachers-in-ontario-and-finland/ (accessed on 29 July 2019).

[30] General Teaching Council for Scotland (2018), Professional Standards, http://www.gtcs.org.uk/professional-standards/professional-standards.aspx (accessed on 6 September 2018).

[8] Goe, L., K. Biggers and A. Croft (2012), Linking teacher evaluation to professional development: Focusing on Improving Teaching and Learning, National Comprehensive Center for Teacher Quality, Washington DC, https://gtlcenter.org/sites/default/files/docs/LinkingTeacherEval.pdf (accessed on 20 January 2019).

[37] GOV.UK (2014), Your future their future: new teacher recruitment campaign, https://www.gov.uk/government/news/your-future-their-future-new-teacher-recruitment-campaign (accessed on 29 July 2019).

[16] Ingvarson, L. et al. (2013), TEDS An Analysis of Teacher Education Context, Structure, and Quality-Assurance Arrangements in TEDS-M Countries Findings from the IEA Teacher Education and Development Study in Mathematics (TEDS-M), International Association for the Evaluation of Educational Achievement (IEA), Amsterdam, https://www.iea.nl/fileadmin/user_upload/Publications/Electronic_versions/TEDS-M_Findings.pdf (accessed on 2 February 2019).

[9] Kitchen, H. et al. (2017), Romania 2017, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264274051-en.

[38] Lane, M. et al. (2019), Your Future, Their Future impact: initial findings - Main report, Department for Education, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/768485/Teacher_marketing_evaluation_2019_-_initial_findings_Main_Report.pdf (accessed on 29 July 2019).

[3] Louden, W. (2000), “Standards for Standards: The Development of Australian Professional Standards for Teaching”, Australian Journal of Education, Vol. 44/2, pp. 118-134, http://dx.doi.org/10.1177/000494410004400203.

[27] MCC (2018), Millennium Challenge Account-Georgia Monitoring and Evaluation Plan Compact II, https://assets.mcc.gov/content/uploads/georgia-compact-ii-me-plan.pdf (accessed on 22 January 2019).

[14] Ministry of Education/UNICEF (2015), “Country Background Report: Georgia”.

[18] MoESCS (n.d.), Forms for teacher appraisal, MoESCS.

[17] National Center for Educational Quality Enhancement (2017), “Teacher Education - Field Specification of Higher Education”.

[35] NIPT (n.d.), About NIPT, http://teacherinduction.ie/en/about/about-nipt (accessed on 27 January 2019).

[31] OECD (2019), A Flying Start: Improving Initial Teacher Preparation Systems, OECD Publishing, Paris, https://dx.doi.org/10.1787/cf74e549-en.

[13] OECD (2019), TALIS 2018 Results (Volume I): Teachers and School Leaders as Lifelong Learners, TALIS, OECD Publishing, Paris, https://dx.doi.org/10.1787/1d0bc92a-en.

[6] OECD (2017), Do new teachers feel prepared for teaching?, OECD, Paris, https://dx.doi.org/10.1787/980bf07d-en.

[28] OECD (2017), Education at a Glance 2017: OECD Indicators, OECD Publishing, Paris, http://dx.doi.org/10.1787/eag-2017-en.

[15] OECD (2016), PISA 2015 Results (Volume II): Policies and Practices for Successful Schools, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264267510-en.

[7] OECD (2015), Education at a Glance 2015: OECD Indicators, OECD Publishing, Paris, https://dx.doi.org/10.1787/eag-2015-en.

[5] OECD (2014), Education at a Glance 2014: OECD Indicators, OECD Publishing, Paris, http://dx.doi.org/10.1787/eag-2014-en.

[1] OECD (2013), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264190658-en.

[34] Queensland Government (2019), Mentoring Beginning Teachers, https://education.qld.gov.au/about-us/budgets-funding-grants/grants/state-schools/core-funding/mentoring-beginning-teachers (accessed on 29 July 2019).

[2] Santiago, P. et al. (2013), Teacher Evaluation in Chile 2013, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264172616-en.

[32] Shulman, L. (1987), “Knowledge and Teaching: Foundations of the New Reform”, Harvard Educational Review, Vol. 57/1, pp. 1-23, http://dx.doi.org/10.17763/haer.57.1.j463w79r56455411.

[36] Teaching Council of Aotearoa New Zealand (2019), Induction and mentoring, https://teachingcouncil.nz/content/induction-and-mentoring (accessed on 29 July 2019).

[26] USAID (2018), Georgia Primary Education Project (G-PriEd), https://chemonics.com/wp-content/uploads/2018/07/Georgia-Primary-Education-Project-G-PriEd-Final-Report.pdf (accessed on 26 October 2018).

[25] USAID (2016), USAID/Georgia External Impact Evaluation of the Georgian Primary Education (G-Pried) Project, USAID, https://pdf.usaid.gov/pdf_docs/pa00m256.pdf (accessed on 6 January 2019).

[11] World Bank (2014), Georgia - Technical assistance to support preparation of education sector strategy : education sector policy review - strategic issues and reform agenda, http://documents.worldbank.org/curated/en/505151488895322292/Georgia-Technical-assistance-to-support-preparation-of-education-sector-strategy-education-sector-policy-review-strategic-issues-and-reform-agenda (accessed on 21 June 2018).

[10] World Bank (2014), SABER Country Report Georgia - Teachers, World Bank Group, http://documents.worldbank.org/curated/en/788461468198234275/pdf/105634-WP-ADD-SERIES-PUBLIC-SABER-Teachers-Georgia-CR-Final-2014.pdf (accessed on 9 January 2019).

[12] World Bank (n.d.), A Review of Teacher Policy Reforms in Georgia - A Case Study, World Bank.

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

https://doi.org/10.1787/94dc370e-en

© OECD 2019

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.