copy the linklink copied!Chapter 2. Improving learning outcomes and equity through student assessment

The curriculum and learning standards communicate what students are expected to know and be able to do

It is important to have common expected learning outcomes against which students are assessed to determine their level of learning and how improvement can be made (OECD, 2013[1]). Expectations for student learning can be documented and explained in several ways. Many countries define them as part of national learning standards. Others integrate them into their national curriculum frameworks (OECD, 2013[1]).

While most reference standards are organised according to student grade level, some countries are beginning to organise them according to competency levels (e.g. beginner and advanced), each of which can span several grades (New Zealand Ministry of Education, 2007[2]). This configuration allows for more individualised student instruction, but requires more training for teachers to properly understand and use the standards when assessing students.

Effective classroom assessment requires the development of teachers’ assessment literacy

Assessment is now seen as an essential pedagogical skill. In order to use classroom assessment effectively, teachers need to understand how national learning expectations can be assessed – as well as the students’ trajectory towards reaching them ‒ through a variety of assessments. Teachers need to know what makes for a quality assessment – validity, reliability, fairness – and how to judge if an assessment meets these standards (see Box 2.2). Feedback is important for students’ future achievement, and teachers need to be skilled in providing constructive and precise feedback.

Many OECD countries are investing increasingly in the development of teachers’ assessment literacy, beginning in initial teacher education. In the past, teachers’ initial preparation in assessment was primarily theoretical, but countries are now trying to make it more practical, for example, by emphasising opportunities for hands-on learning where teachers can develop and use different assessments. Countries encourage initial teacher education providers to make this shift by incorporating standards on assessment in programme accreditation requirements and in the expectations for new teachers in national teacher standards.

It is essential that teachers’ initial preparation on assessment be strengthened through on-going, in-school development. Changing the culture of assessment in schools – especially introducing more formative approaches and performance-based assessments, and using summative assessments more effectively – requires significant and sustained support for teachers. Continuous professional development such as training on assessment and more collaborative opportunities when teachers can share effective assessment approaches provides vital encouragement. Pedagogical school leaders also play an essential role in establishing a collaborative culture of professional enquiry and learning on assessment.

Finally, countries need to invest significantly in practical resources to ensure that learning expectations defined in national documents become a central assessment reference for teachers and students in the classroom. These resources include rubrics that set out assessment criteria, assessment examples aligned to national standards and marked examples of student work. Increasingly, countries make these resources available on line through interactive platforms that enable teachers to engage in the development of standards, which facilitates a greater feeling of ownership over the resources and makes it more likely that they will be used.

Georgia has a well-established national curriculum that has undergone several revisions

In 2005, Georgia introduced the first national curriculum since independence, which determined academic content, established desired learning outcomes and distributed teaching hours across the week for all subjects and grades (MoESCS, 2018[10]). A revised curriculum was introduced in 2011 and another in 2018-19. Changes introduced by the third generation curriculum currently apply only to primary education (Grades 1-6). They will be applied to basic education (Grades 7-9) in 2019-2020.

Changes implemented in the 2011 and 2018-19 versions of the curriculum emphasise a more holistic approach to learning (MoESCS, 2016[12]). Education is seen not only as academic in nature, but also as a means to build student’s social and emotional skills and to develop critical, creative and responsible citizens (MoESCS, n.d.[14]). For example, curriculum goals for general education include that students sustain healthy lifestyle habits, understand and appreciate cultural diversity and are able to make independent decisions.

Georgia’s learning standards emphasise competences

Georgia has developed learning standards as part of the national curriculum. Similar to many OECD countries’ (Peterson et al., 2018[15]), Georgia’s learning standards emphasise the development of cognitive and non-cognitive competencies in real-life contexts (e.g. expressing spatial information using different resources) rather than acquiring factual knowledge (e.g. memorising the capitol of a country). The standards are organised according to subject and student grade level (recently changed to student stages).

Georgia is moving towards a stage-based curriculum, but understanding of this change varies

Unlike previous versions, the 2018-19 curriculum is organised around stages, meaning that there is one set of expected learning outcomes for Grades 1-6 and another for 7-9 (MoESCS, 2018[10]). This significant and innovative development recognises that students in the same grade might differ in acquired competences and allows teachers and schools to exercise greater flexibility in adapting the curriculum to students’ individual needs, which can lead to improved student outcomes (Dumont, Istance and Benavides, 2010[16]).

However, in countries that have transitioned to similarly structured curricula, such as New Zealand and Australia, teachers have generally had a very strong understanding of student progression and assessment. They have also been provided with immense support to implement the new curricula. In Georgia, there is concern that the rationale behind this kind of curriculum reform might not be well understood by all stakeholders. Interviews conducted by the OECD review team revealed that teachers and principals found that proposed changes to the curriculum were unclear and that many were not sure what a stage-based learning outcome was. Some were completely unaware that the reform was occurring.

Formative assessment practices are encouraged by policy but are not widely used

Like many OECD countries, Georgia has made efforts to integrate more formative assessment practices into its classrooms. The national curriculum, for example, mandates that teachers provide only oral or written feedback and not summative marks until grade 5 (MoESCS, 2016[12]). The government has also undertaken projects in partnership with international donors to develop the capacity of teachers to employ more formative assessment methods. This includes the development of 13 formative assessment tools, such as the E-Assess, an online diagnostic assessment software, under the Georgia Primary Education Project (G-PriEd). Teachers and the Ministry of Education, Science, Culture and Sport (MoESCS) staff reported to the OECD review team that they considered E-Assess to be very useful and customisable, helping them to better adapt instruction to student needs (USAID, 2018[17]).

Despite these significant efforts, formative assessment is not widely understood or practiced in Georgian classrooms. OECD interviews suggest that teachers tend to interpret formative assessment as simply providing a description of student progress instead of a numeric mark, as opposed to using assessment information for improving student learning. As part of a survey administered for this review, 43% of teachers indicated that they “never or almost never” or only “a few times” provide written feedback on student work, further suggesting that key formative assessment activities are not being practiced.

Teachers are not effectively prepared to assess students

Teachers in Georgia are not always trained in how to properly assess students. Internationally, 13% of teachers across 31 jurisdictions surveyed in the Teacher and Learning International Survey (TALIS) 2018 indicated that they have a high developmental need in student evaluation and assessment practices. In Georgia, over 25% of teachers indicated a need in this area (OECD, 2019[18]). A large reason that this need exists in Georgia is the lack of training that teachers receive in assessment during initial teacher preparation (ITP) programmes. Many of these programmes have low entrance requirements and are not regarded as particularly rigorous. In fact, because of the age of the Georgian teacher population, almost one-third did not receive any ITP as they entered the profession before formal preparation was introduced (see chapter 3 for a more detailed discussion of ITP).

Student marks are still the focus of classroom assessment but are not reliable

In grades 1 through 4, and partially in grade 5, students only receive descriptive feedback about their achievements. Starting in grade 5, students receive summative marks on a scale of one (lowest) to 10 (highest). Marks of one through four are considered unsatisfactory while all others are passing marks. Students who receive an unsatisfactory mark must repeat that grade, but in practice this occurs very rarely. According to PISA 2015, only 2% of 15 year-old students in the country have ever repeated a grade, compared to 12% on average across OECD countries (OECD, 2016[19]).

In Georgia, students’ summative marks represent the focal point of classroom assessment and the emphasis on using marks to label students distracts from focusing on student learning. A G-PriEd report found that teachers were reticent to use assessment methods that did not produce numeric grades and did not see the value of checking a student’s level of understanding (USAID, 2018[17]). Conversations with university faculty confirmed that classroom instruction is often oriented towards achieving certain marks and understanding the curriculum is not a primary concern of teachers. These persons suggested that being assessed in this manner inhibits students from embracing the types of autonomous learning approaches that are critical to success at higher levels of education.

Despite being the focus of classroom assessment, the marks that teachers provide are not reliable measures of student achievement. Almost all students receive passing marks, though national results on PISA indicate that many students actually struggle to reach baseline levels of competency. One reason for the lack of reliable marking is that teachers lack resources that can help them gauge student achievement with respect to the national learning standards. Another is that societal focus on grades and ranking puts pressure on teachers to provide high marks, even if students do not demonstrate a minimum level of performance.

Reporting of classroom assessment is inconsistent

How student marks are reported in Georgia is not always consistent. Schools are required to issue a report card to document student progress, but Georgia does not have a national report card template that schools should follow. As a result, report cards are not standardised across schools and contain different information. This is unfair for students, who receive different amounts and types of information about their own learning. It can also create difficulties when students change schools as a student’s teachers might not be able to interpret the student’s previous report cards and understand where the student is in their learning. Georgia also lacks national guidelines around how students’ performance should be communicated to parents, creating a situation in which some parents rarely see evidence of their students’ performance (MoESCS, 2018[10]).

Several national assessments are conducted by NAEC using external funding

Since 2015, the National Assessment and Examinations Centre (NAEC) has conducted regular sample-based student assessments for maths and sciences in grade 9 and a census-based assessment for Georgian as a second language in non-Georgian speaking schools in grade 7 (MoESCS, 2018[10]). The three assessments include multiple-choice items and open-ended questions. The results of these are not publicly disclosed and are only made accessible to NAEC staff for the purposes of informing policy-making (MoESCS, 2018[10]).

Georgia’s national assessments have been largely funded by the Millennium Challenge Corporation (MCC) under an initiative to improve education in the Science, Technology, Engineering, and Mathematics (STEM) subjects. MCC’s funding is phasing out starting in 2019. Georgia has started developing a long-term strategy to continue administering national assessments (see chapter 5) and is considering developing standardised diagnostic assessments in primary and secondary school. These latter assessments would not focus on system monitoring, but on helping teachers identify their students’ strengths and weaknesses so they can better tailor their instruction.

From 2011 to 2019 graduation from upper secondary education was based on results in a computer adaptive test

The SGE was introduced in 2011 to improve student attendance (absenteeism in upper secondary education was previously a concern) and increase school accountability (Bakker, 2014[20]). Given the large number of subjects assessed (eight), the SGE was taken by students over the course of two years. At the end of grade 11, students were tested in geography, biology, chemistry and physics. In grade 12, they took tests in their mother tongue, history, mathematics and a foreign language.

The SGE was administered using computer adaptive testing (CAT) (Bakker, 2014[20]). Through an item response theory model, students were presented with questions that varied in difficulty depending upon their previous responses. Students continued with the test until a student’s level of achievement could be determined at a specified level of statistical certainty, meaning that different students did not answer the same questions, nor did they spend the same amount of time taking each test. All questions were multiple-choice.

Passing the SGE in all subjects was necessary to graduate from upper secondary school. Roughly eight out of ten test-takers were successful, although passing rates varied significantly across the country (World Bank, 2014[21]), from over 90% in Tbilisi to 44% in Marneuli (Kvemo Kartli).

The SGE was considered trustworthy, but did not adequately certify student achievement in relation to the curriculum

The CAT format of the SGE helped instil trust in the examination. Because results were returned to students immediately, there was no concern about the integrity of the results and the exam did help to improve student attendance because attending class to prepare for the exam served as significant incentive. From an international perspective, however, using the CAT format for an upper secondary exit examination is unusual. The format does not guarantee that students must demonstrate knowledge and skills across the breadth of the curriculum because it classifies all questions from the same subject along a single continuum of difficulty, regardless of competence area. This was problematic because it meant that students were certified without necessarily showing that they have acquired basic understanding knowledge of the entire curriculum.

Since 2005 selection into higher education has been based on a standardised examination

The UEE, established in 2005, is a standardised examination used for selection into higher education institutions and scholarship allocation (NAEC, n.d.[22]). After a recent change in 2019, students must take three compulsory subjects at the end of grade 12 - Georgian language, a foreign language and either mathematics or history - along with an elective subject. This elective subject is selected according to the requirements of university programs to which the students wish to apply.

Starting in 2016, the UEE questions were presented to students electronically, though students still complete their answers on paper. Questions include multiple-choice and open-ended items. The tests are not strictly based on the curriculum. The UEE is only offered in Georgian, with the exception of the elective general aptitude test, which is offered in ethnic minority languages. Students from ethnic minority schools who take the UEE only take this test and, if successful, then enrol in a university preparation program. Upon completing the preparation program, they begin university studies.

To create items for the UEE, NAEC first sends an examination programme to MoESCS for review. Part of the approval process is determining the extent to which it is aligned with the curriculum. Previously, this part of the review was performed by the curriculum department, which was recently eliminated. It is unclear who performs this review now. After approval is granted by MoESCS, NAEC convenes subject matter groups to begin writing items. Despite this review process, interviews with Ministry officials, university faculty and teachers all suggested that the final items for UEE are not always closely aligned with the curriculum.

The UEE strengthened integrity in higher education admission

In the 1990s and early 2000s, university entrance requirements were not standardised across Georgian higher education institutions. This created space for practices such as bribing officials in order to influence student selection, which hindered equity of access and the capacity of universities to meet expected academic standards (Orkodashvili, 2012[23]).

In 2005, amidst government reform to tackle corruption, Georgia introduced the UEE. The examination was applied to all public institutions and measures were put in place to prevent cheating (test-takers’ names were coded for anonymity and administrations were under camera surveillance). Stakeholders acknowledged that this examination had a key role in increasing transparency and fighting corruption in university admission in Georgia in the following decade. Moreover, it is seen to have improved the participation of ethnic minorities in higher education and fostered greater social cohesion (Orkodashvili, 2012[23]).

Numerous high-stakes examinations had unintended negative effects

High-stakes examinations can have the consequence of creating a backwash effect, which pressures teachers to “teach to the test”, as opposed to focusing on individual student learning, and pressures students to focus primarily on preparing for the exams by seeking tutoring opportunities (OECD, 2013[1]). These effects are particularly pronounced in Georgia. Research shows that 60% of Georgian students enrolled in grade 12 in 2014 registered for private tutoring (World Bank, 2014[21]) and 39% of households with a school-age child in 2016 had a private tutor (Bregvadze, 2012[24]). Georgia’s large private-tutoring sector is also an equity-related concern. Nearly nine in ten advantaged students attend private tutoring, compared to only 36% of their disadvantaged peers (World Bank, 2014[21]).

Several factors contribute to these backwash effects. The large number of subjects tested by the SGE created significant pressure and students who wished to attend university had to take two examinations. Furthermore, because final UEE items are not always strictly aligned with the curriculum, students are unable to prepare for it through regular classroom instruction, which further motivates them to seek private tutoring.

Major changes to examinations were announced in 2019

In 2019, MoESCS announced that significant changes would be made to Georgia’s examinations. In particular, the SGE would be eliminated due to the backwash effects it was exerting on students, teachers and schools. The implementation of this policy was immediate, with graduating students in 2019 being the last cohort to take the SGE. The UEE was maintained but, as mentioned previously, its required subjects were changed to make it more flexible. While discussions have occurred regarding what should replace the SGE, no decisions have been made. In the meantime, certification from upper secondary education will be based on student grades and, depending upon school-level decision-making, internal examinations. There have been preliminary discussions about NAEC strongly supporting schools in developing these internal examinations.

VET upper secondary schools use an examination to select students

The Vocational Education and Training (VET) sector in Georgia enrols less than 2% of upper secondary students (OECD, 2016[9]). A particular concern with the sector is what has been termed “dead ends,” in which students who complete VET upper secondary education are unable to enrol in general education tertiary programmes.

While underdeveloped, there is still demand for VET education and demand often exceeds the very limited supply. In response, NAEC began administering a VET admissions examination in 2013 to select students into in public institutions (passing an examination is not necessary for private VET schools, unless required by individual institutions). The exam assesses Georgian language and literature, mathematics and general aptitude. Students who achieve a 25% of the exam have their student fees fully paid for by the government (Livny, Eric; Stern, Paul, Maridashvili, Tamta; Tandilashvili, 2018[25]).

NAEC oversees examinations and assessments in Georgia

The National Assessment and Examinations Centre (NAEC), established in 2002, was responsible for developing and administering the SGE and currently administers the UEE, VET examination and the national assessments in grades 7 and 9. NAEC also facilitates Georgia’s participation in international assessments, such as PIRLS, TIMSS and PISA.

NAEC is known for its strong technical capacity. Its team of over 200 people (as of early 2018) includes experts in statistics, psychometrics and information technology. Interviews with national stakeholders show that NAEC is recognised in Georgia as a trusted and competent institution, given the credibility of the administration of the examinations as well as the timeliness with which SGE results were released (Bakker, 2014[20]). Reports suggest that the success of the SGE’s introduction was partly due to its association with NAEC and the credibility of the institution (Bakker, 2014[20]). In 2018, changes in NAEC’s leadership led to some staff turnover and, as a result, some technical capacity is being rebuilt.

Teacher Professional Development Centre supports teachers’ assessment literacy

The Teacher Professional Development Centre (TPDC) is a body at arms-length from the Ministry and is responsible for developing teachers in Georgia. Included in its remit is developing teachers’ assessment capacity. To this end, it has provided a professional development module on learning and assessment strategies for over 7 000 teachers between 2016 and 2018 and works closely with programmes such as G-PriEd to further develop teachers (TPDC, 2018[26]).

Use the new curriculum as a policy lever to encourage the use of formative assessment

The recent introduction of a new curriculum represents an opportunity to engage teachers in new pedagogical methods, especially those related to formative assessment. Teachers need motivation to change their practices and confidence to believe that they can do it. A demanding and modern curriculum can help make the case for formative assessment and act as a catalyst to support teachers in changing their practices.

Introducing a new curriculum necessitates addressing several aspects of instruction, such as how to help students develop different skills and how to assess if students have acquired those skills. It is important that these instructional aspects be integrated into the curriculum, in relation to its content and expected outcomes, as opposed to being add-ons. This type of integration weaves an assessment structure throughout the curriculum and reinforces the idea that formative assessment is inextricably tied to student learning and that its practice is mandatory. Box 2.3 explains how Norway explicitly integrates formative assessment into its national curriculum.

Georgia can similarly build in formative assessment into the curriculum by, for example, specifying what types of feedback teachers should provide (see Table 2.3). This information could be adapted and expressed in curricula for different domains, units of study and stages of education (Nicol and Macfarlane-Dick, 2007[29]). Importantly, the curriculum and accompanying materials then need to be made available to all teachers. An online portal would be an ideal solution so teachers can access the curriculum and related instructional resources in one place (Recommendation 2.2.1).

Consider the introduction of portfolio assessment in the curriculum to anchor more formative assessment practices in the classroom

As well as establishing an assessment structure as part of the curriculum, it is important to give teachers guided tasks that they can use in their classrooms to embed key formative assessment practices. There are several ways this can be done, but a particularly effective method is the use of student portfolios, which are collections of a student’s selected work that demonstrates evidence of the student’s progress and learning (Dysthe and Engelsen, 2004[32]; Messick, 1994[33]; Paulson, Paulson and Meyeter, 1991[34]). Using student portfolios is appropriate in Georgia’s educational context because the country wishes to focus student assessment on providing feedback to individual students in order to improve their learning. Using student portfolios requires that teachers continuously provide feedback to students and that students and teachers reflect upon the portfolios to determine what the student’s strengths and weaknesses are.

Introducing student portfolios in Georgia will require strong support to teachers so they understand not only on how to use them, but also why they matter. As part of the survey conducted for this review, teachers were asked to what extent they believe certain methods accurately assess the performance of their students. Student portfolios were considered to be less accurate than question-and-answer, oral presentations and open-ended test questions. These data suggest that teachers in Georgia are not aware of how different methods of assessment can help students learn. They will need to gain this awareness in order to successfully use new practices.

There are several steps that Georgia can take to embed the use of student portfolios. An essential starting point would be to include examples of age and subject-appropriate portfolios as part of the new curriculum. These materials should be made available online so all teachers can access them (see Recommendation 2.2.1). They should also be accompanied with expectations as to when they might be undertaken and submitted, such as at the end of a grading period. Requiring that the portfolio be submitted at this point would help to ground summative judgements in a wider range of evidence and ensure that teachers and students take the portfolio exercise seriously. During regular reviews of portfolio materials, however, teachers should focus on an individual student’s strengths and challenges in relation to standards, rather than making a judgement of their performance.

Georgia will also need to build teachers’ confidence that they can successfully use student portfolios in their own classrooms. In-school support can help accomplish this. As part of the regular, school-based appraisal processes that this review recommends in chapter 3, teachers should be provided with feedback on their use of student portfolios. The feedback would not only help them improve their portfolio-related practices, but also help ensure that all make an effort to use those practices in the first place.

While portfolios can be used at all levels of education, in Georgia they would be particularly helpful in grades 8 and 9 as they could act as evidence to help inform a student’s decision about entering a general or vocational programme (see also Recommendation 2.1.3). If, for example, a student selects material for his/her portfolio that demonstrates an inclination towards a certain vocational field, a teacher can discuss with the student if that would be an appropriate path.

An example of this kind of practice exists in Finland, where students receive on-going, formal feedback about their performance throughout their schooling, which then feeds into the decisions that they make regarding future choices. Students might, based on the feedback they receive, enter a vocational stream, the workforce or higher education (Eurydice, n.d.[35]; Finnish National Agency for Education, n.d.[36]).

Discontinue the practice of continuous log grading

In Georgia, teachers are assigning class work, homework and a range of other tests and recording the grades continually. Though not mandatory, some teachers even report a numeric grade for every child for every lesson, which requires a large investment of time. These grades provide a snapshot summary of student activities at one point in time but do not help teachers understand how their students are learning. Furthermore, these grades do not help the students establish where they are in their learning and, crucially, where they need to go (Stobart, 2008[37]). Discontinuing the practice of daily grading would give teachers more time to dedicate to other activities and reduce the pressure that students feel around receiving a judgement every day.

Help teachers align their summative marking with the new curriculum

The introduction of the new curriculum presents opportunities for embedding formative assessment, but also challenges for determining students’ summative marks. Teachers must understand not only how to discern accurately student learning according to the curriculum, but also how grades correspond to students’ levels of learning.

A substantial amount of training and resources (see Recommendation 2.2.1) will need to be provided for teachers to assess students properly against the curriculum. It will be important that this training engage teachers in the definition of grading criteria so they are not simply told what the criteria are, but are involved in their development. This is important because involving teachers in education reforms increases their “buy in” and the likelihood that the reforms are sustained (Barber, 2009[38]; Persson, 2016[39]).

Moreover, teachers’ judgements will need consistent calibration with respect to their grading. However, the review team was told that teachers rarely meet systematically to talk about examples of marked student work or how each teacher awards grades. Georgia should thus give more support to schools to introduce school-based moderation, which provides time for teachers to convene to discuss how they mark student work and determine student learning (OECD, 2013[1]). Chapter 3 of this review mentions that there is excess time in teachers’ schedules – MoESCS can require schools to use this time to conduct moderation. During these meetings, teachers should come together to learn from each other’s practices, such as using student portfolios.

Providing adequate resources to teachers will be critical to help them align their marking and understand student progress with respect to the standards. Countries that wish to reform their teachers’ assessment practices have invested heavily in resourcing. Box 2.4 describes how the state of Rhode Island in the United States created an assessment toolkit to help its teachers decide on grading criteria and moderate their scoring of student work. Georgia, through TPDC, should consider producing similar materials (e.g. examples of student portfolios) to assist teachers with their marking.

Create a common report card template and procedures around dissemination

Internationally, student record-keeping tends to be explicitly prescribed. In the United Kingdom, legal regulations require that student progress be reported annually and that specific information about their children be included in the information that parents receive (Department of Education, 2015[48]). The Finnish National Agency for Education requires that students receive progress reports at least once a year based on the results of continuous assessments that are administered by teachers (Finnish National Agency for Education, 2017[49]).

Creating a common report card template, specifying what should be included in them and requiring that the report cards be disseminated would ensure that Georgian schools are recording the most vital information about student learning and that parents are made aware of their students’ progress. Importantly, the report card template should make space for, and require, descriptive, formative feedback, especially in grades 1 through 4 (and partially in grade 5), during which students only receive descriptive feedback. This information can help students and parents understand where a student is in their learning more effectively than stand-alone, numeric grades.

Integrate the standardised report card template into the EMIS E-Journal

EMIS has developed an E-Journal in which students’ grades are recorded and can be viewed online by themselves, their parents and their teachers. E-Journal has the advantage of transferability. As the grades are stored in the EMIS database, they can be retrieved when students switch schools and/or teachers. Currently, E-Journal is used by private schools in Georgia and its delivery is being planned for public schools.

With a standardised report card template, student data in E-Journal could then reflect what appears on their report cards, including the descriptive, formative feedback. This information would then be accessible to, and could be used by, a student’s teachers in each school that the student attends. In time, the E-Journal could replace the need for physical report cards, which would allow students and parents to more easily and quickly monitor student progress over their academic career.

Use student assessment information to support vulnerable students

Accurate and complete student records can be helpful in making learning risks more visible. Without consistent record-keeping, struggling students might not be identified as such and are vulnerable to becoming “lost,” especially as they change teachers and/or schools (OECD, 2012[50]). Such information would be particularly useful to principals, who need an efficient way of identifying students who need assistance. For example, a principal might not know what is happening in every classroom every day, but they can review student records to identify at-risk students, bring together teachers around the student’s report cards and together identify a strategy to help the student.

Internationally, several countries rely on student records to feed into early warnings systems, which analyse information about students’ attendance, learning progress and other factors to identify students who might be at risk of dropping out or under-performing (European Commission, 2013[51]; Borgonovi, Ferrara and Maghnouj, 2018[52]). With consistent student record keeping, Georgia can begin implementing such systems, which would help identify at-risk students and address equity gaps in student outcomes.

Develop and record external measures of student performance at key moments to inform decision-making

At present, most students in Georgia complete primary and secondary education without any external measure of their performance except for the UEE in grade 12. This configuration carries many risks. For example, students do not receive reliable indicators on their learning; teacher bias has an outsized effect on student assessment with no moderating measure; and school marks might make students think they have mastered a domain when, in fact, there is still much they can learn (OECD, 2013[1]). These concerns are heightened in Georgia because many students remain in the same school and are assessed by a limited number of teachers.

Over time, improving the quality of teacher’s assessment judgements will help to mitigate some of these risks. Nevertheless, it is equally important to introduce external measures at key points in order to help moderate teachers’ judgements. Many OECD countries regularly administer national assessments, which help improve the reliability of teacher judgements in relation to national standards (OECD, 2015[6]). As Georgia has developed national assessment capacity through its current national examinations system and through partnership with MCC, the OECD recommends that Georgia administer standardised, full-cohort assessments at key stages in a student’s education to help assess student performance (see chapter 5 for more details about the national assessment).

Consider making the external assessment in grade 9 (or grade 10 if compulsory education is extended) a certification examination

Though testing in early years would not be associated with stakes, this review suggests that Georgia consider administering an examination at the end of grade nine (or grade 10, if compulsory education is extended) that would, alongside teachers’ continuous assessment, help inform student choice with respect to the programme of study in upper secondary education. Internationally, there is considerable variety, related to the structure of national school systems, in how countries examine students at the end of lower secondary education. However, considering the structure of schooling in Georgia and the country’s goals to further develop VET, the review team suggests introducing an examination that would help select students into upper secondary pathways. Box 2.5 describes such an examination from Ireland, a country that also requires education through the second cycle of education, after which students can continue in general education or move to a more vocationally oriented track. Ireland’s examination also relies on a wide range of assessment practices, such as a project, which help the examination serve formative as well as summative purposes.

Georgia would be well-positioned to develop such an exam, as it could be based on the existing VET examination. Having such an examination for all students would better inform student choice and support Georgia’s national goal of strengthening the VET sector in upper secondary education. Administering an examination later in a student’s education (as opposed to during primary grades) helps to mitigate the risk of labelling young students while generating positive effects, such as incentivising students to apply themselves and helping students identify their interests. Some teachers told the OECD review team that they would welcome a national examination at this stage to alleviate the pressure of determining student pathways solely via their marks.

Strengthen the emphasis on improving assessment practices during initial teacher education

Teachers who complete ITP should be expected to have a minimum level of competence in the area of student assessment. In Georgia, ITP programmes struggle to provide teacher candidates with a strong basis in student assessment. There are no standards for graduated teachers in general, and none on assessment in particular. These standards would specify the types of skills graduated teachers would be expected to acquire and demonstrate. Georgia should develop these standards and use them to help guide the design of ITP programmes and the pedagogy examination recommended in chapter 3 of this review.

The planned introduction of a new consecutive ITP programme and the review of the extant one-year programme that this review recommends provide an opportunity to improve how teachers are trained to assess their students. Georgia should integrate into these programmes practices that have been internationally recognised to be effective in developing teachers’ assessment literacy. In Finland, which is recognised for having strong ITP programmes, teacher candidates are trained to assess students using a range of approaches. These might include group tasks, individual presentations, quizzes, practical assessments and performances and, crucially, the focus is on involving students in their own assessment (Niemi, 2015[61]; Kumpulainen and Lankinen, 2012[62]). Box 2.6 further identifies research about effective initial teacher education practices related to student assessment. They include mentorship and exposure to several types of assessment during teacher practicums.

Orient school-level professional development towards improving student assessment

Providing effective school-level supports can help teachers adopt new educational practices. With respect to student assessment, teachers can improve their own understanding and methods by observing each other, discussing how to design assessment and reviewing student work and reaching consensus around how to mark it (Harrison, 2005[59]; Tang et al., 2010[66]).

An important ingredient in supporting teachers in schools is allocating time for teachers to develop themselves. In Georgia, teachers currently have a surplus of non-instructional time in school and chapter 3 of this review recommends that this be used more effectively. This time could be used to help teachers collectively understand the curriculum’s assessment requirements and to organise the aforementioned moderation activities.

In addition to allocating time for the purpose, having strong school leadership that recognises the value of supporting teachers is highly important for helping teachers develop. Georgian principals are not accustomed to fulfilling this type of instructional leadership role and will need some guidance in how to best support their teachers. For this reason, the review team recommends that the several MCC sponsored programmes that are dedicated to improving assessment be continued and expanded. These programmes will provide principals with an established structure to follow in support of their teachers until they feel comfortable enough to determine independently how to best help their teachers develop.

Facilitate the development of teacher networks about assessment to support teachers, especially those in smaller and more rural schools

Along with support organised by schools, research shows that peer-to-peer collaboration can help teachers feel confident in appropriating new pedagogical methods (Wilson, 2008[60]). In Georgia, peer-to-peer activities do not occur frequently. In the survey administered as part of this review, 35% of teachers indicated that never, or only once a year or less, do they engage in discussions about the learning development of specific students. When asked if they work with other teachers to ensure common standards for assessing students, 21% said that they never do, or do so once a year or less.

Teacher collaboration in Georgia should be expanded, especially between experienced and less experienced teachers. Although schools have a responsibility to facilitate this collaboration, teachers themselves should also be encouraged to work with each other. While mentor and lead teachers would be suitable candidates to lead collaborative efforts and share their expertise, there are few teachers at these levels and most are concentrated in certain areas of the country (see chapter 3). Georgia also has a large number of small schools with few teachers. These circumstances limit the extent to which constructive collaboration can occur within Georgian schools.

One way of addressing these challenges is to use technology to expand the pool of potential teachers with whom teachers can collaborate. In Georgia, all school communications is already facilitated through EMIS E-Flow, which allows users to see the school and other information about message senders and recipients. While it is designed for sending and receiving messages, E-Flow can be given expanded functions to ease formal collaboration between teachers around topics such as assessment. For example, groups can be formed comprised of a lead teacher and several practitioner teachers from different schools with the expressed purpose of discussing assessment. E-Flow’s link to EMIS data can be harnessed to identify teachers in similar contexts, thus increasing the relevance of their collaboration.

Create an online repository of assessment resources

The OECD review team was told that Georgian teachers are using a limited repertoire of assessment types, such as multiple-choice quizzes, that mostly assess student’ ability to memorise and recall facts. Teachers have expressed a desire to use different types of assessments, such as more complex quizzes and ideas for class projects, but do not know where to access these resources.

An online repository represents an effective way of providing resources to schools, particularly rural ones who might not be able to access universities or even the nearest ERC. Many OECD countries have developed such repositories. For example, Ireland makes available online thousands of assessment resources that are explicitly linked to the national curriculum (Department of Education and Skills, 2019[67]). The Australian Institute for Teaching and School Leadership has created a website to house a comprehensive collection of teaching resources (Australian Institute for Teaching and School Leadership, n.d.[68]). These include lesson plans, guidance on providing feedback and video examples of how to use different methods to assess students.

Georgia has already begun to create an online repository of teaching materials (www.el.ge). Presently, however, this website only contains teaching stimuli, such as books, articles, and videos. It does not include pedagogical materials, such as tests, worksheets, lesson plans or student portfolios examples. It also does not include key strategic documents, such as the curriculum. Adding all such materials to this website would provide teachers with more resources and enable them to relate those resources to learning outcomes, which would increase the likelihood that the resources are used. Enhancing the online portal would also help improve equity as teachers in rural areas would be able to access the same resources as teachers in urban areas.

Engage school leadership to improve assessment literacy in schools

School principals, lead teachers and mentor teachers play a vital role in communicating the usefulness of new methods of assessment and reporting. Given how they were educated, many parents will be understandably sceptical of by the government’s desire to implement new assessment practices and might consider them to be lacking rigour and too lenient on students. It will be the responsibility of school leadership to explain the principles driving the reforms and thus allow teachers to continue implementing them without being concerned with public resistance.

MoESCS should provide school leadership with prepared responses that explain the value of the new curriculum and assessment reforms. When parents ask why their students are not receiving as many summative marks and instead are receiving more descriptive feedback, school staff can respond using MoESCS’s prepared arguments, which would help them feel supported and less exposed in their implementation of the reforms. Through these efforts, the schools would help build assessment literacy in the general population, which would facilitate teachers’ usage of new assessment practices in their classrooms.

Support students and families in understanding the new report card template

In Georgia, some parents have rarely seen a report card from their students’ schools, and others might have seen just a list of numeric marks alongside students’ classes. The new report card template (see Recommendation 2.1.3) will not only include summative marks, but also descriptive feedback and both would be aligned with the newly introduced curriculum (see Recommendation 2.1.2). However, students and parents will not be familiar with the information on the report card and might not know how to interpret what they see.

It will be the responsibility of the schools, with substantial guidance from MoESCS, to help families comprehend the new report card. Internationally, New Zealand recently changed its reporting system and gave schools more responsibility over how student results will be reported (Ministry of Education, n.d.[69]). To help schools communicate their systems to parents, the Ministry of Education created a “practical guide to assessment and reporting,” which provides direction on how to explain key concepts, materials to be used while meeting with parents and case studies from schools (Ministry of Education, n.d.[70]). MoESCS can provide similar guidance, complementary materials and relevant school examples to help principals and teachers clarify the new report card template to parents and students.

Build a communications campaign around the purpose of assessment

To support school-level efforts to reform student assessment, MoESCS should prepare a national communications campaign to help improve the assessment literacy of the general public. International experience shows that reforms, such as re-orienting the purpose of assessment, are frequently met with suspicion by individuals whose views about the subject are already entrenched (Burgess, Robertson and Patterson, 2010[71]). Therefore, the justification for change needs to be carefully considered and effectively communicated in addition to the content of the change (Njeng’ere Kabita and Ji, 2017[72]).

MoESCS has experience with conducting such campaigns. Its outreach around introducing the SGEs in 2011 was very successful, even though the concept was new and specific components, such as CAT, are difficult to understand. Communications strategies that were effective in 2011, such as an announcement by an important government leader and using ERCs to visit every school in the country, can be adapted to communicate the concept and value of assessment for learning.

It is also important to show the public what exactly assessment for learning looks like. To this end, Shanghai education leaders produced videos of what is considered model practice with one camera focused on the teacher and another focused on students. The videos were then disseminated to other teachers around the city (OECD, 2011[73]). At the school-level, Colombia has created a Day of Educational Excellence (now called “E Day”) to celebrate schools that demonstrate what the government considers to be outstanding practices (Colombian Ministry of Education, 2010[74]). These strategies can be utilised in Georgia but adapted to show Georgian teachers and students. In this way, the public can not only visualise what assessment for learning is, but also understand that it is realisable in their country (Fullan, 2006[57]; Fullan, 2004[75]).

Set, as a medium-term objective (next five to ten years), the introduction of a single examination at the end of upper secondary education

When the SGE was initially conceptualised in 2011, there was discussion about combining it with the UEE to create a single national examination that would serve as an exit examination from upper secondary school and an entrance examination into higher education institutions (Bakker, 2014[20]). Ultimately, the exams were kept separate because of the resources required (e.g. testing time) and because the exams were intended to serve different purposes.

Today, research and analysis of the examinations data suggests that the initial concerns around a single examination model are less relevant (described in detail below) and that such a model could be very beneficial in the Georgian context. Therefore, the OECD suggests that MoESCS should aim to develop a single examination model to replace the current UEE and define a reasonable timeframe for its development and introduction.

Creating different versions of subject tests ensures that a single examination can both ensure basic minimum competency and select for entrance into university

Stakeholders were also concerned that a single examination could not fulfil the dual functions of certification from upper secondary education and selection for higher education. This concern, however, is not supported by empirical analysis of the examinations data (see Figure 2.3). On the Georgian language and literature test of the SGE, for example, student results produced a normal distribution, indicating that most students were able to meet minimum standards and that high-performing students could still be discriminated for selection into higher education. There was no need to re-test these students in this subject again (though it occurred through the UEE) as that only added to students’ testing burden.

Nevertheless, not all subjects produced such a distribution. In SGE mathematics, the distribution was bi-modal, with a large number of students receiving a perfect score and an even larger number clustered around the cut-off passing score (see Figure 2.3). This distribution raises two concerns. First, it indicates that too many students barely met minimum standards and it is difficult to say with certainty if these students mastered basic numeracy. Second, it also suggests that another group of students found the test too easy and their perfect results were not able to be discriminated for selection into higher education.

copy the linklink copied!

Figure 2.3. Results from the Georgian language and mathematics tests on the SGE 2018

Source: Data provided to the OECD from NAEC.

The previous solution to this problem was to re-test the high-achieving students by encouraging them to take the mathematics test of the UEE. Despite requiring even more testing time, this method still did not address the issue of the SGE mathematics examination not being able to accurately determine basic numeracy in a large number of students. Instead, the problem can be addressed in the single examination model by developing two mathematics tests of different difficulty levels. Students would take only one version of the test, depending upon their interests and university requirements. This procedure is common in several countries, including England, the Netherlands and Norway (Ofqual, 2012[76]) (see Recommendation 2.3.2).

Determine who should take the single examination in accordance with national goals concerning VET schools

There is clear recognition of the need to address “dead ends” associated with the VET track. Essential components of this reform would include introducing general academic content into the VET curriculum and allowing VET upper secondary students to enter directly into bachelors’ programmes.

Given this situation, a key question that Georgia will need to address is whether the new, single examination should be taken by all students, or whether it would be taken only by general upper secondary students, with potentially a separate mode of examination for students in VET schools. To align the examination with the government’s goals, the review team suggests that the new examination be accessible to all students who complete upper secondary education. Having two versions of a subject test, such as mathematics, would be particularly suitable in this context in order to accommodate the different levels of mathematics that students would need with respect to their future pathways and career aspirations. This type of flexible design also creates space for specific VET competencies to be certified as elective subjects on the exam.

Determine the design features of the examination

Several aspects of the design of the examination will need to be considered in light of the strengths and challenges of the previous examinations structure and the goals of the education system. For example, Georgia needs to consider which subjects are compulsory and which are voluntary, and whether some subjects, in particular mathematics as mentioned, have different versions that students can select.

The decision as to whether the examination should be accessible to all upper secondary students will also influence its design features. If the exam is to be accessible to VET students, as suggested by the OECD, then this might require a limited number of core subjects and a significant elective component, with differentiation of test versions in some subjects. Elective subjects might also include VET fields and NAEC would be responsible for the quality assurance and standardisation of these tests (e.g. checking that administration conditions are equitable). The development and actual administration of the VET tests, however, could be led, as it is in other countries, by industry representatives, such as unions and trade associations (OECD, 2014[77]). Box 2.7 provides an example a single examination model from the Republic of North Macedonia and explains how the model operates in consideration of general and vocational education tracks.

Examine implementation requirements

NAEC has demonstrated the capacity to develop new testing methods and has gained the public’s trust when doing so. It is well-positioned to introduce a medium-term reform agenda of Georgia’s examinations system. Importantly, its previous innovations should continue to be used for some parts of the examination, such as including some computer adaptive items. In this manner, public trust owing to this mode of testing would be sustained, which would make implementation easier.

Georgia is experienced with successfully introducing new examinations, in particular when it introduced the SGE in 2011. The implementation of the new single examination model can follow some of the established guidelines around communications and training, such as an announcement by the head of NAEC and communicating with schools through education resource centres. Additional questions to consider include the timeframe of implementation, which should be several years, the cost to do so and the need for a piloting phase, similar to what occurred before the SGE was implemented.

Strengthen school-level examinations in upper secondary education

Without the SGE, only upper secondary graduates who wish to attend university have their skills assessed and recognised in an external manner. Students who graduate from upper secondary education but who do not wish to attend university do not receive external certification of their achievements. This negatively affects their prospects in the labour market because employers do not have assurance that these students have acquired the basic minimum competencies to be productive in the workplace.

The OECD understands that some schools in Georgia will now administer examinations to their students to assess their readiness to graduate. It will be imperative that these tests are high quality so that graduation represents a signal of achievement that is trusted by the community. The OECD supports the idea that NAEC should help strengthen schools’ examinations by providing clear specifications about test content to ensure that the examinations assess the curriculum at appropriate levels of breadth and depth. NAEC can also develop marking guidelines so all schools evaluate student responses against national curriculum goals and in a consistent manner.

Finally, relying on school-level examinations to certify student achievement represents an opportunity to strengthen assessment literacy (see Recommendation 2.2.1). Teachers are now in a pivotal position and will want to know what they can do to evaluate student learning more accurately. For instance, they might wish to use a variety of item types on the examinations, as they have expressed concern previously about the SGE only using multiple-choice items. NAEC can assist teachers in this regard by providing guidance about how to develop more sophisticated question types and include more of these types of items in the sample examinations that are already being developed.

Better align the UEE with the national curriculum in order to reduce its backwash effects

To coordinate what students learn in school with what they are assessed against centrally, classroom instruction and national assessments and examinations must be based on common reference standards (OECD, 2013[1]). To this end, the majority of OECD countries, including France, Ireland and Poland, create their upper secondary examinations based upon national curriculum goals and national learning standards.

Exams that are not aligned with the curriculum can have negative effects on students’ education. Misaligned examinations compel students to seek out external support to learn the material on the examinations because they do not learn it in school. This incentive, combined with the high stakes of the examinations, can create backwash effects in which some students and teachers focus more on preparing for the examinations than on learning important skills (Bray and Kobakhidze, 2014[80]).

In Georgia, items on the UEE are not always aligned with the curriculum. Although MoESCS must approve of the examination’s programme before items are developed, the final items themselves are not evaluated for their coherence vis-à-vis the curriculum and stakeholder interviews suggest that the items can stray from the curriculum’s aims. As a result, students do not necessarily learn in school what they need to know to succeed the UEE. To aid students, teachers might deviate from the curriculum to teach to the UEE, or offer private tutoring to students with greater resources. These activities distort the education system by focusing teaching and learning on material that is not in the curriculum and widening the achievement gap between advantaged and disadvantaged students.

MoESCS can reduce the negative backwash effects produced by the UEE by implementing a final review of UEE items, after they are fully developed, to validate that the items that appear on the examination are truly aligned with the goals of the national curriculum. Not only would this reduce the negative backwash effects, but it would generate positive on pressure on students to attend classes and engage with the curriculum in order to better prepare themselves for the UEE.