OECD Education Working Papers

ISSN :
1993-9019 (online)
DOI :
10.1787/19939019
Hide / Show Abstract
This series is designed to make available to a wider readership selected studies drawing on the work of the OECD Directorate for Education. Authorship is usually collective, but principal writers are named. The papers are generally available only in their original language (English or French) with a short summary available in the other.
 

Analysis of PISA 2006 Preferred Items Ranking Using the Percent-Correct Method You or your institution have access to this content

Authors:
Ray Adams1, Alla Berezner1, Maciej Jakubowski2, 3
Author Affiliations
  • 1: Australian Council for Educational Research, Australia

  • 2: OECD, France

  • 3: Warsaw University, Poland

Publication Date
31 Mar 2010
Bibliographic information
No.:
46
Pages
24
DOI
10.1787/5km4psmntkq5-en

Hide / Show Abstract

This paper uses an approximate average percent-correct methodology to compare the ranks that would be obtained for PISA 2006 countries if the rankings had been derived from items judged by each country to be of highest priority for inclusion. The results reported show a remarkable consistency in the country rank orderings across different sets of countries’ preferred items when comparing with the rank reported in the PISA 2006 initial report (OECD, 2007). On average, only few countries systemically go up or down in their ranking position. As these countries are in a group of moderate performers with very comparable outcomes, these shifts in the ranking would probably be associated with minor changes in mean performance on the final PISA scale. The analysis suggests that PISA rankings are noticeably stable thanks to the large enough pool of test items able to accommodate diverse preferences. The paper shows how important it is to base a choice of test items on a properly structured process which allows different experts and countries to equally contribute. The evidence presented demonstrates that in PISA, average rank positions of countries across different sets of preferred items are apparently stable and experts are not able to predict which items can elevate performance of their countries in the final test.