The dependence on mathematical theory in TIMSS, PISA and TIMSS Advanced test items and its relation to student achievement

Large-scale Assessments in Education

An IEA-ETS Research Institute Journal

Table 3 Coherence in the full group of scorers on TIMSS grade 8 and PISA

	PISA math 2012	TIMSS grade 8 math 2011
T/NT classification	88.2% [97.6%] (.82)	78.8% [92.6%] (.76)
F/NF classification	77.6% [90.6%] (.74)	65.4% [86.6%] (.65)

The first number is the percentage of items where all six scorers agreed on the classification. The number in the straight brackets is the percentage of items for which at least 5 scorers agreed. The number in parentheses is inter-rater reliability as measured by Fleiss’ kappa