An IERI – International Educational Research Institute Journal
Table 4 Item-by-country kappa (human-human inter-rater reliability)
From: Combining machine translation and automated scoring in international large-scale assessments
Item | C1 | C2 | C3 | C4 | C5 | C6 | C7 | C8 | Average |
|---|---|---|---|---|---|---|---|---|---|
Item 1 | 0.94 | 0.89 | 0.93 | 0.99 | 0.97 | 1.00 | 0.89 | 0.93 | 0.94 |
Item 2 | 0.98 | 0.98 | 0.95 | 0.94 | 0.94 | 1.00 | 0.98 | 0.99 | 0.97 |
Item 3 | 0.97 | 0.94 | 0.98 | 1.00 | 0.90 | 1.00 | 0.98 | 0.99 | 0.97 |
Item 4 | 0.95 | 0.99 | 0.84 | 0.97 | 0.89 | 1.00 | 0.85 | 0.86 | 0.92 |
Item 5 | 0.88 | 0.98 | 0.91 | 0.98 | 0.94 | 1.00 | 1.00 | 0.94 | 0.95 |
Item 6 | 0.97 | 0.98 | 0.96 | 0.99 | 0.91 | 1.00 | 0.96 | 1.00 | 0.97 |
Average | 0.95 | 0.96 | 0.93 | 0.98 | 0.93 | 1.00 | 0.94 | 0.95 | 0.95 |