Skip to main content

An IERI – International Educational Research Institute Journal

Table 5 Mean facet level item difficulty across science facets and levels

From: Comparing the score interpretation across modes in PISA: an investigation of how item facets affect difficulty

Facet

Levels

All items

Mode invariant items

\({\overline{\mathrm{b}} }_{\mathrm{PBA}}\)

\({\overline{\mathrm{b}} }_{\mathrm{CBA}}\)

Wald test χ2 (N = 11861)

\({\overline{\mathrm{b}} }_{\mathrm{PBA}}\)

\({\overline{\mathrm{b}} }_{\mathrm{CBA}}\)

Wald test χ2 (N = 11859)a

Context 1 (2015)

Personal

−0.06 (0.08)

0.10 (0.07)

χ2 (2) = 5.85*

−0.42 (0.08)

−0.29 (0.07)

χ2 (2) = 4.4

Local/National

−0.38 (0.03)

−0.11 (0.03)

p = .05

−0.30 (0.03)

−0.11 (0.03)

p = .11

Global

0.11 (0.03)

0.43 (0.03)

 

0.09 (0.04)

0.35 (0.03)

 

Context 2 (2015)

Health and disease

−0.37 (0.04)

−0.21 (0.03)

χ2 (4) = 24.40*

−0.31 (0.04)

−0.16 (0.04)

χ2 (4) = 3.99

Natural resources

−0.52 (0.04)

−0.18 (0.03)

p < .01

−0.35 (0.04)

−0.13 (0.03)

p = .41

Frontiers

0.15 (0.04)

0.40 (0.04)

 

−0.14 (0.04)

0.06 (0.04)

 

Hazards

−0.76 (0.06)

−0.53 (0.05)

 

−0.78 (0.07)

−0.63 (0.05)

 

Environmental quality

−0.17 (0.04)

0.14 (0.03)

 

0.12 (0.04)

0.35 (0.04)

 

System

Living

−0.06 (0.04)

0.18 (0.03)

χ2 (2) = 8.97*

−0.1 (0.03)

0.07 (0.03)

χ2 (2) = 5.24*

Earth and space

−0.43 (0.04)

−0.07 (0.03)

p = .01

−0.47 (0.04)

−0.23 (0.03)

p = .07

Physical

−0.35 (0.04)

−0.12 (0.04)

 

−0.36 (0.05)

−0.16 (0.05)

 

Competency

Interpret data & evidence scientifically

−0.22 (0.04)

0.08 (0.03)

χ2 (2) = 11.06*

−0.19 (0.03)

0.00 (0.03)

χ2 (2) = 4.02

Evaluate and design scientific enquiry

−0.30 (0.05)

0.06 (0.04)

p < .01

−0.12 (0.05)

0.17 (0.05)

 p=.13

Explain phenomena scientifically

−0.22 (0.04)

−0.03 (0.03)

 

−0.36 (0.04)

−0.21 (0.04)

 

Knowledge

Procedural

−0.30 (0.04)

−0.04 (0.04)

χ2 (2) = 4.39

−0.1 (0.04)

0.08 (0.04)

χ2 (2) = 8.37*

Content

−0.24 (0.04)

0.00 (0.03)

p = .11

−0.33 (0.04)

−0.17 (0.04)

 p=.02

Epistemic

−0.01 (0.05)

0.35 (0.04)

 

−0.27 (0.06)

0.07 (0.04)

 

Depth of Knowledge

Low

−0.37 (0.04)

−0.12 (0.04)

χ2 (2) = 3.42

−0.49 (0.05)

−0.32 (0.05)

χ2 (2) = 1.35

Medium

−0.23 (0.03)

0.05 (0.03)

p = .18

−0.26 (0.03)

−0.04 (0.03)

 p=.51

High

0.30 (0.05)

0.47 (0.05)

 

0.30 (0.05)

0.49 (0.06)

 
  1. *Significant difference between PBA and CBA variation in estimated mean item difficulty within facet
  2. aSample sizes between all items and mode invariant items vary due to limited cases where students recorded no responses across the mode invariant items