Comparing different trend estimation approaches in country means and standard deviations in international large-scale assessment studies

Robitzsch, Alexander; Lüdtke, Oliver

doi:10.1186/s40536-023-00176-6

Large-scale Assessments in Education

An IERI – International Educational Research Institute Journal

Table 3 Relative average root mean square error (RMSE) for cross-sectional estimates and trend estimates in country means and country standard deviations based on the 2PL Model for assessment design Des1 (I₀ = 30, I₁ = 0, I₂ = 0), Design Des2 (I₀ = 30, I₁ = 30, I₂ = 0), and Design Des3 (I₀ = 30, I₁ = 30, I₂ = 30)

From: Comparing different trend estimation approaches in country means and standard deviations in international large-scale assessment studies

Design	Model	Link	Inv	Items	2PL NODIF			2PL UDIF			2PL NUDIF
						N		N			N
					500	1000	2500	500	1000	2500	500	1000	2500
	Cross-sectional mean
Des1	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
Des1	I2	Indirect	No	All	122.8	119.4	117.9	115.7	95.6	91.9	114.3	95.6	89.7
	Cross-sectional standard deviation
Des1	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
Des1	I2	Indirect	No	All	108.8	107.0	106.1	120.5	62.6	45.0	110.0	89.7	85.5
	Trend in mean
Des1	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	121.6	118.0	116.6	128.6	106.3	100.5	132.3	109.8	99.5
	D2	Direct	No	All	121.3	118.0	116.6	115.4	106.3	100.5	115.9	106.2	99.5
Des2	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	182.4	123.1	118.8	197.5	112.2	98.5	213.5	115.9	96.9
	I3	Indirect	Yes	Link	104.2	103.5	103.7	88.9	84.7	81.1	88.5	84.8	79.4
	I4	Indirect	No	Link	234.8	137.6	132.4	222.2	109.5	85.1	244.1	127.0	83.4
	D2	Direct	No	All	139.3	132.1	128.7	115.1	95.7	84.0	112.5	95.4	80.8
	D4	Direct	No	Link	145.9	136.8	132.4	118.6	98.2	85.1	115.7	98.0	81.7
Des3	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	191.7	125.1	119.7	215.9	114.6	98.3	215.0	125.0	96.0
	I3	Indirect	Yes	Link	107.2	107.1	107.6	93.1	88.1	83.5	91.7	87.7	82.7
	I4	Indirect	No	Link	230.2	152.0	145.7	234.5	114.5	90.8	259.9	132.2	87.9
	D2	Direct	No	All	154.1	143.0	139.1	129.0	105.2	88.8	126.5	105.5	85.8
	D4	Direct	No	Link	165.4	151.1	145.7	136.8	110.1	90.8	132.8	110.1	87.8
	Trend in standard deviation
Des1	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	107.7	106.4	105.9	100.2	92.4	80.1	99.8	92.6	76.7
	D2	Direct	No	All	107.6	106.4	105.9	100.2	92.4	80.1	99.4	92.4	76.7
Des2	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	109.1	106.8	106.0	93.8	83.6	65.3	98.5	92.4	83.2
	I3	Indirect	Yes	Link	109.9	109.7	110.0	103.8	98.7	91.8	100.3	94.1	85.7
	I4	Indirect	No	Link	124.1	120.6	119.1	107.0	94.4	72.8	102.3	88.7	67.0
	D2	Direct	No	All	120.5	118.6	117.3	104.7	92.6	71.9	100.1	86.7	65.9
	D4	Direct	No	Link	122.4	120.3	119.1	106.2	94.0	72.8	100.9	87.9	66.9
Des3	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	109.5	107.6	106.2	94.8	85.0	68.4	100.0	95.1	86.9
	I3	Indirect	Yes	Link	118.7	119.9	119.3	111.9	106.0	98.6	108.6	102.2	92.4
	I4	Indirect	No	Link	136.1	133.5	131.4	119.7	105.2	83.8	114.8	101.0	76.7
	D2	Direct	No	All	131.3	129.8	127.7	116.5	102.1	81.7	111.3	97.5	74.4
	D4	Direct	No	Link	134.5	133.2	131.4	119.6	104.9	83.8	114.1	100.1	76.6

Inv assumption of invariant item parameters across countries; indirect indirect linking; direct direct linking; NODIF no differential item functioning; UDIF uniform differential item functioning; NUDIF nonuniform differential item functioning. Method I1 (original trend) was used as the reference for computing the relative RMSE. RMSE entries smaller than 100 are printed in bold

Back to article page