Comparing different trend estimation approaches in country means and standard deviations in international large-scale assessment studies

Robitzsch, Alexander; Lüdtke, Oliver

doi:10.1186/s40536-023-00176-6

Large-scale Assessments in Education

An IERI – International Educational Research Institute Journal

Table 2 Relative average root mean square error (RMSE) for cross-sectional estimates and trend estimates in country means and country standard deviations based on the 1PL model for assessment design Des1 (I₀ = 30, I₁ = 0, I₂ = 0), Design Des2 (I₀ = 30, I₁ = 30, I₂ = 0), and Design Des3 (I₀ = 30, I₁ = 30, I₂ = 30)

From: Comparing different trend estimation approaches in country means and standard deviations in international large-scale assessment studies

Design	Model	Link	Inv	Items	1PL NODIF			1PL UDIF
					N			N
					500	1000	2500	500	1000	2500
	Cross-sectional mean
Des1	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0
Des1	I2	Indirect	No	All	100.2	100.3	100.1	98.2	98.4	97.3
	Cross-sectional standard deviation
Des1	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0
Des1	I2	Indirect	No	All	102.6	102.7	102.1	80.8	69.9	52.4
	Trend in mean
Des1	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	100.1	100.2	100.1	99.2	98.7	98.2
	D2	Direct	No	All	100.1	100.2	100.1	99.2	98.7	98.2
Des2	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	100.3	100.3	100.3	99.2	98.9	98.6
	I3	Indirect	Yes	Link	103.8	104.1	104.0	89.8	84.9	81.4
	I4	Indirect	No	Link	104.5	104.8	104.6	89.6	84.2	80.1
	D2	Direct	No	All	104.3	104.6	104.5	89.6	84.4	80.2
	D4	Direct	No	Link	104.5	104.8	104.6	89.6	84.2	80.1
Des3	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	100.5	100.3	100.3	99.0	98.5	98.2
	I3	Indirect	Yes	Link	107.4	107.0	107.7	93.3	90.6	84.5
	I4	Indirect	No	Link	108.7	107.8	108.6	93.2	90.0	83.3
	D2	Direct	No	All	108.3	107.5	108.4	93.1	90.0	83.4
	D4	Direct	No	Link	108.7	107.8	108.6	93.2	90.0	83.3
	Trend in standard deviation
Des1	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	102.1	102.5	102.3	97.8	93.3	83.9
	D2	Direct	No	All	102.1	102.5	102.3	97.8	93.3	83.9
Des2	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	102.4	101.9	102.2	92.5	84.8	72.2
	I3	Indirect	Yes	Link	111.5	110.9	111.1	107.1	100.9	96.8
	I4	Indirect	No	Link	114.9	114.3	114.6	103.8	93.6	80.8
	D2	Direct	No	All	102.4	101.9	102.2	92.5	84.8	72.2
	D4	Direct	No	Link	114.9	114.3	114.6	103.8	93.6	80.8
Des3	I1	Indirect	Yes	All	100.0	100.0	100.0	100.0	100.0	100.0
	I2	Indirect	No	All	102.5	102.3	102.3	93.7	87.8	73.7
	I3	Indirect	Yes	Link	119.3	119.8	121.1	117.1	113.1	103.1
	I4	Indirect	No	Link	124.5	124.3	126.1	115.4	107.6	90.5
	D2	Direct	No	All	102.5	102.3	102.3	93.7	87.8	73.7
	D4	Direct	No	Link	124.5	124.3	126.1	115.4	107.6	90.5

Inv assumption of invariant item parameters across countries; indirect indirect linking; direct direct linking; NODIF no differential item functioning; UDIF uniform differential item functioning; NUDIF nonuniform differential item functioning. Method I1 (original trend) was used as the reference for computing the relative RMSE. RMSE entries smaller than 100 are printed in bold

Back to article page