Evaluating the risk of nonresponse bias in educational large-scale assessments with school nonresponse questionnaires: a theoretical study

Large-scale Assessments in Education

An IEA-ETS Research Institute Journal

Table 1 Percentages of countries failing the participation rate requirements in IEA studies (last 10 years)

Study, cycle and target population	Number of participating countries	Countries failing participation rate requirements* at
		School level (before replacement)		School level (after replacement)		Within schools (individual level)
		%	Count	%	Count	%	Count
Studies with school students comprising the target population
ICCS 2009—grade 8 students	42	33.3	14	14.3	6	0.0	0
ICILS 2013—grade 8 students	21	33.3	7	23.8	5	9.5	2
PIRLS 2006—grade 4 students	47	12.8	6	4.3	2	4.3	2
PIRLS 2011—grade 4 students	58	13.8	8	1.7	1	1.7	1
TIMSS 2007—grade 4 students	43	20.9	9	7.0	3	0.0	0
TIMSS 2007—grade 8 students	56	14.3	8	7.1	4	0.0	0
TIMSS 2011—grade 4 students	50	20.0	10	6.0	3	0.0	0
TIMSS 2011—grade 8 students	59	6.8	4	3.4	2	0.0	0
TIMSS advanced 2008
Advanced mathematics students	10	10.0	1	10.0	1	0.0	0
TIMSS advanced 2008
Physics students	9	22.2	2	0.0	0	11.1	1
Overall	395	17.5	69	6.8	27	1.5	6
Studies with adults comprising the target population
ICCS 2009—grade 8 teachers	37	37.8	14	29.7	11	21.6	8
ICILS 2013—grade 8 teachers	21	42.9	9	38.1	8	28.6	6
SITES 2006—math teachers	22	59.1	13	40.9	9	22.7	5
SITES 2006—science teachers	22	59.1	13	40.9	9	31.8	7
TEDS-M 2008—future primary
Math teachers**	16	6.3	1	6.3	1	37.5	6
Secondary math teachers**
TEDS-M 2008—future lower	16	31.3	5	31.3	5	56.3	9
TEDS-M 2008
University educators**	16	37.5	6	37.5	6	37.5	6
Overall	150	40.7	61	32.7	49	31.3	47

* Minimum participation rates are 85% for schools; 85% for individuals within schools (computed across all participating schools)
** In TEDS-M, “schools” were institutions that offer education programs for future primary or lower-secondary mathematics teachers