The potential of international large-scale assessments for meta-analyses in education

Large-scale Assessments in Education

An IERI – International Educational Research Institute Journal

Table 4 Results of the random-effects meta-analyses of the gender differences in digital literacy

	Separate meta-analyses				Direct inclusion			Indirect inclusion
	ICILS 2013 data	ICILS 2018 data	ICILS 2013 & 2018 data	Non-ILSA data	One-stage inclusion	One-stage inclusion	Two-stage inclusion	Bayesian meta-analysis
Weighted average effect size
\(\overline{g }\)	0.13	0.21	0.16	0.12	0.13	0.10	0.12	0.12
95% CI	[0.10, 0.17]	[0.15, 0.27]	[0.12, 0.19]	[0.04, 0.20]	[0.05, 0.21]	[0.01, 0.18]	[0.05, 0.19]	[0.03, 0.20]
k	21	13	34	25	59	59	27	25
m	21	13	28	25	24	24	27	25
Heterogeneity
\({\tau }_{(2)}^{2}\)	0.005	0.012	0.003	0.033	0.007	0.016	0.030	0.036
95% CI	[0.002, 0.012]	[0.005, 0.033]	[0.000, 0.013]	[0.016, 0.074]	[0.004, 0.013]	[0.000, 0.079]	[0.014, 0.065]	[0.017, 0.078]
\({\tau }_{(3)}^{2}\)	–	–	–	–	0.027	0.031	–	–
95% CI	–	–	–	–	[0.007, 0.071]	[0.011, 0.077]	–	–
\({\tau }_{(4)}^{2}\)	–	–	0.006	–	–	0.006	–	–
95% CI	–	–	[0.000, 0.015]	–	–	[0.000, 0.012]	–	–
Baseline model	REM2	REM2	REM3	REM2	REM3	CCREM4	REM2	REM2

ILSA = International large-scale assessment, \(\overline{g }\) = Weighted average effect size of gender differences (Hedges’ \(g\)), k = Number of effect sizes (samples), m = Number of studies, \({\tau }_{(2)}^{2}\) = Within-study (between-sample) heterogeneity, \({\tau }_{(3)}^{2}\) = Between-study heterogeneity, \({\tau }_{(4)}^{2}\) = Between-country heterogeneity, REM2 = Two-level random-effects model, REM3 = Three-level random-effects model, CCREM4 = Four-level cross-classified random-effects model (with 31 countries). (2)-(4) refer to the level of analysis