 Research
 Open access
 Published:
No substantive effects of school socioeconomic composition on student achievement in Australia: a response to Sciffer, Perry and McConney
Largescale Assessments in Education volume 12, Article number: 8 (2024)
Abstract
In this journal, Sciffer et al. (Largescale Assessments in Education 10:1–22, 2022), hereafter SP&M, conclude that school socioeconomic compositional (SEC) or school socioeconomic status (schoolSES) effects in Australia are substantial and substantively important for research and policy. This paper demonstrates that these claims are unwarranted. Their SEC estimates are much larger than estimates from comparable studies and a metastudy. Despite plausible theoretical reasons and empirical evidence, SP&M do not consider that school academic composition is a significant predictor of student achievement independent of SEC. SEC effects are confounded by academic composition and are typically trivial when considering academic composition. The second part of this paper compares SP&M’s estimates with analysis of the same data, from the Australian National Assessments in Performance—Literacy and Numeracy (NAPLAN). In a model corresponding to SP&M analyses comprising demographics, SES, schoolSES, and studentlevel prior achievement, the effects of schoolSES are small, with standardized effects mostly less than 0.10. With the addition of academic composition measured by schoollevel prior achievement, schoolSES effects are effectively zero. In contrast, academic composition has significant, albeit small, impacts on student achievement. Therefore, contrary to SP&M’s (2022) conclusion, schoolSES effects on student achievement in NAPLAN are negligible, whereas schoollevel prior achievement has small effects. That is not to say that schoolSES is always irrelevant, but any assessment of its importance must consider both student and schoollevel prior achievement.
Introduction
In this journal, Sciffer et. al. (2022, p. 1) hereafter SP&M, claim that school socioeconomic composition^{Footnote 1} (SEC) “is of practical significance to policy makers and educational researchers” based on analyses of Australian national assessment data. They highlight extraordinarily large schoolSES effects and propose a raft of policies to address the supposedly large SEC effects. These policies include, requiring private schools to enroll a diverse academic mix of students, prohibiting private schools from allowing lower achievers to drop out of school, undefined amelioratory education policies that address school compositional effects, funding reforms, and adjusting students’ tertiary entrance scores for university entry based on school demographic factors (2022, p. 17).
This paper demonstrates that SP&M’s conclusions on SEC effects are unwarranted; thus, their policy recommendations are irrelevant. Their highlighted estimates for SEC or school socioeconomic status (SES) are several times larger than comparable studies. Furthermore, they do not consider school academic composition as influencing student achievement independent of schoolSES.^{Footnote 2} There are sound theoretical reasons why academic composition may be significant, and the empirical literature indicates that it has more substantial effects than schoolSES. Typically, studies find trivial or no effects of schoolSES when considering academic composition. The first part of this paper discusses these issues.
The second part of this paper reports analyses of the same data as SP&M analysed, the 2017 data from the Australian National Assessments in Performance—Literacy and Numeracy (NAPLAN). It shows that schoolSES effects are minor when considering studentlevel prior achievement (model 1). The standardized effects for schoolSES are most often below 0.10. In contrast, SP&M (2022, p. 12) claim that standardized schoolSES effects average 0.32 among primary school students and 0.52 among secondary school students. With the addition of schoollevel prior achievement, schoolSES effects are effectively zero (model 2). Model 3 assesses the importance of schoolSES for student achievement by removing it from the analyses. Its deletion does not reduce the model’s explanatory power and the parameter estimates are mostly unchanged, providing further evidence that according to Australian NAPLAN data, schoolSES is irrelevant to student achievement.
Statistical and methodological issues
Extraordinary large schoolSES effects
SP&M’s (2022, p. 9) equation 1 specifies student achievement as a function of demographic factors, studentlevel SES, schoolSES, and studentlevel prior achievement, but not academic composition. They report substantial indirect standardized schoolSES effects of 0.32 for Year 5 and 0.48 for Year 9.
Published studies report much smaller SEC effects than SP&M (2022) from analyses using the same model specification, that is controlling for student, but not schoollevel, prior achievement. Marks (2015, p. 15) reports standardized schoolSES effects between zero and 0.05 for Year 5 NAPLAN in Victoria. Analysing NAPLAN and tertiary entry scores for students in New South Wales, Lu and Rickard (2014, p. 32) report standardized schoolSES effects of 0.08 for Years 3 to 5, 0.03 for Years 7 to 9 and 0.13 for Years 11 to 12. Armor et. al. (2018) report standardized schoolSES effects around 0.05 for math and reading scores in population data from three US states. In a metaanalysis of schoolSES effects, van Ewijk and Sleegers (2010, p. 145) report that schoolSES effects are, on average, 0.15 standard deviations higher in the absence of studentlevel prior achievement. Since their average standardized effect of schoolSES is 0.32, the average schoolSES effect is 0.07 for studies that include prior achievement. So, SP&M (2022, p. 12) estimates for schoolSES are at least four times larger than estimates from comparable studies.
SchoolSES effect confounded by schools’ academic composition
SP&M (2022, p. 12) specify academic composition as only mediating the effects of schoolSES on student achievement. They do not consider that schools’ academic composition may directly affect student achievement independently of schoolSES. There are strong theoretical reasons why schoollevel prior achievement may influence student performance, net of studentlevel prior achievement. In schools catering to higher achieving students, schools pitch the curriculum at a higher level, teachers demand more from their students, and competition among peers raises standards and expectations. Students have more positive attitudes toward learning, their school, and their teachers. There are fewer disruptions to learning. Overall, the school’s environment is more academic. These social processes cannot be construed as simply reflections of schoolSES.
SchoolSES effects are typically trivial or are not statistically significant, net of schoollevel prior achievement. In a review of school effectiveness research, Scheerens et al. (2001, p. 136) conclude that “it is contextual IQ effects (or contextual effects of previous achievement) rather than contextual SES effects that seem to predominate.” Snidjers and Bosker’s (2012, pp. 84–86, 121–122) graduatelevel textbook on multilevel modeling presents several examples where the effect of school IQ is positive and nontrivial and estimates for schoolSES are much smaller, often negative and mostly not statistically significant. SP&M (2022, p. 4) are aware that schoolSES effects are seriously confounded by schoollevel prior achievement and cite several studies that find smalltozero effects for schoolSES (Dumay & Dupriez, 2008; Lauder et al., 2010; Marks, 2015). Although SP&M (2022, p. 4) acknowledge that schoolSES effects on achievement are often negligible, net of academic context, they perversely interpret the finding of smalltozero SEC effects as indicating that “academic context may mediate socioeconomic context” rather than socioeconomic context is unimportant.
SP&M’s (2022, p. 9) equation 1 does not specify that academic composition has a direct effect on student achievement. In addition, academic composition is specified as mediating the effects of schoolSES. There is no reason to specify direct effects for schoolSES but not for school academic composition. Their specification means that the effects of schoolSES are exaggerated because they incorporate the stronger effects of academic composition. A more appropriate specification would be their model 1 equation with the addition of academic composition.
Tan et. al. (2023, p. 17) conclude from their metaanalysis that schoolSES is not associated with several school processes: instructional programs, educational resources, and parental involvement. If schoolSES is as important to research and policy as SP&M (2022) contend, it should be associated with at least one of these school processes.
Reanalysis of the 2017 NAPLAN data
The data analysed is from the 2017 NAPLAN tests administered by the Australian Curriculum, Assessment and Reporting Authority (ACARA).
The Australian educational system comprises three types of schools: government, Catholic, and private. Government schools have the largest share of enrolments (65%), followed by Catholic schools (20%), and about 15% of students enrol in private schools (Australian Bureau of Statistics, 2021).
Following SP&M, these analyses are of Year 5 (primary school) and Year 9 (secondary school) students. Appropriate prior achievement measures are only available for Years 5 and 9.
Measures
NAPLAN measures student achievement in Years 3, 5, 7, and 9. Multiplechoice and constructed response test items assess student performance in numeracy, reading, spelling, and grammar. A tencriteria rubric assesses students’ writing (ACARA, 2018, pp. 7–9, 31–32). For each domain students’ scores range from 0 to 1000 across all four Yearlevels, with a mean of 500 and a standard deviation of 100. NAPLAN scores are “conditioned” using several predictor variables to increase the precision of the estimates (ACARA, 2018, p. 32). Conditioning has little impact on the overall and subgroup means but reduces the standard deviations and standard errors (von Davier et al., 2009).
The models analysed include students’ socioeconomic status (SES), sex, Indigenous status, language background, schoolSES, school sector, and student and schoollevel prior achievement.
Student SES is measured by the number of years of parents’ formal education, averaged if data from both parents are available. The measures are centred around the respective calendar year means. This SES measure differs from SP&M’s (2022) SES measure, a composite of parental education and occupation. However, the NAPLAN data on parental occupation is crude, with only four very heterogenous occupational categories, and there is considerable missing data.^{Footnote 3} The standardized estimates for studentlevel parental education estimated in this study are almost identical to SP&M’s (2022) standardized estimates for SES (discussed in “Results” section).
For this study, schoolSES is measured by school means for students’ parents’ education.
For the dichotomous categorical variables, female, Aboriginal and Torres Strait Islander (ATSI), NonEnglishspeaking background (NESB) students, and students attending nongovernment (Catholic or independent) schools are contrasted (scored one) with male, nonindigenous, English language background students and students attending government schools (scored zero). In the NAPLAN data, it is not possible to distinguish students attending Catholic and independent schools.
Samedomain prior achievement is measured by students’ NAPLAN scores two years earlier, Year 3 for Year 5 and Year 7 for Year 9. Schoollevel prior achievement was constructed by first standardizing, within domains, Year 3 (and Year 7) achievement scores, then averaging the nonmissing scores for each student and then calculating mean school prior achievement for each school. In contrast, SP&M measure school prior achievement as a latent variable so that student scores are manifestations of an underlying latent (schoollevel) construct.
Univariate statistics
Table 1 presents summary statistics for the analysed variables and Table 2 the bivariate correlations.
Table 1 shows that the data comprises about 300,000 Year 5 students and around 280,000 Year 9 students. These are population data, so sampling statistics are not required. About onequarter of students are from nonEnglish speaking backgrounds, and 6% are Indigenous. About 30% of Year 5 students attend nongovernment schools and about 40% in Year 9.
Differences between prior achievement and achievement are larger for primary school students than for secondary school students. The average difference between Year 7 and Year 9 student achievement scores is 39 score points, substantially smaller than the average 77 score point difference between Years 3 and 5.
Correlations
Table 2 presents the bivariate correlations among the study variables.^{Footnote 4} For numeracy, the mean scores for female students are slightly lower than those for male students. For the other four domains, girls exhibit higher mean scores, especially for writing and grammar. NESB students generally exhibit higher mean scores than Englishspeaking background students, except for reading. The mean scores of ATSI students are consistently and substantially lower than those of nonATSI students. Students attending nongovernment schools exhibit higher average scores than students attending government schools, with larger differences at Year 9.
The correlations between parents’ education and achievement range from about 0.3 to 0.4, higher than that obtained from metastudies in which the average correlations of parents’ education and achievement are generally between 0.2 and 0.3 (Harwell et al., 2017, p. 207; Liu et al., 2022, p. 2879; Sirin, 2005, p. 433; White, 1982, p. 470).
Table 2 shows that schoollevel prior achievement has stronger correlations with achievement than schoolSES for all achievement measures. This finding undermines SP&M’s (2022) contention that academic composition only mediates the relationship between schoolSES and student achievement. Stronger correlates do not usually mediate the effects of a weaker correlate.
SchoolSES and academic composition are highly correlated: 0.78 for Year 5 and 0.85 for Year 9. The strong correlations are important to the debate about SEC effects. If academic composition is not controlled for, then SEC effects on student achievement are spurious because they incorporate the effects of academic composition.
The high correlation between schoolSES and academic composition does not preclude obtaining estimates for both measures in model 2. There is no indication of problematic high multicollinearity. For example, for Year 5 numeracy the tolerance statistics for these variables are about 0.35, well above the cutoff of 0.10 below which indicates unacceptably high multicollinearity. The variance inflation factors are around 2.8, well below the cutoff of 10. For Year 9, the corresponding multicollinearity statistics are 0.23 and 4.2 (for details on the multicollinearity statistics see SchreiberGregor, n. d.).
The most striking feature of Table 2 is the strong samedomain overtime correlations, that is, between achievement and prior achievement (italicized). They range from about 0.60 for writing to 0.87 for secondary school numeracy and spelling, which are as large as some test–retest correlations. Only for numeracy is the prior achievementachievement correlation appreciably larger in Year 9 (0.87) than in Year 5 (0.79). Large intradomain correlations are routinely found in longitudinal studies of student achievement (Marks, 2022). These sizable correlations indicate that student achievement is highly stable, especially for spelling and secondary school numeracy.
What accounts for the stability of student achievement? An obvious candidate is socioeconomic background. However, the relationship between SES and achievement are too weak for SES to account for the stability of student achievement. Prior achievement has much stronger correlations with achievement. For these NAPLAN data, prior achievement accounts for about 36% of the variance in writing and between 50 and 75% in the other domains.
Behavioural genetics provides the most plausible explanation for the high stability of student achievement. Between 40 and 80% of the variance in NAPLAN scores is attributable to genetics (Grasby et al., 2016). Grasby and Coventry (2016, p. 649) conclude that “stability in performance was primary due to genes”.
Another notable feature of Table 2 is the sizable interdomain correlations. Because they all tap skills relating to English literacy, the intercorrelations for reading, writing, spelling, and grammar should be greater than their correlations with numeracy, which involves quite different skills. However, the correlations of numeracy with the four literacy domains are comparable to the intercorrelations for the literacy domains. The high interdomain correlations can also be accounted for by shared genes. Grasby et. al. (2016, p. 644) conclude that genetics accounts for about three‑quarters of the observed interdomain correlations in NAPLAN.
Student and school effects on student achievement
Multilevel modelling is the most common statistical procedure for isolating school effects. The standard twolevel model comprises random effects for schools and fixed effects for studentlevel and schoollevel predictors (see Hox, 2010; Snijders & Bosker, 2012). For this study, the coefficients and other parameters were estimated using PROC MIXED in SAS (Singer, 1998). In contrast, SP&M (2022, p. 13) used multilevel path models with student and school components.
Estimates are from three multilevel models. The first model comprises studentlevel demographic and socioeconomic variables, schoolSES (measured by schoollevel parental education), and samedomain prior achievement, equivalent to SP&M’s (2022, p. 9) model summarized by their equation 1. Model 2 adds schoollevel prior achievement so that SEC effects do not incorporate school differences in prior achievement. Model 3 removes SEC to assess its importance to the parameter estimates and the models’ explanatory power. In other words, is SEC important in accounting for variation in student achievement?
The equation for the general model is:
where \(Ach_{ijt = 2}\) is the achievement score of student i in school j at time 2, \(b_{0}\) is the intercept, \(b_{1}\) to \(b_{4}\) are the coefficients for the respective studentlevel predictor variables, \(b_{5}\) is the effect for attending a nongovernment school, \(b_{7}\) is the coefficient for prior achievement, \(b_{6} \;{\text{and}}\;b_{8}\) are the coefficients for schoolSES and schoollevel prior achievement, \(\delta_{0j}\) is the schoollevel residual variance and \(\varepsilon_{ij}\) is the studentlevel residual variance.
Tables 3 and 4 report estimates for the student and schoollevel predictors, the percentages of variance accounted for by schools, the percentages of the total variance in student achievement accounted for in each model, and the numbers of students and schools. The tables include the standardized coefficients (β) for parents’ education, SEC, and schoollevel prior achievement to allow for comparisons between and within models (LewisBeck et al., 2004).
Cohen (1988, pp. 79–81) classified correlations, which are equivalent to standardized coefficients in bivariate regression, as small (≈ 0.1), medium (≈ 0.3), and large (≈ 0.5). In this study, “very large” refers to standardized coefficients greater than 0.7 and “very small” to standardized coefficients around 0.05 or less.
The interclass correlation (ICC) is the betweenschool variance as a proportion of the total variation in students’ scores (Hox, 2010, p. 15; Snijders & Bosker, 2012, pp. 18, 52). In Tables 2 and 3, the ICCs are the schoollevel residual variance \(\left( {\delta_{0j} } \right)\) as a percentage of the total studentlevel variance for each domain. The pseudo R square is the square of the correlation between the dependent variable and the predicted values from the model [r(Y, Ŷ)]^{2} converted to percentages. It is a measure of model fit and allows comparison of the predictive power of the models. It is equivalent to the unadjusted R square in Ordinary Least Squares regression. In this context, it encompasses variance accounted for by schools as well as the variance accounted for by the predictor variables.
Results
Studentlevel effects
Model 1 in Tables 3 and 4 shows that boys outperform girls in numeracy but underperform compared to girls in writing, spelling, and grammar. For reading, there is little difference. The differences in favor of girls tend to be larger in Year 9 than Year 5. In Year 9 writing, girls score, on average, 18 score points higher than boys.
NESB students outperform Englishspeaking background students in numeracy, writing, and especially spelling but not in reading. Differences in language background are slightly smaller in Year 9 than in Year 5.
Differences between Indigenous (ATSI) and nonIndigenous students are larger than those for the other binary contrasts examined, especially for writing and grammar. Differences in writing by Indigenous status is larger in Year 9 than in Year 5, but not for the other domains.
In Year 5, a oneyear increase in parents’ education is associated with a rise of only four score points in numeracy and five score points in reading. The Year 5 standardized coefficients for parents’ education are around 0.10 but lower for spelling (β = 0.05). The effects of parental education are smaller in Year 9 than in Year 5. These estimates indicate that SES is only a weak predictor of student achievement when considering prior achievement.
Prior achievement is a powerful predictor. The estimates are for a oneunit change in the respective prior achievement measure on achievement. Since achievement measures range from zero to 1000, the effects of prior achievement appear small, always less than one (model 2, Table 3). However, the magnitudes of standardized coefficients for prior achievement are large or very large. For Year 5, the standardized coefficients range from 0.50 for writing to 0.81 for spelling. For Year 9 numeracy, the standardized coefficients for prior achievement range from 0.51 for writing to 0.80 and 0.84 for numeracy and spelling. For comparison, the standardized coefficients for parental education are between 0.04 and 0.09.
The coefficients for attending a nongovernment school on Year 5 achievement hover around zero, and sometimes they are negative. For Year 9, school sector coefficients are positive but small. The small coefficients are not because school sector and schoolSES are strongly correlated. The coefficients for attending a nongovernment school are only slightly larger after removing schoolSES from the model. The absence of school sector differences in NAPLAN achievement is consistent with the literature (Larsen et al., 2023).
Table 5 compares the standardized estimates for prior achievement and SES with the standardized effects from SP&M’s (2022) analyses. The two sets of estimates are similar despite different methodological approaches and slightly different measures of SES. Both analyses show small effects of SES and large effects for prior achievement. The magnitudes of the standardized coefficients for prior achievement are 5 to 20 times those for student SES.
Between school differences
Model 1 indicates that little of the variation in student performance in NAPLAN is between schools. For Year 5, the betweenschool variation as a percentage of the total variance ranges from 4% (reading) to 7% (numeracy, writing, and grammar). For Year 9, the betweenschool variation is largest for numeracy at around 8%, which may reflect differences in mathematics teaching between secondary schools. For the literacy domains in Year 9 the betweenschool variation in is only 3 or 4%. These estimates indicate that schools vary little in average student achievement when considering students’ prior achievement and other variables in model 2.
Unstandardized schoolSES effects
The effects of schoolSES are small, even without including prior achievement (model 1). For Year 5, a onestandarddeviation increase in SEC, on average, increases student achievement by between 2 (spelling) and 7 score points (writing). For Year 9, schoolSES effects are slightly larger, ranging from 3 (spelling) to 12 (writing). The larger coefficients for writing are likely due to its weaker correlation with prior achievement.
SP&M (2022) converted the small estimates from Figure 6 to percentages of achievement growth. They claim that the difference in student achievement growth between low and middle SEC schools is 11% for primary schools and 31% for secondary schools. These are misleadingly high percentages. It is very unusual to convert an estimate to a percentage of average achievement growth. The model analysed already incorporates achievement growth because it includes achievement measures at two time points. The main reason that SEC appears more important in secondary schools than primary schools is because in NAPLAN there is less achievement growth in secondary schools.
Standardized schoolSES effects
For Year 5, the standardized coefficients for schoolSES are small or very small, ranging from 0.02 to 0.07. For Year 9, the respective standardized coefficients are also small or very small, ranging from 0.03 to 0.12.
SP&M’s (2022, p. 12) standardized schoolSES (SEC) estimates can be approximated from Figure 3 which summarizes their simulation analyses. The estimates are similar to the estimates from the present study. The first panel of Figure 3 includes SEC estimates (black dots) with no additional measurement error for Year 5. The estimates are not labeled by their NAPLAN domain nor discussed in the text, but it is possible to match coefficients to domains by their relative magnitudes. The smallest Year 5 SEC estimate is about 0.025, close to the present study’s estimate of 0.02 for spelling. The largest estimate is 0.12, probably for writing, larger than this study’s Year 5 estimate of 0.06. The estimates for numeracy, reading, and grammar are between 0.06 and 0.08, corresponding to estimates between 0.05 and 0.07 in the current study.
The standardized estimates for Year 9 are in the third panel of Figure 3. The smallest estimate (0.035) corresponds to the present study’s estimate of 0.03 for spelling. The largest estimate is about 0.14, corresponding to the current study’s estimate of 0.12 for writing. SP&M’s (2022, p. 12) three other SEC estimates for Year 9 are about 0.08 and 0.10, not too different from the schoolSES estimates estimated in this study of 0.07, for numeracy and reading and 0.08 for grammar.
So, SP&M (2022, p. 12) standardized estimates for schoolSES effects discerned from Figure 3 are small and similar to the estimates in this study. They are not around 0.32 for Year 5 and 0.48 for Year 9.
Academic composition
Model 2 provides the most realistic parameter estimates since it includes academic composition (i.e., schoollevel prior achievement). With the addition schoollevel prior achievement, the schoolSES coefficients are trivial. The standardized effects are all close to zero indicating that schoolSES is of no substantive importance. In contrast, the effects of schoollevel prior achievement are nontrivial. A onestandarddeviation difference in schoollevel prior achievement increases achievement scores by between 5 score points (spelling) and 18 school points (writing) in Year 5 and between 6 and 20 score points for Year 9. The standardized effects for schoollevel prior achievement range from 0.04 to 0.15, so they are small or very small.
Model 3 removes schoolSES from the analysis. The effects of schoollevel prior achievement tend to be slightly larger, although the respective standardized coefficients remain small: 10.5 (β = 0.08) for numeracy, 10.4 (β = 0.07) for reading, 17.2 (β = 0.14) for writing, 4.7 (β = 0.03) for spelling and 14.7 (β = 0.08) for grammar. So, there are small effects in the expected direction for academic composition. The removal of SEC did not substantially change the coefficients for the predictor variables, nor did its removal reduce model fit, reiterating the conclusion from Model 2 that schoolSES has little or no impact on student achievement.
Discussion
SP&M’s (2022, p. 17) proposed reforms to Australian education are based on purportedly large schoolSES effects. However, schoolSES effects are effectively zero in correctly specified models that include student and schoollevel prior achievement. Therefore, their policy recommendations are not credible. Any assessment of the importance of schoolSES or any other schoollevel factor must consider student and schoollevel prior achievement.
SP&M (2022, p. 16) contend that parental choice of governmentsubsidized private schools is a “key driver” of schoolSES effects in Australia, although they were unable to distinguish private school students from other nongovernment school students. Private schools enrol only about 15% of students, so cannot be the “key driver” for schoolSES effects. Even if the definition of ‘private schools’ is extended to include Catholic schools, most students attend government schools. SP&M (2022) provide no evidence that SEC effects are absent from government schools. The key driver for the schoolSES effects estimated by SP&M, and in model 1 in the current study, is schoollevel prior achievement. SEC and schoollevel prior achievement are highly correlated and SEC effects disappear with the addition of schoollevel prior achievement.
Australia’s research and policymaking communities are wedded to the sociodemographicschools paradigm, which assumes that student outcomes differ because of SES, demographic factors, and schools. This paradigm dominates educational research and policy in Australia, and in other OECD countries (OECD, 2019; Thomson, 2018). SP&M’s (2022) paper belongs to the sociodemographicschools paradigm, focusing on the supposed importance of schoolSES.
This sociodemographicschools paradigm cannot account for the empirical realities of student achievement: the large interdomain correlations, the even larger samedomain overtime correlations, the weak contemporaneous effects of SES, the very much larger effects of prior achievement, the small proportions of variation in student performance attributable to schools, the negligible impact of schoolSES and the small effects of academic composition. However, the educational research and policymaking communities, almost without exception, ignore these empirical realities and continue to focus on SES.
Availability of data and materials
The data can be obtained from Australian Curriculum, Assessment and Reporting Authority (ACARA). There is a data access web page, https://www.acara.edu.au/contactus/acaradataaccess.
Notes
In this paper schoolSES, socioeconomic context and school socioeconomic composition (SEC) are synonymous.
Similarly, academic composition, academic context and schoollevel prior achievement are synonymous.
ACARA provide details of the four parental occupational categories (https://www.nap.edu.au/information/glossary).
The correlations can be converted to Cohen’s d effect sizes for a more meaningful metric (https://www.escal.site/). The effect sizes are about twice the respective correlations.
Abbreviations
 ACARA:

Australian Curriculum, Assessment and Reporting Authority
 NAPLAN:

National Assessments in Performance—Literacy and Numeracy
 SP&M:

Sciffer, Perry and McConney (2022)
 SEC:

School socioeconomic composition
 SES:

Socioeconomic status
 LOTE:

Language background other than English
 ATSI:

Aboriginal and Torres Strait Islanders
References
ACARA. (2018). NAPLAN 2018 technical report. https://nap.edu.au/docs/defaultsource/defaultdocumentlibrary/2018_naplan_technical_report_full_v1.pdf
Armor, D. J., Marks, G. N., & Malatinszky, A. (2018). The impact of school SES on student achievement: Evidence from US statewide achievement data. Educational Evaluation and Policy Analysis, 40(4), 613–630. https://doi.org/10.3102/0162373718787917
Australian Bureau of Statistics. (2021). Schools. https://www.abs.gov.au/statistics/people/education/schools/latestrelease
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.
Dumay, X., & Dupriez, V. (2008). Does the school composition effect matter? Evidence from Belgian data. British Journal of Educational Studies, 56(4), 440–477. https://doi.org/10.1111/j.14678527.2008.00418.x
Grasby, K. L., & Coventry, W. L. (2016). Longitudinal stability and growth in literacy and numeracy in Australian school students. Behavior Genetics, 46, 649–664. https://doi.org/10.1007/s1051901697960
Grasby, K. L., Coventry, W. L., Byrne, B., Olson, R. K., & Medland, S. E. (2016). Genetic and environmental influences on literacy and numeracy performance in Australian school children in grades 3, 5, 7, and 9. Behavior Genetics, 46(5), 627–648. https://doi.org/10.1007/s105190169797z
Harwell, M., Maeda, Y., Bishop, K., & Xie, A. (2017). The surprisingly modest relationship between SES and educational achievement. Journal of Experimental Education, 85(2), 197–214. https://doi.org/10.1080/00220973.2015.1123668
Hox, J. (2010). Multillevel analysis: Techniques and applications (2nd ed.). Routledge.
Larsen, S. A., Forbes, A. Q., Little, C. W., Alaba, S. H., & Coventry, W. L. (2023). The public–private debate: School sector differences in academic achievement from year 3 to year 9? The Australian Educational Researcher, 50, 275–306. https://doi.org/10.1007/s1338402100498w
Lauder, H., Kounali, D., Robinson, T., & Goldstein, H. (2010). Pupil composition and accountability: An analysis in English primary schools. International Journal of Educational Research, 49(2), 49–68. https://doi.org/10.1016/j.ijer.2010.08.001
LewisBeck, M., Bryman, A., & Futing Liao, T. (2004). Standardized regression coefficients. In M. S. LewisBeck, A. Bryman, & T. F. Liao (Eds.), The SAGE encyclopedia of social science research methods. Sage. https://doi.org/10.1016/0049089X(74)90006410.4135/9781412950589
Liu, J., Peng, P., Zhao, B., & Luo, L. (2022). Socioeconomic status and academic achievement in primary and secondary education: A metaanalytic review. Educational Psychology Review, 34(4), 2867–2896. https://doi.org/10.1007/s1064802209689y
Lu, L., & Rickard, K. (2014). Value added models for NSW government schools. http://www.cese.nsw.gov.au/images/stories/PDF/VAPaper_v31Final.pdf
Marks, G. N. (2015). Are schoolSES effects statistical artefacts? Evidence from longitudinal population data. Oxford Review of Education, 41(1), 122–144. https://doi.org/10.1080/03054985.2015.1006613
Marks, G. N. (2022). The overwhelming importance of prior achievement when assessing school effects: Evidence from the Australian national assessments. School Effectiveness and School Improvement, 34, 1–25. https://doi.org/10.1080/09243453.2022.2102042
OECD. (2019). PISA 2018 results (volume II): Where all students can succeed (Vol. II). OECD Publishing.
Scheerens, J. C. J., Bosker, R. J., & Creemers, B. P. M. (2001). Time for selfcriticism: On the viability of school effectiveness research. School Effectiveness and School Improvement, 12(1), 131–157. https://doi.org/10.1076/sesi.12.1.131.3464
SchreiberGregor, D. N. (n. d.). Multicollinearity: What is it, why should we care, and how can it be controlled? SAS Institute. https://support.sas.com/resources/papers/proceedings17/14042017.pdf
Sciffer, M. G., Perry, L. B., & McConney, A. (2022). The substantiveness of socioeconomic school compositional effects in Australia: Measurement error and the relationship with academic composition. LargeScale Assessments in Education, 10(1), 1–22. https://doi.org/10.1186/s40536022001428
Singer, J. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioural Statistics, 24(4), 323–355. https://doi.org/10.3102/10769986023004323
Sirin, S. R. (2005). Socioeconomic status and academic achievement: A metaanalytical review of research. Review of Educational Research, 75(3), 417–453. https://doi.org/10.3102/00346543075003417
Snijders, T., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modelling (2nd ed.). Sage.
Tan, C. Y., Hong, X., Gao, L., & Song, Q. (2023). Metaanalytical insights on school SES effects. Educational Review. https://doi.org/10.1080/00131911.2023.2184329
Thomson, S. (2018). Achievement at school and socioeconomic background—An educational perspective. Npj Science of Learning, 3(1), 5. https://doi.org/10.1038/s4153901800220
van Ewijk, R., & Sleegers, P. (2010). The effect of peer socioeconomic status on student achievement: A metaanalysis. Educational Research Review, 5(2), 134–150. https://doi.org/10.1016/j.edurev.2010.02.001
von Davier, M., Gonzalez, E., & Mislevy, R. J. (2009). What are plausible values and why are they useful? In M. von Davier & D. Hastedt (Eds.), IERI monograph series: Issues and methodologies in largescale assessments (Vol. 2, pp. 9–36). Education Testing Service.
White, K. R. (1982). The relationship between socioeconomic status and academic achievement. Psychological Bulletin, 91(3), 461–481. https://doi.org/10.1037/00332909.91.3.461
Acknowledgements
The author thanks the Australian Curriculum, Assessment and Reporting Authority (ACARA) for preparing and allowing access to the consolidated studentlevel NAPLAN data.
Funding
No funding was received for this paper.
Author information
Authors and Affiliations
Contributions
Sole authored Paper.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
There are no competing interests, financial or otherwise.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Marks, G.N. No substantive effects of school socioeconomic composition on student achievement in Australia: a response to Sciffer, Perry and McConney. Largescale Assess Educ 12, 8 (2024). https://doi.org/10.1186/s4053602400196w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s4053602400196w