Effects of classroom and school climate on language minority students’ PISA mathematics self-concept and achievement scores

Abstract

Grounded in ecological theory, this study investigated relative contributions of perceived classroom and school climate variables to mathematics self-concept and achievement of English-at-home and English learner (EL) students using PISA 2012 data for American middle-grade students. For both outcomes, results of 3-step hierarchical linear regression models for the combined sample closely mirror those of English-at-home students and mask the unique characteristic of ELs. For self-concept, six (classroom management, cognitive activation, disciplinary climate, teacher support, sense of belonging, and teacher student relations) out of seven predictors were statistically significant and positive predictors for English-at-home students (teacher support being the strongest); only two predictors (disciplinary climate, and teacher student relations) were significant and positive for ELs. Similarly, group discrepancies were found for mathematics achievement. Five variables (classroom management, teacher support, disciplinary climate, sense of belonging to school, and teacher student relations), were significant predictors of English-at-home students. Yet, only three variables (classroom management, disciplinary climate, and teacher support) significantly predicted achievement of ELs. Classroom climate was consistently an important predictor across outcomes and student populations and was the strongest contributor for ELs. Implications and future directions are discussed.

Introduction

Mathematics is an important subject for all students to learn. Not only does it afford students a critical foundation to pursue science, technology, engineering, and mathematics (STEM) careers, it cultivates thinking skills (e.g., logical reasoning, problem solving) and dispositions (e.g., perseverance) that can support students in immerging knowledge-based economy (Melguizo & Wolniak, 2012). There is accumulating evidence that learning in general and learning mathematics in particular is a complex and nuanced process impacted by students’ individual characteristics, current and accumulated classroom experiences, and school-related factors (Engberg & Wolniak, 2013; Han, 2019). These factors not only impact students’ mathematics achievement on standardized tests, but also shape their mathematics self-concept, the evaluative perceptions about themselves as doers of mathematics (Dicke et al., 2018; Parajes & Schunk, 2001). Yet research about how these factors may impact different populations of students, in particular language minority students, is still limited (Sandilos et al., 2020).

As the student population in the United States becomes more diverse, the teaching profession as a whole becomes less diverse. Currently, over five million students enrolled in the U.S. public schools speak a language other than English at home (Hussar et al., 2020). At the same time, schoolteachers and leaders currently lack diversity—the majority are white, middle class, monolingual—and are projected to become even less diverse (Hansen & Quintero, 2019). As reported by the U.S. Department Of Education National Center for Statistics (NCES, 2020), the demographics of teachers nationally indicate that 79.3% of the teachers are White, non-Hispanic; 6.7% are Black, African American, non-Hispanic; 9.3% are Hispanic, regardless of the race; 2.1% are Asian, non-Hispanic; 0.2% are Native Hawaiian/Pacific Islander, non-Hispanic; 0.5% are American Indian/Alaska Native, non-Hispanic; and 1.8% are Two or more races, non-Hispanic. This demographic imbalance and a lack of preparation among school teachers and leaders to work with diverse students (e.g., Goodwin, 2017; King & Butler, 2015; Okhremtchouk & Sellu, 2019) may be one of the factors underlying the long-existing achievement gaps between language minority (LM) and English-speaking language majority students, especially in STEM subjects. In 2019, for example, the average National Assessment of Educational Progress (NAEP) mathematics score for 4th-grade LM students was 24 points lower than that of English-speaking majority students; the 8th-grade LM gap was 42 points (Hussar et al., 2020, pp. 85–86). For these reasons, there is a need to understand what students with culturally and linguistically different backgrounds might need to support their mathematics achievement and self-concepts, especially in higher grades.

Recognizing that LM students negotiate multiple academic and social contexts that influence their learning and the still-limited body of research focused on this important student population, our exploratory study seeks to examine the impacts of two such layers of social contexts. In particular, within an ecological model (Bronfenbrenner, 1977) and using U.S. data from the large-scale assessment Programme for International Student Assessment (PISA), we examine the contributions of classroom learning environment and school climate to students’ mathematics achievement and self-concept with particular attention to how these factors impact LM versus English-speaking majority students.

Developing more nuanced understandings of what works for different student populations to achieve “success for all students” is paramount as literature highlighting what works for advantaged students “typically fails to reveal the social and cultural advantages that make their success possible” (p. 76) and generating more research on successfully educating minority students remains a high priority (Ladson-Billings, 2014). Focusing on better understanding the impacts of classroom learning environment and school climate is particularly important as current culturally and linguistically responsive pedagogy literature (e.g., Aguirre & Zavala, 2013; Fulton, 2009; see Aronson & Laughter, 2016; de Araujo et al., 2018) emphasizes the importance of teachers and schools in supporting LM students.

We recognize that LM students are a heterogeneous group of students who could be referred to in the literature as culturally and linguistically diverse, emergent bilingual, English learner, or multilingual students. In the present investigation, we use the term LM students to more closely align with how PISA categorizes student populations by language-at-home status.

Study background and conceptual framing

Large international assessments play a vital role allowing educational researchers and policymakers to compare effectiveness, gauge progress, and identify areas for improvement. Starting in 2000, PISA began administering large-scale standardized learning assessments to 15-year-olds in such fields as reading, mathematics, and science, with one of these major academic fields selected as primary (i.e., additional data collected) during each 3-year cycle. Pisa’s main objective is to use student achievement data to document and benchmark the effectiveness of international educational systems across academic domains near the end of secondary school. This age bracket is chosen by PISA because students at this age are approaching the end of compulsory education in many countries that are Organisation for Economic Co-operation and Development (OECD) members. Overall, PISA evaluates students’ abilities to produce knowledge, predictions, and applications on a specific issue, based on their understanding of concepts and situations (OECD, 2009a, 2009b). This data informs both international comparisons (what are other countries doing that works well?) as well as internal comparisons (what groups of students should we focus policies towards?).

PISA also allows for comparisons across student population subgroups, such as language at home, and provides additional background and contextual information, such as family backgrounds, attitudes towards learning, and perceptions of school environments (OECD, 2013). Collecting comparable data across countries with various cultural backgrounds and different languages spoken is challenging (Vonkova et al., 2018). Addressing some of such challenges, the present study used USA-only sample data from the PISA 2012 cycle (OECD, 2014). The primary domain of the PISA 2012 cycle was mathematical literacy, which was defined as the capability of the individuals to create, engage, interpret, and assess mathematical concepts in a variety of settings (OECD, 2013). Variables used in the present study were not manipulated, but rather used as provided, reflecting the original context.

Aligned with PISA Framework (OECD, 2014), this study is grounded in Bronfenbrenner’s (1977) ecological systems framework which considers factors affecting learning and development across multiple systems. Such systems of interest to the present study range from the individual, an individual’s prior experiences and resources such as home language, to microsystems, settings where interaction leading to learning or development occurs, such as the classroom and the school (Crawford et al., 2019). In other words, a learning or a developmental outcome is understood as a function of interaction among individual characteristics and immediate and larger environmental contexts. The framework has been applied across age groups and learning domains (e.g., Ardasheva & Tretter, 2013; Dai et al., 2023; Farrant & Zubrick, 2012). Regarding mathematics in particular, there is emergent evidence that individual characteristics and immediate and larger environmental contexts impact both motivational (self-concept) and cognitive (academic) outcomes, with some variation depending on students’ linguistic and cultural backgrounds (Han, 2019; Sandilos et al., 2020). Thus, there is a need to explore both the existence and the directionality of such impacts on language minority versus language majority students.

In what follows, we first discuss self-concept and how it relates to academic achievement, then transition to discussing classroom and school climate as theoretical and/or empirical predictors of self-concept and achievement. While these concepts overlap and interact with one another, the following structure was adopted to provide readers with clarity and allow the authors to discuss each one in turn.

Self-concept and academic achievement

Self-concept, broadly defined, is a person's perception of themselves (Shavelson et al., 1976). It is distinct from self-efficacy, which is typically about the completion of discrete tasks and involves immediate feedback (success or failure), whereas self-concept includes other appraisals including how the task may make you feel (e.g., anxious) or whether you belong (e.g., ‘if no one in class looks like me, do I belong in this class?’; Bong & Clark, 1999). These broader self-concept perceptions are formed through one’s experience with and interpretations of one's environment, and are influenced by reinforcements, evaluations of significant others, and one's attributions for one's own behavior.

Self-concept can be defined by seven critical features outlined in Shavelson et al. (1976) (For a quantitative integration of these theoretical constructs, see Marsh et al., 2018). First, self-concept is organized in such a way that people can categorize information about themselves and relate these categories to one another (Aronson & Steele, 2005). Second, self-concept is multifaceted, with each facet reflecting the category system adopted by a particular individual (regardless of what that category system is). Third, self-concept becomes increasingly multi-faceted as the individual develops from infancy to adulthood (Cvencek et al., 2011). Fourth, self-concept is hierarchical, with perceptions of behavior at the base, moving to inferences about self in subareas, and then to general self-concept (Green et al., 2012). Fifth, general self-concept is stable but, as one ascends the hierarchy, self-concept becomes increasingly situation-specific and, therefore, less stable (Anderman & Midgley, 1997) Sixth, self-concept has both a descriptive and an evaluative dimension such that individuals may describe themselves (‘I am happy’) and evaluate themselves (e.g., ‘I am good at school’) (Byrne & Shavelson, 1986). And finally, self-concept can be differentiated from other constructs, such as academic achievement (Bong & Skaalvik, 2003). (The specific aspects of mathematics self-concept captured by PISA items are summarized in Appendix C.

For example, let us take Robert, an 18-year-old student whose primary language is Spanish, as he enrolls in his first engineering class at Local State University. Having completed high school, Robert has a relatively well-developed self-concept including his ability at academic disciplines, sports, social life, etc. Overall, his self-concept should be fairly stable (e.g., ‘I am a happy, sociable person who is good at school’), but as he engages in different courses, his self-concept for specific disciplines may become more nuanced, continually updated to reflect his experiences and interpretations of those experiences (‘Overall I am a good student; I still need to work on improving my English skills, but I am great at mathematics’). Such continuous changes over time may create a tension for Robert, because while some courses are explicitly language-based (English 101), others may rely heavily on English for instruction (Mathematics 101). This tension may have drastic outcomes for Robert’s future academic choices. For example, if Robert continues experiencing significant difficulties in college mathematics courses, and attributes those difficulties to his mathematics self-concept (rather than his English language skills); he may be discouraged from pursuing a degree in engineering (or perhaps drop out of the university entirely).

Self-concept can be distinguished from academic achievement, although research on the relation between self-concept and indices of academic achievement such as grades and test scores has found positive correlations and reciprocal influences (Arens et al., 2017). And as one would expect, as the measure of self-concept becomes more nuanced (from general self-concept to academic self-concept to mathematics self-concept), the relationship between self-concept and achievement becomes stronger. For example, while general self-concept predicts academic achievement, academic self-concept is a stronger predictor, and indeed subject matter self-concept (mathematics or English, for example) is an even better predictor (e.g., Lauermann et al., 2020; Marsh & Shavelson, 1985). Said another way, thinking of yourself as a science person would positively predict achievement in a physics class, whereas thinking of yourself as a physics person would most likely be an even stronger predictor of physics achievement.

Let us return to Robert. As he continues through college, he begins to delineate and further refine his mathematics self-concept. He may begin to specialize in courses that are less reliant on English and seek out classes and majors in which he feels he can excel. This self-selected delineation has been demonstrated as students move through their formal academic settings. For example, Marsh (1986) found that academic self-concept had a direct effect on subsequent school performance beyond the effects of academic ability and prior school performance. Thus, after accounting for both ability and previous school achievement, academic self-concept still predicted a significant amount of the variance in school performance, indicating that the way students feel in the classroom (if they are challenged, feel they belong, and feel that they can do well) is carried not only into the specific grade, but since academic self-concept is relatively stable, they carry this with them throughout their schooling experience. So, while a number of researchers have focused on younger students in an attempt to alter their science or mathematics trajectories (Polikoff et al., 2018; Sinatra et al., 2017), any intervention has the potential to influence how students engage with coursework in the future.

Kurtz-Costes and Schneider (1994) and Arens et al., (2017) found that self-concept and achievement are bi-directionally related, thus creating a positive or negative spiral. This is similar in nature to the “Matthew Effect” as described by Stanovich in 1986 (albeit related to mathematics rather than reading). For example, consider a student that feels they have a strong mathematics self-concept. They will likely enroll in higher level (and more challenging) mathematics courses, spend more time engaged with mathematics content (in and out of school), report more positive attitudes towards mathematics concepts, and ultimately do better on standardized tests of mathematics achievement. These actions will further strengthen their mathematics self-concept, producing a feedback loop wherein they continue to lean into mathematics content. In contrast, a student who feels that they have a weak mathematics self-concept will likely avoid higher level mathematics courses (and perhaps enroll in “easier” math courses to fulfill requirements), spend less time engaged with mathematics content (in and out of school), report more negative attitudes towards mathematics concepts, and ultimately perform worse on tests, further weakening their mathematics self-concept.

It is important to note that beside this self-fulfilling prophecy effect, environmental factors (e.g., parents, teachers, classmates) can have a large influence on students’ self-concepts as well as on academic achievement, which is not surprising, provided the reciprocal relationships (Arens et al., 2017; Kurtz-Costes & Schneider, 1994) between these two outcomes. That is, left alone, these feedback loops can exacerbate the academic gaps between students with relatively small differences in mathematics self-concept. However, parents, teachers, and classmates can play a large role in supporting (or not) positive mathematics self-concept. We now turn to two of these environmental factors in students’ lives—the dynamics of the classroom and the dynamics of the school.

Classroom learning environment

Classroom learning environment has been defined as “the social, psychological, and pedagogical contexts in which learning occurs and which affect student achievement and attitudes” (Fraser, 1998, p. 3). Classroom learning environment research has been conducted around the world to study a host of learning domains such as disciplinary attitudes, achievement, teacher practices, and program evaluation (see Lim & Fraser, 2018). Typically, although with some variations, learning environments are operationalized in terms of classroom climate, teacher support, and cognitive activation (Chionh & Fraser, 2009; Fast et al., 2010).

Both theoretical (Bourdieu, 1986) and empirical (e.g., Braun, 1976; Lucas et al., 1990) research identify positive classroom learning environments as essential for supporting students’ effective learning and their positive perceptions of themselves as learners. In a large-scale study of middle school students, without sub-analyses by language status, Fraser and Kahle (2007) found that although classroom, home, and peer environment significantly contributed to students’ attitudes toward mathematics and science, only classroom environment significantly predicted achievement. In a meta-analysis of 119 studies conducted across a range of educational settings but with no sub-analyses by language status, Cornelius-White (2007) found strong associations between classroom environments and student social and academic outcomes. Teacher-student relationships had the strongest associations with student outcomes. Analyses also indicated that classroom characteristics had positive association with both student social/behavioral outcomes such as self-beliefs (average r = 0.35) and cognitive/academic outcomes such as grades (average r = 0.31). Regarding academic outcomes, Cornelius-White (2007) found that the highest associations were with mathematics (r = 0.36) and verbal (r = 0.34) achievement.

Studies focused on mathematics (e.g., Lewis et al., 2012; Riconscente, 2014) also found that different aspects of classroom learning environments may impact student mathematics attitudes and achievement either directly or indirectly via self-beliefs and/or disciplinary attitudes. In a longitudinal study of 1163 upper elementary students, Fast et al. (2010) found that classroom environments perceived by students as more caring, challenging, and mastery-oriented were associated with higher levels of mathematics self-beliefs, which, in turn, predicted higher levels of mathematics performance. Mathematics-focused research also documented higher associations between classroom characteristics and student social/behavioral outcomes (self-beliefs, motivation, interest) than between classroom characteristics and academic outcomes (end-of-the-year grades, achievement; Fast et al., 2010). Riconscente (2014) found that classroom environments—in combination with student individual difference characteristics such as beginning-of-the-year self-beliefs—accounted for about 61% and 36% of the variance in students’ mathematics interest and self-beliefs, respectively, but only for about 24% of the variance in student end-of-the year grades.

Although limited and inconsistent (e.g., Hsieh et al., 2019), there is evidence to suggest that positive learning environments may have a greater impact on LM students’ perceptions of ability and achievement. In a study of 1119 secondary language majority and minority students, den Brok et al. (2010) found that classroom environment accounted for a substantial amount of the variance in student subject-related attitudes and achievement across comparison groups. The relationship with student academic achievement was only indirect among majority and second-generation LM students. Mediated by attitudes toward the subject studied, this relationship was direct among first-generation LM students. Interestingly, for this student population, attitudes toward the subject studied had no direct effect on academic achievement. Den Brok et al. attributed such greater reliance on the teacher—rather than on one’s own attitudes toward the subject—to plausibly fewer economic resources available to this group of students.

Similarly, Lewis et al.’s (2012) study found that a positive classroom learning environment bolstered can-do attitudes and achievement in mathematics; with this association being more pronounced among LM students. By contrast, Hsieh et al. (2019) found that although perceived support from teachers positively predicted disciplinary (science) self-concept and attitudes, LM (immigration) status was not significantly associated with student subject-specific motivation either initially, or in terms of changes over time. The generalizability of these findings to mathematics, especially to mathematics self-concept beliefs is yet to be investigated.

School climate

School climate has been broadly defined as school life experiences reflecting “norms, goals, values, interpersonal relationships, teaching and learning practices, and organizational structures” (National School Climate Council [NSCC], 2007, p. 5). Recognizing the potential that distinctive school cultures may affect the life and learning of students, research on school climate has seen an increase in systematic inquiry around the world (see Thapa et al., 2013). These studies have documented relationships between school climate and such outcomes as healthy relationships, engagement, and improved efforts. Schools with positive climates, in particular, were found to have lower rates of negative behaviors (e.g., suspensions), fewer student–teacher issues, and, most pertinent to our study, higher rates of socioemotional, behavioral, and academic adjustments (e.g., Shochet et al., 2006; Thapa et al., 2013).

While there is not a definitive consensus on the dimensions of school climate that are most important, there does seem to be a general agreement that the following four, interrelated facets matter: safety, teaching and learning, relationships, and the institutional environment (NSCC, no date; Thapa et al., 2013). We briefly discuss each facet below.

Safety

Maslow (1943) argued that a feeling of safety—mental, emotional, physical, and intellectual safety—is a basic human need that must be met in order for individuals to flourish. Safety is typically established through the supportive norms, rules, relationships, and structures of a school. In schools lacking these supportive structures, students are more likely to suffer “violence, peer victimization, and punitive disciplinary actions” (Thapa et al., 2013 p. 4). As a consequence, these schools also tend to have high levels of absenteeism and lower academic achievement (Astor et al., 2009). Unfortunately, although the adults associated with schools report perceiving these threats as “mild-to-moderately severe,” students tend to report them as “severe” (see Cohen, 2006). Presently, research on school safety has largely focused on school rules, structures, and bullying behaviors. While this work is important, the present study heightens the need to broaden the scope of this work to focus future research on LM students as well. The little research that exists on this student population, indicates that LM students may find themselves in less safe and supportive schools or may “perceive their schools to be less safe and supportive than their peers” (Sanders et al., 2018, p. 649; see Watkins & Melde, 2009). Importantly, while overall school climate measures are important to “take the temperature” of the school, they may not accurately capture the experiences of subgroups within the school.

Teaching and learning

At the heart of a student’s school experience is teaching and learning, one of the core facets of school climate. Schools with positive climates tend to promote teaching and learning characteristics, such as “cooperative learning, group cohesion, respect, and mutual trust,” that positively impact students’ learning environments (Thapa et al., 2013, p. 365; see Kerr et al., 2004). At all levels of schooling, research has shown that school climate is directly related to academic achievement. Though this is also true for LM students, there are particular characteristics of the school climate related to the learning environments that matter. In their examination of California schools successful in supporting Hispanic/Latino LM students, Lucas et al. (1990) found that these schools placed value on students' languages and cultures, placed high expectations on students, provided supports for teachers to specifically serve LM students more effectively and offered a variety of courses and programs to support LM students. In general, it seems the climate of teaching and learning at these schools supported teachers to meet students’ learning needs and see students’ LM status as an asset rather than a deficit.

Relationships

Another key facet of school climate is the relationships between various players in the school and the level of connection people feel toward one another (Thapa et al., 2013). Unsurprisingly, students who feel a greater sense of connection and belonging tend to exhibit fewer behavioral issues, more engagement, higher self-esteem, better grades, and less depressive symptoms (e.g., Gregory & Cornell, 2009; Jia et al., 2009). For students, some of the key relationships that impact their experiences are teacher-student and the student–student relationships. Research has shown, however, that different groups of students may perceive different relational aspects as more or less impactful. For example, Slaughter-Defoe and Carlson (1996) found that Hispanic/Latino students considered “teacher fairness, caring, praise of effort as well as the importance of moral order” (p. 60) as essential dimensions of school climate while their African American peers valued being listened to and teachers being available to comfort and help as most important. Again, not much research has explored the specific school relational-experiences of LM students. However, based on existing literature (e.g., Lucas et al., 1990), when the school staff (at all levels) are committed to empowering their LM students and take concrete actions (such as “taking extra time,” p. 325), LM students are more likely to achieve academic success.

Institutional environment

A final facet of school climate is the institutional environment, which includes both school connectedness, and the physical resources and structure of the school. School connectedness can be thought of as “the belief by students that adults and peers in the school care about their learning as well as about them as individuals” (Center for Disease Control and Prevention, 2009, p. 3). Like other facets of school climate, research has shown that when students experience a feeling of school connectedness or a sense of belonging, they have fewer behavior issues, engage in less risky behavior, and have better academic outcomes (e.g., Loukas et al., 2006; Ruus et al., 2007). The limited research on LM students does suggest that a sense of belonging, connection, and feeling welcomed at their schools is also important for this student population (Lucas et al., 1990).

Much like classroom climate, there are three main points that arise when reviewing the literature on school climate. First, the perceived school climate has important consequences for student’s physical and mental wellbeing and academic achievement and is theoretically expected to have impacts on students’ self-concepts as well (Arens et al., 2017; Kurtz-Costes & Schneider, 1994). Second, while commonalities exist, there are still important group differences which need to be better understood so that teachers and schools can support all students. Finally, for LM students in particular, there is little research about how their needs may differ from the general population. As numbers of LM students increase, there is urgency to better understand the unique impact of factors like classroom climate and school climate for this important student population.

It is noteworthy that in the present study, we have not considered the social-economic status (SES) of the participants in our analyses. This decision has been made for the following reasons. When creating the composite variable of SES—referred to as economic, social and cultural status (ESCS) in the PISA datasets—PISA integrates indicators of parents’ educational background, family wealth, and home educational resources and possessions (OECD, 2013). However, Hauser, (1994) and Sirin, (2005) argued that combining these factors when measuring SES is problematic as this traditional understanding of SES has not always been strongly associated with educational outcomes across students from different backgrounds (Thomson, 2018). For example, highly educated immigrants may not reach the same levels of wealth as non-immigrants at the same level of education.

The present study

Overall, limited work has focused on LM students, especially in STEM, and examined the relative contributions of both the classroom and school learning environments to their academic achievement and self-concept. This is unfortunate, provided that “the environment of schooling extends beyond the classroom to hallways and after-school activities of students” (Carhill et al., 2008, p. 1160) to impact students’ behavioral, socioemotional, and academic outcomes. Hence, we sought to investigate the following research questions:

• 1. What are the contributions of the classroom learning environment and school climate to students’ mathematics self-concept?

• 2. What are the contributions of the classroom learning environment and school climate to students’ mathematics self-concept when disaggregated by home language?

• 3. What are the contributions of the classroom learning environment and school climate to students’ mathematics achievement?

• 4. What are the contributions of the classroom learning environment and school climate on students’ mathematics achievement when disaggregated by home language?

Methods

This study used the 2012 PISA data on U.S. students and employed separate hierarchical linear regression models on the combined sample (all students) and on the subsamples disaggregated by home language status (to allow for main analyses of interest).

Participants

The sampling strategy of PISA 2012 administration consisted of six main phases: (a) defining the age of students to be surveyed; (b) creating a list of schools in which eligible students are enrolled; (c) selecting school sample from the list; (d) developing a list of eligible students in terms of age; and (e) selecting student sample from the list (OECD, 2014).

The current study focuses on the 4978 U.S. students who participated (number of students per analysis may vary, per administration). Almost 86% of participants indicated that English was the primary language spoken at home; 14% reported their primary language spoken at home as Spanish or Other languages. Approximately 49% of the participants self-reported their gender as female.

Data sources: PISA scales

Mathematics served as the major domain of PISA 2012 cycle, which also collected a wide range of student-, teacher- and school-level variables specific to mathematics (OECD, 2013). We used seven student-reported independent variables in our study (all measured on a 4-point Likert scale, with anchors ranging from strongly disagree to strongly agree or from never/hardly ever to every lesson, or from rarely to always) and two dependent variables (see below). Given that PISA utilized item response theory as a scaling method for the variables of interest listed below, the means and standard deviations represent the means and standard deviations after the scaling procedures (OECD, 2014).

Independent variables

Perceived prior mathematics classroom learning environment

• 1) Mathematics teacher’s classroom management (M = 0.197, SD = 1.002, a = 0.75). This scale included 4 items measuring classroom management (sample item: Thinking about the mathematics teacher who taught your last mathematics class: “My teacher keeps the class orderly”). Four response categories ranged from “strongly agree = 1” to “strongly disagree = 4.”

• 2) Mathematics teacher’s support (M = 0.254, SD = 1.055, a = 0.84). This scale included 4 items measuring mathematics teacher support (sample item: Thinking about the mathematics teacher who taught your last mathematics class: “My teacher provides extra help when needed”). Four response categories ranged from “strongly agree = 1” to “strongly disagree = 4.”

• 3) Cognitive activation in mathematics lessons (M = 0.388, SD = 1.122, a = 0.87). This scale included 9 items measuring cognitive activation in mathematics lessons (sample item: Thinking about the mathematics teacher who taught your last mathematics class: “The teacher asks questions that make us reflect on the problem”). Four response categories ranged from “strongly agree = 1” to “strongly disagree = 4.”

Perceived current mathematics classroom learning environment

• 4) Disciplinary climate (M = 0.049, SD = 0.999, a = 0.89). This scale included 5 items measuring disciplinary climate (sample item: How often to these things happen in your mathematics lessons?: “There is noise and disorder”). Four response categories ranged from “every lesson = 1”, “most lessons = 2”, “some lessons = 3”, to “never or hardly ever = 4.”

• 5) Teacher support (M = 0.163, SD = 0.973, a = 0.87). This scale included 5 items measuring teacher support (sample item: How often to these things happen in your mathematics lessons?: “The teacher gives extra help when students need it”). Four response categories ranged from “every lesson = 1”, “most lessons = 2”, “some lessons = 3”, to “never or hardly ever = 4.”

Perceived school climate

• 6) Sense of Belonging to School (M = -0.056, SD = 1.001, a = 0.32). This scale included 9 items measuring sense of belonging to school (sample item: “I feel happy at school”). Four response categories ranged from “strongly agree = 1” to “strongly disagree = 4.” Because sense of belonging to school significantly correlated with all other variables of interest in the present study, a decision was made to include it in the analyses despite relatively low reliability. This decision was grounded in previous empirical studies (i.e., Jeremie, 2017; Stankov et al., 2017). Further, higher Cronbach’s alpha does not always reflect higher degree of internal consistency as it can be influenced by other factors such as the length of a test (Tavakol & Dennick, 2011).

• 7) Teacher Student Relations (M = 0.179, SD = 0.973, a = 0.83). This scale included 5 items measuring student–teacher relations outside of mathematics classrooms (sample item: “Students get along well with most teachers”). Four response categories ranged from “strongly agree = 1” to “strongly disagree = 4”.

Please see Appendix C for a full list of items for each scale.

Dependent variables

• 1) Mathematics self-concept (M = 0.303, SD = 1.004, a = 0.90). This scale comprised 5 items measuring mathematics self-concept. Sample items include “I learn mathematics quickly” and “I get good grades in mathematics.” Four response categories ranged from “strongly agree = 1” to “strongly disagree = 4”.

• 2) Mathematics achievement (M = 481.694, SD = 86.683, a = 0.99). This scale included 5 plausible values (OECD, 2013) items gauging students’ mathematical literacy. Examples of mathematical literacy domains tested included, formulating situations mathematically; employing mathematical concepts, facts, procedures and reasoning; interpreting, applying and evaluating mathematical outcomes; space; shape, change and relationships; quantity and uncertainty and data. Appendix B provides descriptive statistics (e.g., means, standard deviation, skewness, kurtosis). Since PISA assessments cannot measure all mathematics outcomes of all students (due to time and student fatigue constraints), missing data occurred in achievement scores by design (different assessment booklets given to different students). Therefore, PISA employed an imputation strategy in the achievement scores to generate plausible variables. In other words, plausible variables refer to those achievement scores which were obtained via imputations based on existing scores (OECD, 2014).

Data analyses

Before running our analyses, we used the International Database Analyzer (IDB Analyzer) to extract the relevant data as well as to generate SPSS codes/syntax to run all statistical analyses. The IDB Analyzer is a windows-based software application created by the IEA Data Processing and Research Center (IEA-DPC). IDB Analyzer creates SAS code or SPSS syntax allowing users to work with (inter)national large-scale assessments such as PISA. The analyzer makes use of replicate weights, in addition to the sample weights, in the PISA dataset to adjust standard errors to account for the clustered sample design and allows for performing statistical analyses with plausible values (IEA, 2016; OECD, 2009a, 2009b).Once extracted, the data were screened to ensure all statistical assumptions were met. Please see Appendix A for more details.

Missing data

Although they are not classified as missing data, plausible values for mathematics achievement scores have been imputed by PISA and are provided for the public use. Plausible values can be defined as “random values from the posterior distributions” for the achievement scores which occur due to the design of PISA. That is, PISA utilizes a design wherein not all participants take the same set of questions for the mathematics achievement scores (also see the description of dependent variables above) (). That is, these imputed scores are by design (students did not have the opportunity to answer), which is in contrast to the truly missing data for the predictor variables. To handle the missing data that occur in the predictor variables, we applied a listwise deletion method, which allowed us to use all complete cases. Listwise deletion typically removes the cases with one or more missing values, and it subsequently provides all complete cases to conduct statistical analyses. Listwise deletion is one of the most common missing data handling methods (Briggs et al., 2003; Peugh & Enders, 2004). It also has an advantage to be used with any kind of statistical analysis (Nakai & Ke, 2011). Several studies in which PISA 2012 data was used have been conducted using only complete cases within the predictor variables (e.g., Caro et al., 2016; Salas-Velasco & Sánchez-Campillo, 2018).

Analysis

For both dependent variables, separate 3-step hierarchical linear regression models were run on the combined sample (to establish baseline) and on the subsamples desegregated by home language status (to allow for main analyses of interest). This approach helps explain the unique variance of predictors entered into the model in order of their theoretical/hypothesized importance through significant improvements in R² (Cohen, 2014). The order of variable entry into models was as follows: perceived prior mathematics classroom learning environment (mathematics teacher’s classroom management, mathematics teacher’s support, and cognitive activation in mathematics lessons), perceived current mathematics classroom learning environment (disciplinary climate, teacher support), and finally school-related factors (sense of belonging to school, teacher student relations).

Results

Research question 1

Table 1 summarizes the results of the school and classroom climate contributions to mathematics self-concept in three steps. These included, perceived prior mathematics classroom learning environment (Step 1), perceived current mathematics classroom learning environment (Step 2), and perceived school climate (Step 3) factors. Results indicated that, except for mathematics teacher’s support, all classroom- and school-related climate variables were significant predictors across all models.

Table 1 School and Classroom Climate Contributions to Mathematics Self-Concept: Hierarchical Regression Model Results in the Aggregated Sample

The combined baseline results in Step 1, which consists of perceived prior mathematics classroom learning environment variables, indicated that mathematics teacher’s classroom management, mathematics teacher’s support, and cognitive activation in mathematics lessons explained 7.4% of the variance in student mathematics self-concept scores (R² = 0.074, p < 0.001, f² = 0.080), suggesting a small effect size (Cohen, 1988). When disciplinary climate and teacher support representing perceived current mathematics classroom learning environment variables were added in Step 2, the model explained 2.4% additional variance in self-concept scores (ΔR² = 0.024, p < 0.001, f² = 0.027), indicating a small effect size. When perceived school climate factors including sense of belonging to school and teacher student relations were entered in Step 3, the model explained an additional 1.3% of the variance in student mathematics self-concept scores (ΔR² = 0.013, p < 0.001, f² = 0.015), suggesting a small effect size (Cohen, 1988).

In the final model and considering relative (standardized) contributions, teacher support, a classroom-related predictor, was the strongest statistically significant contributor to mathematics self-concept (β = 0.121, p < 0.001), whereas sense of belonging to school, a school-related predictor, was the weakest contributor (β = 0.064, p < 0.01). Mathematics teacher’s support (p = 0.306) was the only non-significant predictor of mathematics self-concept in the combined sample.

Research question 2

Table 2 reports self-concept results disaggregated by language at home. In the baseline Step 1 model, mathematics teacher’s classroom management, mathematics teacher’s support and cognitive activation in mathematics lessons explained 7.7% of the variance in student mathematics self-concept scores for the English-at-home group with a small effect size (R² = 0.077, p < 0.001, f² = 0.083) and 5.2% of the variance for the other-language-at-home group (R² = 0.052, p < 0.001, f² = 0.054), suggesting a small effect size (Cohen, 1988).

Table 2 School and Classroom Climate Contributions to Mathematics Self-Concept: Hierarchical Regression Model Results Disaggregated by International Language at Home

In Step 2, when disciplinary climate and teacher support were entered, the model explained 2.4% additional variance in student mathematics self-concept scores (ΔR² = 0.024, p < 0.001, f² = 0.026) for the English-at-home group and 3.9% of additional variance for other-language-at-home group (ΔR² = 0.039, p < 0.001, f² = 0.043) suggesting small effect sizes for both groups (Cohen, 1988).

In Step 3, when sense of belonging to school and teacher student relations were added, 1.1% of additional variance were explained for the English-at-home group (ΔR² = 0.011, p < 0.001, f² = 0.012) and 2.6% of additional variance for the other-language-at-home group (ΔR² = 0.026, p < 0.001, f² = 0.029) indicating small effect sizes for both groups (Cohen, 1988).

In the final model, teacher support was the strongest significant contributor to mathematics self-concept for the English-at-home group (β = 0.130, p < 0.001); the relative contributions of the remaining school and classroom variables were about the same (β range: 0.065–0.092), except for mathematics teacher’s support which, similar to the combined sample, was the only non-statistically significant predictor of self-concept for this group (p = 0.270). For the other-language-at-home group, only disciplinary climate, β = 0.183, p < 0.001, CI 95% [0.085, 0.281], and teacher student relations, β = 0.142, p < 0.01, CI 95% [0.022, 0.262], significantly predicted student mathematics self-concept. Statistically, these contributions were the same for the English-at-home group as indicated by the overlapping confidence intervals for disciplinary climate, β = 0.075, p < 0.001, CI 95% [0.026, 0.124], and teacher student relations, β = 0.067, p < 0.01, CI 95% [0.008, 0.126], effects.

Research question 3

Table 3 provides a summary of the results of the perceived school and classroom climate contributions to mathematics achievement in three steps. The baseline results in Step 1 indicate that mathematics teacher’s classroom management, mathematics teacher’s support and cognitive activation in mathematics lessons explained 4.9% of the variance in student mathematics achievement (R² = 0.049, p < 0.001, f² = 0.029), suggesting a small effect size (Cohen, 1988). When disciplinary climate and teacher support were entered in Step 2, the model explained an additional 4.5% of the variance in mathematics achievement (ΔR² = 0.045, p < 0.001, f² = 0.050) with a small effect size (Cohen, 1988). When sense of belonging to school and teacher student relations were added to the model in Step 3, the model explained an additional 0.8% of the variance in mathematics achievement (ΔR² = 0.008, p < 0.001, f² = 0.009), indicating a small effect size (Cohen, 1988).

Table 3 School and Classroom Climate Contributions to Student Mathematics Achievement: Hierarchical Regression Model Results in the Aggregated Sample

Except for cognitive activation in mathematics lessons, prior mathematics classroom experiences variables remained statistically significant across models. Among overall mathematics classroom experiences variables, only disciplinary climate was a statistically significant predictor in Models 2 and 3. Similarly, among school-related climate variables, only teacher student relations was a statistically significant predictor in Model 3. In the final model and considering relative contributions, disciplinary climate, and mathematics teacher’s classroom management, both classroom-related variables, were the strongest significant contributors to mathematics achievement (β = 0.230 and β = 0.117, respectively, p < 0.001) immediately followed by teacher student relations, a school-related predictor (β = 0.103, p < 0.001). Cognitive activation in mathematics lessons, teacher support, and sense of belonging to school were not statistically significant contributors to mathematics achievement in the final, combined sample model.

Research question 4¹

Table 4 summarizes mathematics achievement results desegregated by language at home. In the baseline model, mathematics teacher’s classroom management, mathematics teacher’s support and cognitive activation in mathematics lessons explained 4.4% of the variance in student mathematics achievement for the English-at-home group (R² = 0.044, p < 0.001, f² = 0.046) and 9.5% of the variance for the other-language-at-home group (R² = 0.095, p < 0.01, f² = 0.105), suggesting small effect sizes for both groups (Cohen, 1988). The Step 2 including disciplinary climate and teacher support explained an additional 5% of the variance in student mathematics achievement for the English-at-home group (ΔR² = 0.050, p < 0.001, f² = 0.055) and 3.7% for other-language-at-home group (ΔR² = 0.037, p < 0.001, f² = 0.043) with small effect sizes for both groups (Cohen, 1988). School-related predictors explained 0.6% additional variance for the English-at-home group (ΔR² = 0.006, p < 0.001, f² = 0.009) and 1.1% for other-language-at-home group (ΔR² = 0.011, p < 0.001, f² = 0.013), indicating small effect sizes for both groups (Cohen, 1988).

Table 4 School and Classroom Climate Contributions to Student Mathematics Achievement: Hierarchical Regression Model Results Disaggregated by International Language at Home

In the final model and unlike for the combined sample, teacher support and cognitive activation in mathematics lessons were the only statistically nonsignificant predictors of mathematics achievement for the English-at-home group. The relative contributions of other predictors closely mirrored those of the combined model: Disciplinary climate and mathematics teacher’s classroom management, both classroom-related variables, were the strongest significant contributors to mathematics achievement (β = 0.232 and β = 0.105, respectively, p < 0.001) immediately followed by teacher student relations, a school-related predictor (β = 0.110, p < 0.001).

Similar to the English-at-home group, disciplinary climate and mathematics teacher’s classroom management were the closely strongest contributors to mathematics achievement for the other-language-at-home group (respectively, β = 0.228, CI 95% [0.157, 0.299] and β = 0.174, CI 95% [0.047, 0.301], p < 0.001). Statistically, these contributions were the same for the English-at-home group as indicated by the overlapping confidence intervals for disciplinary climate, β = 0.232, p < 0.001, CI 95% [0.177, 0.287], and mathematics teacher’s classroom management, β = 0.105, p < 0.001, CI 95% [0.046, 0.164], effects. Unlike for the English-at-home group, teacher support was a significant and negative predictor of achievement for the other-language-at-home group (β = − 0.108, p < 0.05) and neither of the two school-related predictors were statistically significant.

Discussion

Ample evidence suggests that student socioemotional, behavioral, and academic adjustments are influenced by school-related contextual variables such as school climate (e.g., Arens et al., 2017; Thapa et al., 2013). In recent decades, teacher-related contextual variables also received heightened attention fueled by teacher quality and merit-based compensation and promotion for teachers (see Riconscente, 2014). At the same time, as evidence suggests that teacher and school quality are particularly critical when students are at risk (Ardasheva et al., 2012; Riconscente, 2014), research on factors affecting LM students’ learning experiences, especially in STEM, is still only emergent (de Araujo et al., 2018; Sandilos et al., 2020). This is especially the case when it comes to simultaneously examining influences of both teacher- and school-related contextual variables. Contributing to filling this gap, our study investigated relative contributions of perceived classroom environment and school climate variables to mathematics self-concept and achievement of English-at-home and LM students using 2012 PISA USA data.

Although there have been studies that examined the relationship between mathematics achievement and mathematics self-concept (see a meta-analysis by Luo et al., 2014; Möller et al., 2020; Sewasew et al., 2018), there have also been studies examining those two variables separately (e.g., Lindberg et al., 2013; Luo et al., 2014; Sewasew et al., 2018; Trautwein et al., 2006). We chose to follow the second line of investigation because in the present study we were interested in exploring contributors to mathematics achievement and mathematics self-concept for LM vs. English-speaking majority students, rather than in the relationship between mathematics achievement and self-concept.

Implications for theory and research

From a theoretical perspective, this study contributes to a limited body of work focusing on understanding factors that may shape LM students’ mathematical self-concept and achievement.

With respect to students’ mathematics self-concept, the final model for English-at-home students closely mirrors the final model for the combined student sample: the same six factors are significant, positive predictors of mathematics self-concept for both the combined and the English-at-home sample. In contrast, for LM students, only two of these factors—perceived disciplinary climate and teacher-student relationships, a classroom- and a school-related factor, respectively—appeared important for mathematics self-concept. In other words, congruent with other research (Maltese & Tai, 2011), the mathematics self-concept of all students is supported by safe, organized mathematics classroom environments and a school climate in which students perceive teachers to like students and treat them fairly.

Similar to self-concept results, the final models for the aggregated and disaggregated samples indicate that disciplinary climate is an important predictor of mathematics achievement. In addition, in all three mathematics achievement final models, the perceived classroom climate in last mathematics classroom was a significant predictor of student achievement. This factor was a weaker predictor for English-at-home students (ß = 0.105) as compared to for LM students (ß = 0.174). One possible explanation for this difference in strength may be that LM populations can be more vulnerable. In particular, the ways in which teachers may stereotype and treat LM students in their classrooms may set both a precedent for other students and a tone that can negatively impact LM students' learning (Appel et al., 2015).

There are some surprising negative predictors across mathematics achievement models. For LM students in particular, perceived teacher support was a negative predictor of achievement, suggesting that when students perceived their teachers’ as offering more support, the students had lower mathematics scores. It could be that although LM students are receiving some support, the type of support may not have been targeted to meet their learning needs (de Araujo et al., 2018). Indeed, this would align with research suggesting that teachers are generally underprepared to meet the needs of their diverse learners, particularly LM students (e.g., Okhremtchouk & Sellu, 2019). Because, similar to English-at-home students, LM students are not a heterogeneous group, this finding needs further investigation. In particular, large-scale assessments such as PISA need to make efforts to cast a larger net in sampling LM students and collecting more variables regarding their home and school language proficiency to allow for more nuanced analyses by proficiency subgroups (Sandilos et al., 2020). For English-at-home students, the negative contribution of sense of belonging to school to mathematics achievement was also unexpected. Possible explanations of this finding may be the conflation of student perceptions of school belonging with that of positive relationships with peers, who may sometimes negatively influence academic motivation (Goodenow & Grady, 1993) or a possibility of a mediated relationship among students’ perceptions of the school climate, their sense of school belonging, and their achievement (see Maxwell et al., 2017). Future studies exploring these relationships may also consider the extent to which schools emphasize academics to more fully understand the role school belonging may play in achievement of both English-at-home and LM students.

Consistently, perceived classroom disciplinary climate, a classroom-related variable, was an important predictor across outcomes and student populations and it was most impactful for LM students. This classroom-related variable was captured by items gauging high-to-low frequency (from ‘every lesson’ to ‘never or hardly ever’) of such problematic classroom events as noise/disorder, students’ inability to focus on work, loss of instructional time. Our finding linking classroom environments perceived as safer and more organized with higher mathematics self-concept and achievement across student populations is consistent with Maslow’s (1943) theory. The theory holds that the feeling of physical, emotional, mental, and intellectual safety—with physical and emotional safety, arguably, being foundational for other safety types—is a basic human need that must be met in order for individuals to flourish. In a classroom, students need to feel that they are safe, and this safety is typically established through supportive norms, rules, relationships, and structures. In environments lacking these supportive structures, students are more likely to experience victimization, violence, and disciplinary punishments, often associated with higher levels of absenteeism and lower academic achievement (Astor et al., 2009; Thapa et al., 2013).

Perceived relationships with other school teachers, a school-related variable, was also an important contributor to mathematics self-concept of both English-at-home and LM students; however, its contribution to mathematics achievement was statistically significant only for English-at-home students. This finding, as well as the finding that perceived teacher support (a classroom-related variable) were positive contributors to English-at-home students’ mathematics self-concept and achievement, is consistent with empirical evidence suggesting that fostering positive student–teacher (or staff) relationships is associated with students’ socioemotional, behavioral, and academic adjustment and with higher academic engagement and achievement (Anderson et al., 2004; Murray & Malmgren, 2005). Indeed, students’ feelings of greater connections have been found to be associated with a host of positive learning outcomes, including higher perceptions of ability and better grades (e.g., Jia et al., 2009; Thapa et al., 2013).

However, research also has shown that the specific aspects of relationships students perceive as more impactful may differ for learners of different backgrounds. For example, Schneider and Duran (2010) found that middle school Hispanic/Latino LM students “considered personal relationships with teachers as more important than modeling of positive behaviors—contrasting with the preference of White and Asian students” (Thapa et al., 2013, p. 364). Notably, differences in student backgrounds are also known to impact teacher perspectives of student–teacher relationships. In a meta-analysis on this topic, Nurmi (2012), for example, found that teachers reported better relationships with their students exhibiting higher levels of engagement and motivation. In turn, research suggests that LM students may be perceived as less motivated and less capable (Datnow et al., 2005; Gage, 2017) due to their not sharing the same values as their teachers. Datnow and colleagues (2003, 2005) concluded that cultural stereotyping was a major hindrance in improving academic achievement for LM students as teachers in their educational-reform-focused work perceived LM students as lacking abilities and skills, linking such perceptions to students’ cultural backgrounds rather than to effort. This body of work may provide an alternative explanation as to why teacher support may be a negative predictor of LM students’ mathematics achievement, as, teachers showing “interest in every student’s learning” or continuing teaching “until the students understand” (example teacher support items) may come from a different set of assumptions, expectations, and instructional approaches for LM students, which may interfere with LM students’ academic persistence and performance (Zurawsky & Gordon, 2004).

Small amounts of variance explained across models, suggests that future studies need to explore additional classroom and school climate characteristics not explored in the present study (Cornelius-White, 2007; NSCC, 2007), while also considering other-than-language-at-home individual differences (e.g., Arens et al., 2017; Lauermann et al., 2020).

Implications for policy and practice

The results of this study suggested that, for all students and LM students in particular, classroom climate is impactful on both mathematics self-concept and mathematics achievement. As mentioned earlier in the discussion, a prerequisite for student learning is a physically and intellectually safe environment. This strongly suggests that schools committed to their students need to consider ways to improve and support teachers in creating safe classrooms and supportive classroom management. Various interventions such as professional development or peer-teacher mentorships could be utilized to help teachers develop better classroom management. Additionally, schools can give consideration to how veteran versus novice teachers are assigned to classes since veterans tend to have better classroom management.

Though according to our findings LM students will also benefit from better classroom management, there are still unique supports they need from their classroom and schools. These needs have implications again for the type of support and professional development mathematics teachers receive to meet the specific needs of these students. For example, research has shown that curricula that include multimodal learning opportunities better support LM students in learning mathematics (de Araujo et al., 2018). Teachers could be supported to either use this type of curricula or create tasks that provide space for multimodal communication. This type of instruction is even more impactful when it draws on students’ funds of knowledge by using culturally relevant and meaningful contexts and tasks (de Araujo et al., 2018). For culturally relevant pedagogy to work, however, it must be embraced “as a guiding ethos” (Aronson & Laughter, 2016, p. 198) both within the classroom and the school, centering LM student affirmation as a commitment that is individual and school-wide (Khalifa et al., 2016).

In their meta-analysis of studies focused on policies and practices supporting LM students in schools, de Araujo et al. (2018) argued that their findings brought to light the importance of building intellectual spaces for LM students. This is because participation in rich classroom communication about mathematics is not only possible for LM students, “but also vitally important for their development of mathematical understandings” (p. 907). For that aim, Araujo et al. argued, educators need to challenge their beliefs about LM students often rooted in deficit notions of speakers of other languages to trump ideologies inhibiting implementation of most optimal pedagogies for LM students. A potential venue to addressing this issue suggested by the meta-analysis is for educators and policy makers to be aware of harmful effects of social constraints (e.g., low expectations, power dynamics) and structural restraints (e.g., course placements and tracking/lack of access to gifted education; Barajas-López, 2014; Lauermann et al., 2020) undermining schooling experiences and learning of LM students to systematically identify and eliminate the causes of such inequities. An important component of this is for policy makers, educators, school personnel, and students to “work against the grain when they attempt to create democratic, long-term, and trusting relationships that challenge the institutionalized norm of alienation” (Barajas-López, 2014, p. 17).

Limitations

All studies have limitations, and this research is no different. First, there are likely a number of other variables (e.g., relationships with peers) that would increase the amount of variance predicted, and we encourage future studies to introduce other variables to the model. For example, as mentioned in Footnote 1 and as suggested by anonymous reviewers, we introduced such student-level control variables as PISA’s ESCS (a proxy for SES) as well as dummy variables for race/ethnicity and immigration status. The results aligned with Li et al. (2020), indicating that while SES variables do predict achievement, the connection between SES and self-concept, in particular, is more opaque.

Additionally, we want to acknowledge that while these variables do contribute to the overall predictive variance of the model, these measures may be interpreted as causing the reduced achievement, rather than reflecting a larger societal change. This relationship has been long substantiated (e.g., Korous et al., 2022; Liu et al., 2022) and has changed little across the nearly 50 years since the publication of the Coleman report (Coleman, 1966) nor across over 20 years of PISA data collection. However, there are a number of issues with SES, both as a construct (Andreoni et al., 2021) and as a predictor of outcomes, especially when it comes to comparing its impacts within individual students vs. across schools. The latter, in particular, seems to be more impactful than the former (that is, lower SES students who attend higher SES schools, on average, outperform higher SES students who attend lower SES schools, Fischer et al., 2016; Thomson, 2018).

However, our goal for the present investigation was not to evaluate the relative contributions of select predictors for both LM and non-LM students, but rather, to highlight that the predictors themselves may serve differential functions across groups. We believe this is an important distinction as while SES may predict achievement, teachers and administrators can often do little to influence individual or community SES. However, if our analysis reveals that inclusive classroom practices and safety (e.g., positive teacher-student relations) are stronger predictors for LM students, teachers and practitioners can choose to adopt and modify pedagogy to be more inclusive for all learners. Also, these results may highlight that while teachers are engaged in practices that work for most students (e.g., teacher support or sense of belonging to school), these practices may not be significant predictors of achievement for all students.

The second limitation is that the present study focused only on participant’s languages spoken at home, without considering the immigration status. However, since immigrants with the same backgrounds are culturally heterogeneous (e.g., Ladison-Billings, 2014; Niedenthal et al., 2018), future studies are encouraged to add the immigration status of the participants to the model. This is important as even though the primary language at home may not be English even after several generations in a country, the phenomenon of cultural syncretism may occur, plausibly resulting in different predictors of achievement and self-concept. Of interest to note to future studies considering immigration, most native English speakers who participated in PISA 2012 lived in the U.S. for at least two generations; most participating LM students, in turn, were first or second generations, suggesting that either students themselves, or their parents, or both were foreign-born (OECD, 2013).

Finally, the present study used large scale, self-reported data which prevent drawing causative conclusions. While the large-scale PISA data has advantages, future studies should also employ experimental or multi-wave designs which would allow for the inclusion of additional control variables or researcher-created scales (e.g., the use of instructional practices recommended for LM students in STEM education, such as translanguaging; Suárez, 2020), which may explain some of the variance in the achievement gap between LM and non-LM students. Of course, this is not an intervention study, so our goal here is simply to illustrate whether the relationships of interest are present, and if they are, future studies may create interventions to further expand on and potentially determine causal relationships. For a similar argument (although a dissimilar context), we look to Lombardi et al. (2016). In this experimental study, the authors presented pre-service teachers with either a refutation or expository text to determine how the intervention would influence their thinking processes. The authors found (among other things) that the intervention was effective in promoting learners to rely more on their critical thinking abilities, whereas those without the intervention relied more heavily on their prior knowledge.

Appendix

See Tables 5, 6 and 7

Table 5 Pearson Correlations

Table 6 Descriptive Statistics

Table 7 The List of Items which were used to create the variable of interest by PISA (OECD, 2014)