Does teachers’ motivation have an impact on students’ scientific literacy and motivation? An empirical study in Colombia with data from PISA 2015

Abstract

In this study we use data from the Programme for International Student Assessment (PISA) to investigate the effect of teachers’ motivation on students’ scientific literacy and motivation in Colombia. These relationships are explored using a multilevel modeling framework and through the lens of Self-Determination Theory. Although difficulties in achieving education quality in developing and emerging economies are commonly attributed to teacher motivation issues, and important policy measures are implemented based on this assumption, this topic remains largely empirically unexplored. The purpose of the study is to contribute to fill this gap and provide empirically based insights for a broader and more informed dialogue regarding the effect of motivation in the development of scientific literacy, and to the design and implementation of evidence-based policies, instructional practices, and interventions. In this analysis, we did not find a significant relationship between teacher motivation and either students’ scientific literacy or motivation. However, students’ interest in science and sense of self-efficacy were significantly associated with their own achievement. The results also show that teacher-directed instruction is the strongest predictor of scientific literacy as opposed to inquiry-based teaching. However, inquiry-based teaching was found to be a positive predictor of increased students’ motivation.

Introduction

In 1994, the Colombian government launched an ambitious educational reform with the goal of improving educational quality and access throughout the country. The reform was particularly concerned with improving the quality of science education, which is viewed as one of the sectors capable of generating economic growth. As a result, in 2006, national science standards were introduced, the curriculum shifted from content-based prescribed learning to competency-based learning, and teacher education programs underwent significant reforms (UNESCO, 2013).

This reform was critical in significantly modernizing the country’s educational infrastructure, improving teacher working conditions, and increasing primary school enrollment from 69 percent at the end of the 1990s to 98 percent in 2020 (World Bank, 2021). Today, Colombia devotes a higher share of the GDP than the OECD average at non-tertiary levels (OECD, 2020). Despite these efforts, students’ improvement in scientific literacy has been slow. In PISA 2018, Colombia was ranked 62 of 78 countries with a score of 413 while the average of the OECD score was 489. This is lower than the results of PISA 2015, and a modest increase since Colombia first participated in PISA in 2006.

The slow rate of progress in scientific literacy led to policy makers and researchers shifting their attention to teachers, their preparation, and their motivation to work (García et al., 2014; Ministry of Education, 2013, 2021). This is in line with research suggesting that teachers are among the most important factors in student learning (Metcalfe & Game, 2006; Valerio, 2012), and their motivation plays a critical role in how students engage and embrace their educational experience (Bong & Skaalvik, 2003; Sheldrake et al., 2017). Numerous studies suggest that motivated teachers are essential in creating classroom environments that foster enhanced learning, well-being, and motivation (Fong et al., 2018; Govorova et al., 2020; Lam et al., 2009; Marshik et al., 2016).

In fact, teacher and student motivation can have relevant economic and social implications, particularly in emerging and developing countries where it is estimated that about 25% of the budget allocated to basic education is lost because of teacher absenteeism as a result of lack of interest and motivation (Lee et al., 2015; Msosa, 2020; Transparency International, 2013). Also, it has been observed that a considerable percentage of students who drop out of school say that they do so because of boredom and loss of interest (Ali et al., 2021; Puicon et al., 2022; Sanchez Zinny, 2013; Saturnino, 2021).

Although the role of teacher motivation in shaping learning outcomes is widely addressed in the literature, the myriad of definitions and conceptual approaches used in its study (Dörnyei & Ushioda, 2011; Guerriero, 2015), the preponderance of anecdotal reports and exploratory theoretical investigations (Brookhart, 2012; Wen-ying & Xi, 2016), and the scarcity of empirical research (Alamer, 2021; Theodotou, 2014; Zhang et al., 2022) make it difficult for the practitioner and policy maker to identify which motivation-based programs or interventions are most effective in increasing student achievement. The main reasons for the dearth of empirical research range from the ethical and administrative challenges faced by academics in obtaining representative samples of students and in-service teachers (Mercer, 2018) to the political concerns that can arise when attempting to establish a causal relationship between teachers and learning outcomes (Dörnyei, 2018).

It is in the light of these challenges that large-scale cross-national studies in education such as the Programme for International Student Assessment (PISA), can help close knowledge gaps left by current research and provide insight for policymakers, researchers, and practitioners into how educational practices, policies, and socioeconomic and cultural factors can affect students’ learning and socioemotional well-being (Kirsch & Braun, 2020), given the vast array of student, teacher, and school variables available for analysis. Apart from assessments of reading comprehension, math, and scientific literacy (OECD, 2016), PISA also collects data on teachers’ and students’ personal characteristics, as well as a variety of school-levels factors associated with student learning. Therefore, in this study we take advantage of the wealth of data available from the PISA study to empirically investigate how teachers’ motivation affect students’ outcomes (Hoy et al., 2006; Palmer, 2007; Treviño, 2010). The main question that guides this study is: what is the effect of teachers’ motivation on students’ scientific literacy and motivation after controlling for individual and school-level variables?

We seek to answer this question using a multilevel modeling approach to statistical analysis (Raudenbush & Bryk, 2002) and through the lens of Self-Determination Theory (Ryan & Deci, 2000). The purpose of the study is to contribute to a broader and more informed dialogue regarding the effect of motivation in the development of scientific literacy, and to the design and implementation of evidence-based policies, instructional practices, and interventions.

Review of the literature

Teachers’ perceptions of their work and their emotional abilities are commonly identified in the literature as important variables affecting their own well-being, the teaching process, and student outcomes (Keller et al., 2016; Mostafa & Pál, 2018; Ruiz-Alfonso & León, 2016; Singh & Ryhal, 2021; Slemp et al., 2020). Teachers’ motivation in particular, defined as engaging in an activity for its own sake because it is inherently interesting or enjoyable (Laschke & Blömeke, 2016; Ryan & Deci, 2018), is regarded as critical in terms of how teachers may assist in developing their students’ learning and skills (Bong & Skaalvik, 2003; Camargo-Mayorga, 2017; Franco et al., 2018; Sheldrake et al., 2017).

This issue is especially relevant in emerging and developing economies, where teacher demotivation is widely regarded as a major impediment to improving educational quality (Khanal et al., 2021). A demotivated teacher is more likely to be absent from work or to leave the teaching profession entirely, limiting students’ learning time and having major effects for pupils from minority or low socioeconomic backgrounds in particular (Abadzi, 2007; Bugg Conradson, 2021). This problem has prompted some countries to employ financial incentives to attract and retain high-quality teachers in the classroom and to motivate them to increase student learning (Duflo et al., 2012; Muralidharan, 2012; UNESCO-IICBA, 2017). However, evidence indicates that using monetary incentives to improve teacher performance is unsustainable in the long run and that eliminating them once they have been implemented can have a detrimental effect on student learning and attainment (Jinnai, 2016; Visaria et al., 2016).

Lack of motivation has also been reported as a reason for students’ absenteeism and ineffective learning in science and other STEM areas (De Loof et al., 2019). According to some estimates, approximately 40% of students who drop out of school, do so out of boredom (Sanchez Zinny, 2013; Saturnino, 2021) or because they do not feel that what they are studying is useful; and although in developing and emerging economies school dropout is more prevalent among children of lower socioeconomic groups and minorities, dropping out due to lack of commitment and motivation is prevalent among all socioeconomic groups (Sanchez Zinny, 2013).

Self- determination theory and intrinsic motivation

Research on the importance of teacher motivation and other affective factors has increased significantly over the last 15 years (Neves de Jesus & Lens, 2005), and has been conceptualized from a variety of theoretical lenses, including self-efficacy (Bandura, 1985), expectancy-value (Eccles, 1983), achievement goals (Elliot, 2005), and Self-Determination Theory or SDT (Deci & Ryan, 1985; Ryan & Deci, 2000; Slemp et al., 2020). Among these theories, SDT stands out as one of the most prominent theoretical frameworks for understanding how motivation operates in educational settings, its primary psychological drivers, and the contextual factors that promote or inhibit it (Deci & Ryan, 1985; Ryan & Deci, 2000; Slemp et al., 2020). At the core of this theory is the thesis that teacher-related characteristics, such as teaching style and teacher motivation, can influence student motivation and learning (De Loof et al., 2019).

SDT conceptualizes motivation along a continuum ranging from intrinsic motivation to amotivation (Fig. 1) and proposes three universal, innate psychological needs that motivate an individual to undertake a certain behavior (Ryan & Deci, 2002): competence (sense of efficacy and opportunity to exercise and express one’s capabilities), autonomy (self-direction and personal support in initiating and regulating one’s behavior), and relatedness (establishing ties with the community with a sense of care and respect). In accordance with the SDT, the satisfaction of these needs would result in greater behavioral autonomy. Moreover, psychological needs serve as mediators of the effects of social context on autonomous regulation levels (Vallerand, 1997).

Fig. 1

Taken from Center for Self- Determination Theory (2018)

Representation of the Motivation Continuum.

Intrinsic motivation is regarded as the most self-determined type of behavior, the highest motivational and performance quality, and is defined as “the inherent propensity to engage one’s interests and exercise one’s abilities and, in doing so, to seek and master optimal challenges” (Deci & Ryan, 1985, p. 45).

In the middle of this continuum lies extrinsic motivation, which is described as the performance of an activity for instrumental reasons and includes lower levels of self- determination: integration (performing an activity because it is congruent with other aspects of lifestyle), identification (performing an activity because you accept its value), introjection (performing an activity for ego and seeking approval from others), and external regulation (performing an activity to receive an incentive or avoid punishment) (Gagné et al., 2010; Ryan & Deci, 2002).

Teacher intrinsic motivation and student outcomes

There is widespread agreement in the literature that intrinsically motivated teachers can significantly contribute to improving students’ learning, self-esteem, autonomy and motivation for optimal achievement in the classroom (Franco et al., 2018; Govorova et al., 2020; Marshik et al., 2016; Ryan & Deci, 2000; Slemp et al., 2020; Valerio, 2012). It is argued that teachers who are intrinsically motivated are more likely to find their work enjoyable and satisfying and perform better because they do not need external rewards to perform their activities (Ryan & Deci, 2000, 2018; Ryan et al., 1984).

In a study of secondary school instructors and students in Hong Kong, Lam et al. (2009) found that teachers’ intrinsic motivation is positively associated with student intrinsic motivation both directly and indirectly through teaching support: when teachers reported to be motivated during a learning activity, their students tended to report higher levels of intrinsic motivation as well, which the authors attribute to imitation or modeling. Additionally, when students received stronger instructional support from their teachers, they reported greater intrinsic motivation. The study of Lam et al. (2009) suggests that instructional strategies may act as a buffer between teachers’ intrinsic motivation and students’ intrinsic motivation, and it is in line with previous research (Gagné et al., 2010; Mostafa & Pál, 2018; Sheldrake et al., 2017) that shows that teachers’ intrinsic motivation is transmitted through its attitudinal and behavioral manifestations that influence the pace of classes and the learning relationship they develop with their students. Likewise, intrinsic motivation appears to be an important predictor for student learning.

Analyses of PISA data (OECD, 2013) indicated that students with a higher level of intrinsic motivation (Ryan & Deci, 2000) are more likely to perform better in science and have higher expectations for future careers in science than students driven primarily by instrumental motivation, or external rewards (e.g., future studies, desired careers, or well-paying jobs) students expect to receive for mastering the content (Eccles & Wigfield, 1995). A comparative study between Greece, Iceland, and Luxembourg showed similar findings: using PISA 2015 data, Karakolidis et al. (2019) found that, despite efforts to boost students’ instrumental motivation in the classroom their performance lagged behind the average of the other OECD countries.

However, students’ intrinsic motivation, as evidenced by increased enjoyment and interest in science studies, did predict improved test performance. These findings lend support to the hypothesis that intrinsic motivation can have a positive impact on student learning and achievement (Kim, 2020; Laschke & Blömeke, 2016; Teppo et al., 2021; Wei et al., 2019).

Motivation in PISA

In the PISA study, socioemotional factors related to motivation have gained relevance for both teachers and students (Guerriero, 2015; Thien et al., 2015). The inclusion of these variables offers the opportunity to analyze the relationship between teacher motivation and students’ learning outcomes. In the case of students, PISA 2015 used the SDT to develop and operationalize a set of measures related to intrinsic motivation (Ryan & Deci, 2000), including enjoyment of science, interest in broad science topics, and achievement motivation (OECD, 2016). While the 2015 PISA study did not explicitly measure teacher motivation, in this study this construct was measured through a proxy composite variable indicating job satisfaction (satisfaction with their current job environment and satisfaction with the teaching profession), which is a common heuristics used as a proxy for intrinsic motivation in several studies (Corduneanu, 2019; Hayati & Caniago, 2012; Paais & Pattiruhu, 2020). Although job satisfaction and intrinsic motivation are distinct constructs, the connection between them, rooted in the fulfillment of essential psychological needs, implies a symbiotic relationship which provides a strong basis for considering job satisfaction as a proxy of intrinsic motivation in the teaching profession. SDT posits that individuals are driven by three innate psychological needs: autonomy, competence, and relatedness (Ryan & Deci, 2000). When teachers experience job satisfaction, it often signifies that their basic psychological needs are being met within the work environment (Broeck et al., 2010; Caprara et al., 2003; Skaalvik & Skaalvik, 2011; Song et al., 2021). Autonomy refers to a teacher’s sense of control over their instructional methods, classroom management, and decision-making processes. Competence refers to teachers who feel competent and effective in their roles. Relatedness emphasizes the importance of social connections and a sense of belonging. Job satisfaction can be influenced by positive relationships with colleagues, administrators, and students.

While job satisfaction is not a direct measure of intrinsic motivation, it is closely intertwined with the underlying psychological needs outlined in SDT. Therefore, job satisfaction can be seen as indicative of intrinsic motivation in the teaching profession.

Methods

Data

Students’, teachers’, and principals’ responses from the 2015 cycle of the Programme for International Student Assessment (PISA) are the primary data sources for this study. These data consist of teacher and student characteristics, school characteristics, assessments of students’ scientific literacy, and teacher and student motivation measurements. In Colombia, the sample includes 8095 students and 1314 science teachers from 313 schools. The information was collected from the program’s website (OECD, 2015).

Measures

Outcome measures

Scientific literacy A measure of students’ ability to use and apply scientific knowledge to identify questions, acquire new knowledge, explain natural phenomena, and draw evidence- based conclusions (OECD, 2016). This scale has a mean of 500 points and a standard deviation of 100. For each student in PISA 2015, ten plausible values were calculated. All ten plausible values were included in the analysis. In Colombia, the mean score across the 10 plausible values was 415 (SD = 80).

Students’ intrinsic motivation A composite index with a mean value of zero and standard deviation of 1.0, with higher values indicating greater motivation (OECD, 2017b). The index combines questions on students’ enjoyment of and interest in learning science (ST094) and their motivation for academic achievement (ST119) (Appendix A). The results of the reliability assessment indicated a two-factor structure of the index with a good internal consistency (Cronbach α = 0.853, 95% CI 0.852–0.854).

Predictors

Level 1 predictors included the student’s grade at the time of taking the PISA test, as compared to the country’s modal grade, educational resources available at home, availability of information and communication technology (ICT) resources, an index of the student’s socioeconomic and cultural status (ESCS), students’ interest in science topics, students’ expected occupational status, grade repetition, students’ self-efficacy in science, and students’ gender.

Teacher, classroom, and school variables were used as level 2 predictors in the model, including science teaching approaches (inquiry-based teaching and teacher-directed instruction), teacher support in science classes, the type of school (public or private), total student enrollment, average class size, student and teacher behaviors that affect school climate, and the school’s socioeconomic status (School ESCS). A composite index of teacher motivation was obtained by merging measures of teachers’ satisfaction with the teaching profession and teachers’ satisfaction with their current work environment. The resulting index has a mean of 0 and a standard deviation of 1 and was derived following the same approach explained above. The results of the reliability assessment indicated a two-factor structure of the index with a good internal consistency (Cronbach α = 0.85, 95% CI 0.847–0.851). A detailed description of the variables is provided in Appendix A. For an overview of the descriptive statistics of the variables used in this study, including measures of central tendency, dispersion, and distribution, please refer to Appendix B.

In accordance with methodological standards for the implementation of multilevel models for the analysis of two-stage sampling data (Foy, 2018; OECD, 2009; Rutkowski et al., 2010), the teacher data were aggregated at the school level and imported into the student data file (OECD, 2009), thereby adding a value for each aggregated variable to each student record.

Data analysis

Hierarchical Linear Modeling (HLM) (Raudenbush & Bryk, 2002) was used as the primary analysis technique in this study. Students’ science literacy and intrinsic motivation scores were regressed on students’ individual characteristics, teacher characteristics, and school characteristics. HLM is a variation of the ordinary least squares regression technique that is used to investigate relationships between variables when there is a nested structure within the dataset. In such cases, where data points are not independent but rather grouped or nested within higher-level units (e.g., students within classrooms, employees within departments), HLM allows for the examination of both within-group and between-group variations, providing a more comprehensive understanding of how individual-level and group-level factors interact and influence the outcome of interest. This is the case with PISA data, which involves students and teachers nested within schools (OECD, 2009).

Unlike simple linear regression models, the HLM calculates the effect that can be derived from how the units of analysis are chosen for the study, as well as their group and individual characteristics. In Colombia, students’ socioeconomic characteristics largely determine the type of school they attend, so there may be little variation in students’ socioeconomic backgrounds within each school. In other countries, schools may house students from various socioeconomic backgrounds; however, within the school, the student’s socioeconomic background may determine the type of classes to which they are assigned, causing variance within the school to be affected. A linear regression model that does not account for the hierarchical structure of the data would be useless for determining differences between these two systems (OECD, 2009).

In this study two-level hierarchical models were used to evaluate the association of both students’ scientific literacy and intrinsic motivation (Y_ij) and a linear combination of K student demographic characteristics and W teacher and school characteristics. The general from of the student-level model used in the study was:

$${Y}_{ij}={\beta }_{0j}+\sum_{k=1}^{K}{\beta }_{kj}{\left(Studentcharacteristics\right)}_{kij}+{\epsilon }_{ij;}$$

This model states that the expected students’ scientific literacy and intrinsic motivation scores are composed of a unique intercept β_0j and a regression coefficient for each student characteristic β_kj, as well as a random student effect ϵ_ij. The general form of the school-level was defined as:

$${\beta }_{0j}={\gamma }_{00}+{\sum }_{w1=1}^{W1}{\gamma }_{0w1}{\left(SchoolCharacteristics\right)}_{w1j}+{\sum }_{w2=1}^{W2}{\gamma }_{0w2}{\left(TeachingCharacteristics\right)}_{w2j}+{u}_{0j,}$$

and

$${\sum }_{k=1}^{K}{\beta }_{kj}={\gamma }_{k0}$$

$${\sum }_{l=1}^{L}{\beta }_{lj}={\gamma }_{l0},$$

where W₁ represents school demographic characteristics (curricular orientation, type of school, total student enrollment, average class size, school climate, school’s ESCS status), W₂ is teacher and teaching-related measures (teaching approach, teacher support, and teacher motivation), and ${\gamma }_{k0}$ and ${\gamma }_{l0}$ represent the average effects of factors at the group level, and they are derived from the individual-level coefficients. They help us understand how characteristics or factors at the higher levels (school and teaching practices) impact the overall group outcomes. With these models we obtained the regression coefficients, their significance, and the associated standard errors to answer the research question.

In accordance with the method detailed by Bloom et al. (Bloom et al., 2017), we implemented a modeling strategy that combines random coefficients and fixed intercepts. The rationale behind employing this approach is twofold:

Firstly, it addresses the concern that estimating a two-level model with random school intercepts could introduce bias if these random intercepts are correlated with treatment effects. This can occur when there is a relationship between unobserved school-level factors and the treatment being studied.

Secondly, it takes into account the potential pitfalls associated with the conventional strategy of specifying a fixed-effects model for schools. This conventional approach may yield biased estimates of the average treatment effect, incorrect standard errors, and potentially misleading interpretations.

Given the complex design of the PISA study, the analysis was carried out using the HLM 8 statistical software (Raudenbush et al., 2019). This tool enables the implementation of two-level models with random coefficients and fixed intercepts, and is particularly suited for the analysis of large-scale assessment (LSA) data, where plausible values and sampling weights should be used to obtain unbiased statistics (Mislevy, 1993; OECD, 2022; Rutkowski et al., 2010; Tat et al., 2019; Wu, 2005). As mentioned above, PISA 2015 reported students’ scientific literacy in the form of $k=10$ plausible values for each student. To account for the wide range of plausible values, HML 8 performs independent statistical analyses on each of these ten plausible values and aggregates the results to obtain correct final estimates of the statistics and their standard errors, ensuring accurate treatment of data (Raudenbush et al., 2019). Following Mang et al.’s (2021) recommendation for the analysis of LSA data, only the final school weights (W_SCHGRNRABWT) were applied to the level two of the model. This recommendation stems from their analyses, which show that using only the school weights provides the most unbiased estimates for hierarchical models (Mang et al., 2021).

Results and discussion

Is there a relationship between teachers’ motivation and students’ scientific literacy?

The results of the unconditional model revealed that 28.1% in students’ overall scientific literacy was attributable to differences between schools and 71.9% of the variance was attributable to individual differences within the school plus error.

As shown in Table 1, the analysis did not find a significant relationship between teachers’ motivation and students’ level of scientific literacy (β = − 4.37, p = 0.3), and, further, removing the teacher motivation variable resulted in no change in the model’s significant predictors. To further support this finding, we compared the level of motivation among teachers from a random sample of countries stratified by level of performance in scientific literacy in PISA 2015 (below average, average, above average). We found that teachers from above-average performing countries have lower levels of motivation than teachers from below-average performing countries such as Colombia and the Dominican Republic, which ranked 57th and 70th respectively in scientific literacy (Fig. 2).

Table 1 Summary of parameter estimation for a two-level hierarchical linear model of students’ scientific literacy

Fig. 2

Teachers’ motivation in sampled countries grouped by level of scientific literacy (N = 93,499)

This finding is in line with previous research (Mostafa & Pál, 2018) that has shown that, after controlling for students’ and schools’ socioeconomic characteristics, students in schools with more motivated teachers outperformed students in schools with less motivated teachers in only 8 of the 72 countries participating in PISA 2015.

Among the teacher controls, the teaching strategies used in the classroom appear to be more significant in explaining students’ scientific literacy, a pattern also consistent with previous research (Abrahams, 2009; Areepattamannil, 2012; authors blinded for review). The analysis shows that teacher-directed instruction (TDI) is associated with higher levels of scientific literacy (β = 56.35, p < 0.001), whereas effect of inquiry-based strategies (IBT) is not significant (β = − 16.67, p = 0.09).

These findings may be indicative of a tension between an idealistic view of science education and the purpose and content of the PISA test. Inquiry-based instruction is predicated on the premise that students acquire scientific knowledge indirectly through their own scientific experiments, rather than directly from teachers (Caswell & LaBrie, 2017; Salchegger et al., 2021). PISA on the other hand, evaluates students’ ability to “explain phenomena scientifically, evaluate and design scientific inquiry, and interpret data and evidence scientifically” (OECD, 2017b, p. 2). Some authors consider that, although it is convenient for students to experience exercises and practices based on inquiry (Riffert et al., 2021), science learning involves dealing with complex and abstract ideas that are not accessible to them through inquiry alone. Aspects such as scientific concepts and vocabulary, conventions, or use of equipment are best learned through directed instruction (Aditomo & Klieme, 2020; Harlen, 2013; Osborne et al., 2004). This study adds evidence to this argument by showing that teacher directed instruction is a strong predictor of scientific literacy.

As for other school-related variables, base model estimates revealed that the school’s socioeconomic status is an important predictor of scientific literacy, with a one-point increase in the ESCS index associated with a 21-point increase in students’ science scores (β = 21.18, p < 0.001). Higher levels of teacher support were also found to have a negative relationship with scientific literacy (β = − 23.65, p < 0.05). This result is comparable to that of an earlier study that looked at the effect of teacher support on students’ math achievement (You et al., 2021). In line with those authors, we argue that this finding does not necessarily prove that a reduction in scientific literacy is caused by more teacher support. Instead, it can be an indication that the students who perform worse academically are the ones who get more support from their teachers. In the literature, there is still debate about the impact of teacher support on student achievement (Klem & Connell, 2004; Rueger et al., 2010; Saroughi & Cheema, 2022). Further research is required to determine the precise mechanism by which teacher support contributes to students’ learning and academic achievement.

The analysis reveals that teacher-directed instruction (TDI) is the strongest predictor of scientific literacy among Colombian students, where 1-point increase in the TDI scale predicts a 56-point increase in scientific literacy scores. This finding pushes back against the consensus in the literature that students’ socioeconomic and personal characteristics are the strongest predictors of learning (Pieros & Rodrguez, 1998; Trevio, 2010) and that it is challenging to identify school-level variables with the same level of predictability (Hoy et al., 2006). However, the results of this study suggest that even after adjusting for students’ socioeconomic characteristics, changes in teaching practices might have a greater impact on student learning.

Regarding the influence of individual student factors, in a pattern similar to that of other countries in Latin America (Fig. 3), the students’ gender in Colombia is a strong predictor of science performance (β = -16.45, p < 0.05) with girls performing lower than boys. Although these large gender gaps in science are only present in 25 of the 72 countries participating in PISA 2015, the region accounts for about a third of those countries (Mostafa & Pál, 2018).

Fig. 3

Gender Differences in Scientific Literacy Between Boys and Girls in Latin America

This could be explained by the social and educational contexts in which boys and girls are raised and educated, which reflect societal expectations, biases, and institutional designs that perpetuate traditional gender roles and stereotypes about science learning and careers (Abadía & Bernal, 2017; Archer et al., 2013; Weisgram et al., 2011). As research demonstrates, parents and schools play a critical role in shaping young people’s perceptions of femininity/masculinity and gender roles, which ultimately shapes their attitudes toward science in school and steers girls toward “people- oriented” careers such as nursing, teaching, or social services, while men are more likely to gravitate toward “thing-oriented” careers such as science and engineering (Auyeung et al., 2012; Barrett, 2021; Breda & Napp, 2019; Frome et al., 2006; Hayford & Halliday Hardie, 2021).

The model also shows that students who had not repeated a grade were more likely to have a higher level of scientific literacy (β = 16.68, p < 0.05) compared with students who did repeat a grade. Colombia uses a vertical stratification system in which students are held back an additional year in the same grade if they do not meet certain criteria or miss a specified number of school days. As a result, all students are 15 years old when they sit for the PISA test, but not all of them are in the modal grade that corresponds to their age (9–10 grade in Colombia).

Approximately 89% of Colombian students who submitted the PISA test and were enrolled in grade 7 reported having repeated at least one school year, which contrasts with students in grade 11, of whom only 3% reported having repeated a year (Fig. 4). As shown in Fig. 5, vertical stratification has an important impact on the test results and for each year of schooling, scientific literacy increases by approximately 17 points (β = 16.97, p < 0.05). This seems to add to the evidence that grade repetition is one of the main reasons that can lead to disparity between student age and grade (UNESCO, 2012) and is harmful to students’ chances of academic success (Brophy, 2006; Cabrera-Hernandez, 2021; Cheung et al., 2021), because it can exacerbate science learning inequality, particularly among children from lower income families (Pressler et al., 2016).

Fig. 4

Prevalence of grade repetition among Colombian students who participated in PISA 2015

Fig. 5

Science performance of Colombian students in PISA 2015 by grade

Students’ broad interest in science and sense of self-efficacy were also found to have a significant association with their scientific literacy scores. In the context of the SDT framework, these results may indicate that learning outcomes are largely determined by students’ own interest (Karakolidis et al., 2019; Ryan & Deci, 2018), and less by the intrinsic motivation of their teachers.

Does teachers’ motivation influence students’ motivation?

The null model results indicate that the variance in the index of students’ intrinsic motivation that can be attributed to school effects is 2%. Although some methodologists have previously suggested that in the presence of a low ICC, a hierarchical linear model is unnecessary and that the data could be analyzed using Ordinary Least Squares regression (OLS) (Hayes, 2006), recent reviews of this position have demonstrated that even with an ICC of 0.01, the probability of Type I error can reach 0.20, four times higher than the commonly used conventional alpha of 0.05 (Musca et al., 2011; Nezlek, 2008). As a result, modeling the effect of nested data using HLM or other alternative multilevel analysis techniques is currently recommended as a best practice (Huang, 2016, 2018).The results in Table 2 indicate that two of the school level variables showed a significant relationship with students’ motivation: the use of inquiry based-teaching (IBT) strategies (β = 0.63, p < 0.05), which are usually perceived by students as strategies that allow teachers openly explain ideas, generate opportunities for discussion, and consider students’ questions and opinions in learning process; and school socioeconomic status that showed a negative relationship (β = − 0.32, p < 0.05).

Table 2 Summary of parameter estimation for a two-level linear hierarchical model of student intrinsic motivation

Previous studies have shown that IBT can be instrumental in the development of a learning environment that stimulates students’ intrinsic motivation, and where learning is pursued for its own sake, not for external rewards (Valerio, 2012). The use of active methodologies such as IBT can help to dynamize science classes and motivate students to become interested in a subject that is complex and in which the transmission of content by the teacher has traditionally predominated (Freeman et al., 2014). In this case it is not surprising that students find this type of methodology interesting and motivating as they can get involved in learning experiences that are different to the classroom and use equipment or other materials to make observations or study natural phenomena.

Among the student controls, student broad interest in science (β = 0.63, p < 0.05) and sense of self-efficacy (β = 0.12, p < 0.05) were the only ones positively and significantly related to their intrinsic motivation. As mentioned above, these variables are usually related as promoters of self-regulation, life satisfaction and academic achievement (Saroughi & Cheema, 2022).

Scope and limitations of the study

Large-scale cross-national studies such as PISA offer invaluable datasets for empirical analyses, particularly in emerging and low-income countries where financial and logistical constraints often hinder individual researchers. Nonetheless, it is crucial to acknowledge the inherent limitations of working with secondary data (Denscombe, 2010).

Firstly, given the cross-sectional nature of the data used in this study, drawing definitive causal inferences poses a significant challenge. The relationship between teacher motivation and students’ outcomes should be subject to further exploration through longitudinal or experimental studies. These approaches hold the potential to provide more nuanced insights into the causality and dynamics of these relationships.

Secondly, it is important to recognize that the interpretation of our results is circumscribed by the distinct conceptual and operational approaches used to measure motivation for teachers and students. While PISA employs the SDT for the operationalization of the intrinsic motivation construct for students, in this study the assessment of teacher motivation primarily relies on job satisfaction as its lens (Dicke et al., 2019; Klingebiel & Klieme, 2016; OECD, 2016). Hence, it is important to recognize the limitations of using job satisfaction as a proxy for motivation and to interpret the results with caution. While the two variables are likely to be correlated, they are not the same, and job satisfaction may be influenced by factors that are not related to motivation (Dicks et al., 2023; Erarslan, 2021; Zakariya & Wardat, 2023).

However, the use of a proxy offers a starting point for understanding the potential influence of teacher motivation on students, which is a critical aspect of education that has been underrepresented in research. Although the proxy may not capture the entirety of teacher motivation, its use contributes to bridging the gap in our knowledge and underscores the importance of further investigation (Bardach & Klassen, 2020; Bernal et al., 2016). Additionally, the use of a proxy serves as a practical approach within the constraints of available data, highlighting the feasibility of exploring this crucial area of study. Another limitation of the findings is that the available data do not allow to know the moderating or mediating effect that other factors may exert on the relationship between motivation and learning. Teachers with low motivation, for example, are more likely to be late or absent from school, or to leave the teaching profession, all of which have an impact on student learning and academic progress, as well as on the finances of the educational system (Khanal et al., 2021; Msosa, 2020; Transparency International, 2013).

Despite these limitations, this research makes a novel contribution to the study of motivation and learning from an empirical perspective, an important line of inquiry that has hitherto been largely anecdotal and speculative, and highlights the necessity to unifying constructs, operational definitions, and instruments for measuring intrinsic motivation, as well as to expand data collection to include measurements of teachers’ behaviors and attitudes that mediate or moderate the motivation-learning relationship, particularly in emerging and developing economies, where motivation-related problems have been associated with the undermining of learning opportunities for children and youth.

Conclusions

The findings of this study support previous research that highlights the strong effect that teaching strategies and school context can have on educational outcomes, although it did not reveal a significant association between teacher motivation and students’ scientific literacy and motivation However, based on the findings of previous observational research (De Loof et al., 2019; Marshik et al., 2016), it is plausible that teachers can play a significant role in helping students fulfill their needs of competence, relatedness, and autonomy, which in turn can contribute to enhanced learning and motivation. This is a hypothesis worthy of further empirical evaluation.

Teacher-directed instruction and teacher support were found to have important effects on students’ academic performance. Hence, improving teachers’ teaching skills and developing effective strategies for supporting students in the classroom can contribute to a greater enjoyment of learning and a more productive school life for students.

Although more evidence is needed to understand the operational mechanisms underlying teacher motivation and its consequences on students, the results of this research contribute to the identification of variables within the control of schools and teachers that can have a significant and positive impact on students, regardless of their socioeconomic characteristics. The results of this study show that, although socioeconomic characteristics continue to be strong predictors of academic outcomes, teaching practices can have an even larger impact.

Finally, schools must recognize and address the role they have in perpetuating learning gaps between students and gender inequalities in science education. This requires rigorous investigation of the impact of evaluative practices and vertical stratification policies on students’ learning and sociopsychological development, and rooting out teachers’ expectations, biases, and institutional designs that perpetuate traditional gender roles and stereotypes about science learning and careers.

Appendices

Appendix A. Description of variables used in the study

First level variables: individual student characteristics

• Grade the student is enrolled in at the time of taking the test, compared to the modal grade in the country (question ST001 What < grade > are you in?).

• Index of student’s economic, social, and cultural status (ESCS): Composite index created by PISA with an average of 0 and a standard deviation of 1, and includes indicators of parents’ educational level and occupation, as well as the resources available in the home. The rationale for using these three components is that socio-economic status has usually been seen as based on education, occupational status and income (OECD, 2017a).

• Expected occupational status of the student Question ST114: What kind of job do you expect to have when you are about 30 years old?

• Gender

  Question ST004 Are you female or male?
  1	Male
  2	Female

• Educational resources available at home

  Question ST011. Which of the following are in your home?
  ST011Q01NA	A desk to study at
  ST011Q02NA	A room of your own
  ST011Q03NA	A quiet place to study
  ST011Q04NA	A computer you can use for school work
  ST011Q05NA	Educational software
  ST011Q06NA	A link to the Internet
  ST011Q07NA	Classic literature (e.g. < Shakespeare >)
  ST011Q08NA	Books of poetry
  ST011Q09NA	Works of art (e.g. paintings)
  ST011Q10NA	Books to help with your school work
  ST011Q11NA	< Technical reference books >
  ST011Q12NA	A dictionary
  ST011Q16NA	Books on art, music or design
  ST011Q17NA	< Country-specific wealth item 1 >
  1 = Yes; 2 = No

• Availability of ICT resources

  Question ST011. Which of the following are in your home?
  ST011Q05TA	Educational software
  ST011Q06TA	link to the Internet
  1 = Yes; 2 = No

  Question ST012. How many of these are there at your home?
  ST012Q05NA	< Cell phones > with Internet access (e.g. smartphones)
  ST012Q06NA	Computers (desktop computer, portable laptop or notebook)
  ST012Q07NA	Tablet computers (e.g., < iPad® > , < BlackBerry® PlayBook™ >)
  ST012Q08NA	E-book readers (e.g. < Kindle™ > , < Kobo > , < Bookeen >)
  1 = None; 2 = One; 3 = Two 4 = Three or more

• Broad interest in science topics

  Question ST095. To what extent are you interested in the following < broad science > topics?
  ST095Q04NA	Biosphere (e.g. ecosystem services, sustainability)
  ST095Q07NA	Motion and forces (e.g. velocity, friction, magnetic and gravitational forces)
  ST095Q08NA	Energy and its transformation (e.g. conservation, chemical reactions)
  ST095Q13NA	The Universe and its history
  ST095Q15NA	How science can help us prevent disease
  1 = Not interested; 2 = hardly interested; 3 = Interested; 4 = Highly interested; 5 = I don´t know what this is

• Grade repetition: This variable was analyzed as REPEAT index, from Question ST127

  Did you repeat a < grade >?
  0	Did no repeat a < grade >
  1	Repeated a < grade >

• Self-efficacy in science

  Question ST129. How easy do you think it would be for you to perform the following tasks on your own?
  ST129Q01TA	Recognise the science question that underlies a newspaper report on a health issue
  ST129Q02TA	Explain why earthquakes occur more frequently in some areas than in others
  ST129Q03TA	Describe the role of antibiotics in the treatment of disease
  ST129Q04TA	Identify the science question associated with the disposal of garbage
  ST129Q05TA	Predict how changes to an environment will affect the survival of certain species
  ST129Q06TA	Interpret the scientific information provided on the labelling of food items
  ST129Q07TA	Discuss how new evidence can lead you to change your understanding about the possibility of life on Mars
  ST129Q08TA	Identify the better of two explanations for the formation of acid rain
  1 = I could do this easily; 2 = I could do this with a bit of effort; 3 = I would struggle to do this on my own; 4 = I couldn’t do this

Second level variables: school context characteristics

• Economic, social, and cultural status index ESCS of the School: It is the average at the school level of the variable ESCS.

• Curricular orientation of the school (ISCEDO): Indicates whether the curricular content of the program was general, pre-vocational or vocational (OECD, 2017a).

• Teacher’s motivation: Is a composite index constructed for the purposes of this research with items related to teachers’ satisfaction with their profession and with their current work environment.

  Question TC026. We would like to know how you generally feel about your job. How strongly do you agree or disagree with the following statements?
  TC026Q01NA	The advantages of being a teacher clearly outweigh the disadvantages
  TC026Q02NA	If I could decide again, I would still choose to work as a teacher
  TC026Q04NA	I regret that I decided to become a teacher
  TC026Q05NA	I enjoy working at this school
  TC026Q06NA	I wonder whether it would have been better to choose another profession
  TC026Q07NA	I would recommend my school as a good place to work
  TC026Q09NA	I am satisfied with my performance in this school
  TC026Q10NA	All in all, I am satisfied with my job
  1 = Strongly disagree; 2 = Disagree; 3 = Agree; 4 = Strongly agree

• Practice of inquiry-based teaching

  Question ST098. When learning topics at school, how often do the following activities occur?
  ST098Q01TA	Students are given opportunities to explain their ideas
  ST098Q02TA	Students spend time in the laboratory doing practical experiments
  ST098Q03TA	Students are required to argue about science questions
  ST098Q05TA	Students are asked to draw conclusions from an experiment they have conducted
  ST098Q06TA	The teacher explains how a idea can be applied to a number of different phenomena (e.g. the movement of objects, substances with similar properties)
  ST098Q07TA	Students are allowed to design their own experiments
  ST098Q08TA	There is a class debate about investigations
  ST098Q09TA	The teacher clearly explains the relevance of concepts to our lives
  ST098Q10TA	Students are asked to do an investigation to test ideas
  1 = In all lessons; 2 = In most lessons; 3 = In some lessons; 4 = Never or hardly ever

• Practice of directed science instruction

  Question ST103. How often do these things happen in your lessons for this < school science > course?
  ST103Q01NA	The teacher explains scientific ideas
  ST103Q03NA	A whole class discussion takes place with the teacher
  ST103Q08NA	The teacher discusses our questions
  ST103Q11NA	The teacher demonstrates an idea
  1 = Never or almost never; 2 = Some lessons; 3 = Many lessons; 4 = Every lesson or almost every lesson

• Teacher support in a science class of the students’ choice

  Question ST100. How often do these things happen in your < school science > lessons?
  ST100Q01TA	The teacher shows an interest in every student’s learning
  ST100Q02TA	The teacher gives extra help when students need it
  ST100Q03TA	The teacher helps students with their learning
  ST100Q04TA	The teacher continues teaching until the students understand
  ST100Q05TA	The teacher gives students an opportunity to express opinions
  1 = Every lesson; 2 = Most lessons; 3 = Some lessons; 4 = Never or hardly ever

• Student-related factors affecting school climate

  Question SC061. In your school, to what extent is the learning of students hindered by the following phenomena?
  SC061Q01TA	Student truancy
  SC061Q02TA	Students skipping classes
  SC061Q03TA	Students lacking respect for teachers
  SC061Q04TA	Student use of alcohol or illegal drugs
  SC061Q05TA	Students intimidating or bullying other students
  1 = Not at all; 2 = Very little; 3 = To some extent; 4 = A lot

• Teacher-related factors affecting school climate

  Question SC061. In your school, to what extent is the learning of students hindered by the following phenomena?
  SC061Q06TA	Teachers not meeting individual students’ needs
  SC061Q07TA	Teacher absenteeism
  SC061Q08TA	Staff resisting change
  SC061Q09TA	Teachers being too strict with student
  SC061Q10TA	Teachers not being well prepared for classes
  1 = Not at all; 2 = Very little; 3 = To some extent; 4 = A lot

• Type of school

  Question SC013. Is your school a public or a private school?
  1	A public school (This is a school managed directly or indirectly by a public education authority, government agency, or governing board appointed by government or elected by public franchise.)
  2	private school (This is a school managed directly or indirectly by a non-government organization; e.g. a church, trade union, business, or other private institution.)

• Size of the school (total enrollment)

  Question SC002. As at < February 1, 2015 >, what was the total school enrolment (number of students)?
  SC002Q01TA	Number of boys
  SC002Q02TA	Number of girls

• Average class size (Question SC003)

  Question SC003. What is the average size of < test language > classes in < national modal grade for 15-year-olds > in your school?
  1	15 students or fewer
  2	16–20 students
  3	21–25 students
  4	26–30 students
  5	31–35 students
  6	36–40 students
  7	41–45 students
  8	46–50 students
  9	More than 50 students

Outcomes variables

• Scientific Literacy (PVSCIE1): Measurement of students’ academic performance in the natural sciences assessed in PISA. One of the ten plausible values calculated by PISA with average of 500 points and standard deviation of 100 is used. This test measures the academic skills and readiness of 15-year-old students in the use and application of scientific knowledge to identify questions, acquire new knowledge, explain natural phenomena, and draw conclusions based on scientific evidence (OECD, 2016).

• Student’s intrinsic motivation: Is a composite index constructed with questions from the student questionnaire related to enjoyment in learning natural sciences and motivation for academic achievement.

  Question ST094. How much do you disagree or agree with the statements about yourself below?
  ST094Q01NA	I generally have fun when I am learning < broad science > topics
  ST094Q02NA	I like reading about < broad science >
  ST094Q03NA	I am happy working on < broad science > topics
  ST094Q04NA	I enjoy acquiring new knowledge in < broad science >
  ST094Q05NA	I am interested in learning about < broad science >

  Question ST119. To what extent do you disagree or agree with the following statements about yourself?
  ST119Q01NA	I want top grades in most or all of my courses
  ST119Q02NA	I want to be able to select from among the best opportunities available when I graduate
  ST119Q03NA	I want to be the best, whatever I do
  ST119Q04NA	I see myself as an ambitious person
  ST119Q05NA	I want to be one of the best students in my class
  1 = Strongly disagree; 2 = Disagree; 3 = Agree; 4 = Strongly agree

Appendix B. Descriptive statistics

Variable name	N	Mean	SD	Minimum	Maximum
Level-1 descriptive statistics
Broad interest in science (INTBRSCI)	8095	0.34	0.88	− 2.55	2.60
Science self-efficacy SCIEEFF	8095	− 0.09	1.16	− 3.76	3.28
Expected occupational status (BSMJ)	8095	66.32	15.88	10.00	89.00
Grade (GRADE)	8095	− 0.31	1.01	− 3.00	1.00
Repeated grade (REPEAT)	8095	0.38	0.48	0.00	1.00
Home educational resources (HEDRES)	8095	− 0.78	1.03	− 4.41	1.18
ICT resources ICTRES	8095	− 1.04	1.11	− 3.38	3.50
Student socioeconomic status (ESCS)	8095	− 0.77	1.05	− 4.86	2.23
Plausible value 1 in science PV1SCIE	8095	432.63	76.47	159.41	696.73
Plausible value 2 in science PV2SCIE	8095	432.42	76.61	192.24	711.13
Plausible value 3 in science PV3SCIE	8095	432.45	77.25	190.95	688.42
Plausible value 4 in science PV4SCIE	8095	431.90	76.91	204.71	718.77
Plausible value 5 in science PV5SCIE	8095	432.30	76.54	194.47	701.83
Plausible value 6 in science PV6SCIE	8095	432.91	77.11	194.39	706.56
Plausible value 7 in science PV7SCIE	8095	431.95	76.95	181.11	724.54
Plausible value 8 in science PV8SCIE	8095	432.82	76.98	202.28	712.30
Plausible value 9 in science PV9SCIE	8095	431.79	77.51	208.26	712.70
Plausible value 10 in science PV10SCIE	8095	433.46	76.69	200.36	717.01
Student motivation (STU_MOTI)	8095	0.18	1.11	− 4.95	2.27
GENDER (F)	8095	0.54	0.50	0.00	1.00
Level-2 descriptive statistics
School size (SCHSIZE)	313	1190.11	984.53	0.00	7156.00
Average class size (CLSIZE)	313	35.66	11.66	13.00	53.00
Student behavior affecting climate (STUBEHA)	313	0.07	1.23	-2.39	3.89
Teacher behavior affecting climate (TEACHBEHA)	313	− 0.00	1.21	− 2.12	4.26
Teacher support (TEACHSU)	313	0.30	0.30	− 0.96	1.18
Inquiry based teaching (IBTEACH)	313	0.21	0.29	− 0.93	1.48
Directed instruction (TDTEACH)	313	− 0.00	0.29	− 1.66	0.81
School’s Economic, Social, and Cultural Status (ESCSSCHO)	313	− 0.86	0.75	− 3.25	1.13
Teacher motivation (TEACHMOT)	313	0.63	0.41	− 0.78	1.64
School type (SCHOOLTY)	313	0.71	0.45	0.00	1.00