Affective characteristics and mathematics performance in Indonesia, Malaysia, and Thailand: what can PISA 2012 data tell us?

Abstract

Background

The results of the Programme for International Student Assessment (PISA) 2012 showed that Indonesia, Malaysia, and Thailand underperformed and were positioned in the bottom third out of 65 participating countries for mathematics, science, and reading literacies. The wide gap between these three countries and the top performing countries has prompted this study to examine the influence of students’ affective characteristics on their performance in mathematics literacy using a multilevel analysis. The purpose of this study is to examine the relationships among affective characteristics-related variables at the student level, the aggregated school-level variables, and mathematics performance by using the Programme for International Student Assessment (PISA) 2012 dataset.

Method

The data used for the analysis were retrieved from the official PISA website. The student samples from Indonesia, Malaysia and Thailand were 5, 622, 5, 192 and 6, 602, respectively. The data were analysed using descriptive statistics, and a hierarchical linear modeling (HLM) approach with the HLM version 7.0 computer programme.

Results

Different patterns of relationships were found between student- and school-level variables and mathematics performance in the three countries. The common student-level variable is attitudes towards learning outcomes, which predicted an increase in scores for the Indonesian, Malaysian, and Thai models. At the student level, the strongest predictor on mathematics literacy performance was mathematics self-efficacy for both Indonesian and Malaysian models, and perseverance for the Thai Model. At the school level, school average mathematics self-efficacy was the strongest predictor of mathematics performance in the Indonesian model; average openness to problem-solving in the Thai model; and school average instrumental motivation, mathematics behaviour, and attitudes towards learning outcomes predicted a decrease in scores for the Malaysian model.

Conclusion

The inconclusive results of the multilevel analysis has demonstrated some interesting points for discussion, though the results could be attributed to the differences in education system and a diversity of cultural context in each individual country. This study contributes to providing evidence-based policy making in addition to informing the mathematics teachers the particular students’ affective characteristics, which should be strengthened to ensure better mathematics learning outcomes in Indonesia, Malaysia, and Thailand. Implications of the findings and limitations are discussed.

Background

Programme for International Student Assessment (PISA) is a 3-year cycle of large scale international assessments organised by the Organisation for Economic Co-operation and Development (OECD) since 2000. The first PISA survey was launched in 2000, followed by the cycles of 2003, 2006, 2009, and recently in 2012. In each cycle, PISA has assessed three key domains of knowledge and skill, namely reading, mathematics, and science literacies. Each of the three domains is cyclically treated as a major domain. For PISA 2012, the focus was on mathematics literacy with two newly added domains: problem-solving and financial literacies. PISA offers insight for education policy and practice, which helps to monitor trends in students’ acquisition of knowledge, and skills across countries, economies and different demographic subgroups within each country. More specifically, PISA provides reliable empirical evidence to identify the strengths and weaknesses of the education systems and presents examples of good practices in the sense of benchmarking (Prensel et al. 2013).

Notably, the East Asian countries have a higher performance than their western counterparts in PISA 2012. Shanghai-China, Hong Kong-China, Japan, Korea, and China-Taipei are the top performing East Asian countries in mathematics, science, and reading literacies (OECD 2013). Literature shows that scholars attribute the high educational performance of students to the Asian model of learners with the focus on the Chinese ways of learning and teaching (e.g., Li 2004; Schneider and Lee 1990; Stevenson and Stigler 1992; Watkins and Biggs 1996; Wong 2004). For instance, students’ values and aspirations are shared with their parents (Schneider and Lee 1990; Yao 1985), students’ motivation for socioeconomic advancement or self-perfection are developed through education (Lee 1996; Salili et al. 2001), learning activities are undertaken at home (Schneider and Lee 1990; Stevenson and Stigler 1992), and teacher-student relations are more respectable (Hau and Salili 1991; Schneider and Lee 1990; Stevenson and Stigler 1992).

Compared to the above mentioned East Asian countries, the five participating Southeast Asian countries showed a diversity of performance in mathematics, science, and reading literacies in PISA 2012. Singapore maintained its position as one of the top performers in mathematics, science, and reading literacies. Vietnam has shown a high performance in reading, mathematics, and science literacies and was positioned 17th out of 65 countries. In contrast, Indonesia, Malaysia, and Thailand ranked in the bottom third countries participating in PISA 2012. The mean scores of mathematics, science, and reading literacies in Indonesia, Malaysia, and Thailand are relatively lower than the corresponding OECD average in PISA 2012. This revealed the challenges faced by these three countries to enhance students’ performance in mathematics, science, and reading literacies.

The results have thus raised questions in regards to what PISA 2012 data can tell us about factors that might relate to performance in mathematics literacy in Indonesia, Malaysia, and Thailand. In relation to this, affective characteristics have gained prominence in education and psychology research due to its remarkable effect on schooling processes and outcome (McCoach et al. 2013; Hattie 2009). Affective is often referred, but not limited by the researchers as students’ emotions or feelings towards mathematics (Reyes 1984; Zan et al. 2006). Affective research in the 1960s and 1970s has often referred to mathematics anxiety and attitudes towards mathematics (Zan et al. 2006). Later, various characteristics such as beliefs, motivation, interest, and values were studied in mathematics education context (e.g., Grootenboer and Hemmings 2007; Zan et al. 2006). A voluminous of previous studies indicated that affective characteristics were associated with their mathematics performance such as mathematics anxiety (Ma and Xu 2004; Reyes 1984), students’ self-efficacy (Bandura 1997; Marsh 1987), and their perception of the classroom environment during mathematics lessons (Barth et al. 2004). Hattie (2009) emphasised that self-efficacy, self-concept, motivation, engagement, and persistence were highly correlated with performance. Further, attitudes towards mathematics have often shown positive relationships in mathematics performance (Chow 2011; Kumar and Morris 2005; Wong 1993). Students who have shown positive attitudes towards mathematics have performed better in their mathematics performance (Chow 2011). A recent study by Kim, Park, and Cozart (2014) on affective factors revealed that students’ motivation and affective emotion, such as enjoyment and anxiety, played an important role in contributing to their mathematics performance.

Apart from empirical support, literature provides reasons behind why personality variables may have an effect on performance (O’Connor and Paunonen 2007). First, there are behavioural tendencies reflected in personality traits that can affect certain habits influencing academic performance, such as perseverance and conscientiousness. Second, cognitive ability reflects what an individual can do, and personality traits reflect what an individual will do. Wilkins and Ma (2003) emphasised the significance of affective in enabling daily mathematics knowledge application. On the other hand, the earlier study by Bloom (1976) indicated that affective characteristics accounted for 25 per cent of the variance in student performance. Reyes (1984) highlighted the importance of positive attitudes towards learning by creating a motivational learning environment, which was believed to increase performance level.

In a related vein, previous studies related to students’ affective characteristics and students’ performance using PISA dataset with multilevel analysis is considered rare in Indonesia, Malaysia, and Thailand despite their early participation since the first cycle of PISA in 2000 for Indonesia and Thailand, and 2009+ for Malaysia. The individual country report was more focused on the mean score and percentages of students’ performances by certain demographic characteristics such as gender and school location based on the secondary dataset (e.g., Ministry of Education 2013). It is important to highlight that the ignorance of using multilevel analysis in dealing with the PISA dataset could jeopardize the important empirical evidence such as a decomposition of variance of student performance by school and student level in line with the nested structure of the dataset (Raudenbush and Bryk 2002).

With reference to the Indonesian, Malaysian and Thailand have lower performance than the OECD average on mathematics in PISA 2012, the effort to undertake an investigation into the students’ affective characteristics is justified based on several common characteristics. First, Indonesia, Malaysia, and Thailand are the Southeast Asian neighbouring peer groups, being grouped as countries and economies with per capita Gross Domestic Product (GDP) less than USD 20,000 (Ministry of Education 2013; OECD 2014). Secondly, the three countries are categorised in the medium human development group of countries with the human development indicators (HDI) of 2012 of 0.773, 0.722, and 0.684, respectively (UNDP 2014). Specifically, Malaysia and Thailand have had good access to basic education since the eighties (Jimenez et al. 2012). Both countries had also achieved gross secondary rates of 68 and 76 %, respectively; and the tertiary gross enrolment rates of 36 and 45 %, respectively (Jimenez et al. 2012). Hence, it deserves to examine affective-related factors that might associate with students’ performance in mathematics literacy across Indonesia, Malaysia, and Thailand.

In addressing the research gap, the purpose of this study is to examine the relationships among affective-related variables at the student level, the aggregated school-level variables, and students’ performance in mathematics literacy. Two research questions were advanced in accordance to the purposes of this study.

• (a) Which student and school-level variables are associated with mathematics performance in Indonesia, Malaysia, and Thailand?

• (b) What is the proportion of variance explained at the student and school levels in Indonesia, Malaysia, and Thailand?

This study provides evidence-based insight for policy makers of the respective countries of the affective factors that affect quality of students’ performance in mathematics literacy, and to learn from others while enhancing its own strengths in the education system.

Theoretical underpinning

Bloom’s (1976) student learning model has identified students’ affective entry behaviours are related to student learning outcomes. According to Bloom (1976), the affective entry behaviours includes non-cognitive characteristics such as the academic self-concept of the students, and in general their attitudes towards particular subjects in school, such as reading and mathematics. In accordance to Bloom’s (1976) student learning model, educational and psychological theories of achievement, such as the social cognitive theory (Bandura 1989) has highlighted distinct non-cognitive, social-emotional characteristics of students that impact on student performance. Wang, Haertel and Walberg (1993) found that students’ affective-motivational attitudinal disposition was an important factor in influencing student-learning outcomes based on 91 meta analyses. Bloom’s (1976) student learning model was further supported by voluminous studies that indicate students’ affective characteristics influenced their learning outcomes, including academic self-concept (e.g., Marsh and Yeung 1997), self-efficacy (e.g., Pajares 1996), anxiety (e.g., Everson et al. 1994), and attitude towards learning (Fantuzzo et al. 2004). Therefore, investigating the affective variables and their relationship with students’ performance in mathematics literacy is undertaken in this study.

On the other hand, multilevel relationships among school and student characteristics and educational outcomes at both school and student levels should be taken into consideration due to the hierarchical structure of PISA dataset (Raudenbush and Bryk 2002). The argument involved is supported theoretically by the multilevel organisation theory (MOT). MOT explains that the interaction process can occur simultaneously at the lower (student level) and higher levels (school level). MOT specifies the relationships between the lower and higher levels by highlighting the top-down processes, referring to the direct effects from a higher-level unit (school characteristics) to a lower-level unit (such as student performance) (Kozlowski and Klein 2000). In the school context, the higher-level unit is referred to as school factors such as school types at school level whereas the lower-level unit refers to an educational outcome such as student performance. In this study, the student-level variables refer to the affective variables, whereas the school-level variables refer to the aggregated student-level variables at school level.

Student- and school-level variables

Bloom’s (1976) student learning model and MOT are used to derive the student- and school-level variables that are hypothesised to have relationships with the Malaysian students’ performance in mathematics literacy.

Affective characteristics-related measures in PISA 2012 can be categorised into three components: (1) student engagement with and at schools, (2) students’ drive and motivation, and (3) mathematics self-beliefs, dispositions, and participation in mathematics-related activities. Table 1 shows the operational definitions of the measures for each component. For the ease of interpretation, measures of student learning characteristics were capitalised throughout the study. The school-level variables are referred to as the aggregates of student-level variables.

Table 1 Measures and its operationalisation

Sampling design and data sources

PISA used a two-stage stratified design sampling (Willms and Smith 2005). In the first stage of sampling, schools having age-eligible students were sampled systematically with probabilities proportional to the school size, which is a function of the number of eligible students enrolled. A minimum of 150 schools was selected in each country. This was followed by randomly selecting a number of students around 15 years of age in the selected schools. Table 2 shows the sample demographic characteristics in Indonesia, Malaysia, and Thailand. The sample size ranged from 5197 to 6606. The gender ratio was fairly equal, except the Thai student sample. The number of Thai female students is higher than male students compared to Indonesia and Malaysia. One of the possible reasons in explaining the difference is due to the fact that most of the Thai male students are studying in the religious schools, such as the Buddhist or Islamic schools (Precharn, public communication, August 4, 2015). The religious schools were not selected in PISA 2012.

Table 2 Demographic characteristics

The data used for the analysis was retrieved from the official PISA website (http://www.pisa.oecd.org). Each index was then standardised, with the average score across OECD countries set at zero and the standard deviation set at one. A positive value on an index indicates that scores obtained in a particular country were higher than the OECD average, which in turn indicates that students in this country have more positive perceptions on the undertaken affective characteristics compared to the students from other OECD countries, and vice versa for an index with negative value. However, such interpretation needs to be handled with caution, as it is not consistent for all indexes. The interpretation should be relevant in how each undertaken affective characteristic has been operationalised as shown in Table 1. For example, positive value of the index of Mathematics anxiety indicates that students reported higher levels of anxiety towards mathematics than the OECD average and vice versa for the negative value. However, a positive value of high responsibility in failing in mathematics indicates students tend to attribute the responsibility for failure in solving mathematics problems to themselves; a negative value indicates students are more likely to see other individual or factors as responsible to their failure in mathematics (OECD 2013).

HLM analysis procedure

As students are nested within schools, multilevel analysis was conducted to examine the effects of a higher-level unit such as school factors, and on a lower-level unit such as student factors (Hox 2010; Raudenbush and Bryk 2002). In fact, there have been numerous studies that have attempted to ascertain the amount of variance that can be attributed to the school input through multilevel modelling procedures (Fitz-Gibbon and Kochan 2000; Teddlie et al. 2000). In a related vein, hierarchical linear modelling (HLM) is a statistical technique for analysing the hierarchical structure of PISA 2012 database. We used two-level HLM to examine the relationships between student- and school-level variables and mathematics performance using HLM 7.0 computer software. HLM is not only able to calculate using plausible values and handling missing values; it can also replicate the analysis across all of the plausible values and compute standard errors of the coefficients based on the full analysis (Willms and Smith 2005). The HLM2 module of HLM 7 program can handle missing data at level-1 of the hierarchy. Observations with missing data will be deleted using listwise deletion at either the MDM creation stage or when the analysis is run. HLM assumes level-2 data files to be completed. If any of the higher-level files contain missing data, units with missing data will automatically be deleted when the MDM file is created (Raudenbush et al. 2011). In PISA 2012, the probabilities selection varied although the sample was chosen randomly. Each index was scaled using a weighted maximum likelihood estimate method with its multiple questions and responses. As such, student and school weight variables abbreviated as W_FSTUWT and W_FSCHWT, respectively in the database were incorporated into the analysis to ensure that each sampled student and school represented the correct number of students and schools in the population.

Centering is an important issue in multilevel analysis. According to Raudenbush and Bryk (2002), centering could be presented in the form of grand mean centering and group mean centering. Grand mean centering is achieved by subtracting the sample mean from each student score ($x_{ij} - \bar{x}$). Meanwhile, group mean centering is done by subtracting the mean of the school that the student attends from the scores for each student within that school ($x_{ij} - \bar{x}_{j}$). In this study, we centred all student and school variables around the grand mean except for gender, where the variable is included in the equation uncentered. The purpose of this is to reduce the multicollinearity among variables and bias in variance estimates so that a more meaningful interpretation can be made (Kreft and de Leeuw 1998).

The multilevel model building begins with a null model. The null model contains only the dependent variable, namely mathematics performance, and no dependent variables except an intercept. The null model is statistically equivalent to one-way random effects analysis of variance (Raudenbush and Bryk 2002). The null model has two purposes. First, to estimate the grand mean of mathematics performance with adjustments for clustering of students within schools and for different sample sizes across schools. Second, to estimate variance components available by decomposing the total variance in mathematics performance into variance attribute to students (within school variance) and variance attribute to schools (between school variance). Practically, the null model serves as the baseline model to compare with the results of the final model. The final model is developed by adding the undertaken student- and school-level variables to the null model. The final model includes the student- and school-level variables that demonstrated a statistically significant relationship with mathematics performance. In this study, separate analyses were conducted for the three countries using the above procedures. However, the models for these three countries were identical to maintain their comparability and to enable between countries comparisons to be made.

Results

Table 3 shows the results of the HLM analysis with the null model. Intraclass correlation coefficient (ICC) refers to the ratio between the amount of variance at the school level to the total amount of variance available at both student and school level. Table 3 shows that about 47 % of the total variance in mathematics performance is attributed to schools in Indonesia, about 35 % in Malaysia, and about 58 % in Thailand. The results indicated that variability of average mathematics performance between schools in these three Southeast Asian countries was sizeable. The results confirmed the need to investigate the relationships between student- and school-level variables, and mathematics performance for each country using a multilevel approach. Building on the null model, the student-level variables were added into the null model. The non-significant student-level variable with the highest p value was first excluded using the backward elimination approach. The analysis was repeated until all non-significant student-level variables were excluded from the analysis. The same procedure was repeated for the school-level variables.

Table 3 HLM results of the fixed and random effects of the null models

Indonesia

Table 4 shows that mathematics self-efficacy (MATHEFF) and attitudes towards learning outcomes (ATTSHCL) had statistically significant and positive effects on mathematics performance at the student level in Indonesia. MATHEFF was the strongest predictor of mathematics performance with one standard deviation increase in MATHEFF was associated with an increase in mathematics performance of about 13 points after controlling all other variables. Meanwhile, one standard deviation increase in ATTSCHL was associated with an increase in mathematics performance of about five points when controlling all other variables. The other three student-level variables: Anxiety in mathematics (ANXMAT), mathematics self-concept (SCMAT), and mathematics behaviour (MATBEH) were found negatively related to mathematics performance. One point standard deviation increase in ANXMAT was associated with a decrease in mathematics performance of about 10 points. This was followed by one standard deviation increase in SCMAT and MATBEH, which were associated with a decrease in mathematics performance of about eight and five points, respectively after controlling other variables.

Table 4 HLM results of the fixed and random effects of the final models

Variables that start with the capital “S” represent the school-level variables (aggregated from student-level variables),

$${\text{Proportion of variance explained at school level}} = \tau_{00} (null) - \tau_{00} (final)/\tau_{00} (null)$$

$${\text{Proportion of variance explained at student level}} = \sigma^{2} (null) - \sigma^{2} (final)/\sigma^{2} (null).$$

At the school level, school average mathematics self-efficacy (SMATHEFF) is the strongest predictor that is positively associated with mathematics performance. One standard deviation increase in SMATHEFF was associated with an increase in mathematics performance of about 61 points. This indicated that schools with high mathematics self-efficacy have better average performances in mathematics compared to schools with lower mathematics self-efficacy. School average sense of belonging (SBELONG) and school average openness to problem-solving (SOPENPS) were also found to be positively associated with mathematics performance, with one standard deviation increase in SBELONG and SOPENPS being associated with an increase in mathematics performance of about 30 and 23 points, respectively. However, school average mathematics interest (SINTMAT) was found to be negatively associated with mathematics performance, indicated by one standard deviation increase in SINTMAT, and associated with a decrease of mathematics performance of about 66 points. Similarly, school average mathematics self-concept (SSCMAT) and school average mathematics behaviour (SMATHBEH) were found to be negatively associated with mathematics performance. One standard deviation increase in SSCMAT was associated with a decrease in mathematics performance of about 36 points, whereas one standard deviation increase in SMATBEH was associated with a decrease in mathematics performance of about 26 points. Relative to the null model, the final model explained about 44 % of variance at the school level and 7 % of variance at the student level.

Malaysia

Similarly to Indonesia, MATHEFF was the strongest predictor of mathematics performance in Malaysia. One standard deviation increase in MATHEFF was associated with an increase in mathematics performance of about 25 points after controlling all other variables. This was followed by ATTSCHL and instrumental motivation (INSTMOT). One standard deviation increase in ATTSCHL and INSTMOT contributed to an increase in mathematics performance of about nine and eight points, respectively. However, MATBEH and BELONG were found to be negatively associated with mathematics performance in Malaysia. One standard deviation increase in MATHBEH and BELONG was associated with a decrease in mathematics performance of about 10 and 6 points, respectively after controlling other variables. It is worthy to highlight that three school-level variables: SMATBEH, school average instrumental motivation (SINSTMOT), and school average attitude towards learning outcomes (SATTSCHL) were also found to be negatively associated with mathematics performance in Malaysia. One standard deviation increase in SCMATBEH was associated with a decrease in mathematics performance of about 55 points after controlling all other variables. Similarly, one standard deviation increase in SINSTMOT and SATTSCHL was associated with decreases in mathematics performance of about 34 and 33 points, respectively. Overall, the final model explained about 58 % of variance at the school level and 17 % of variance at the student level in Malaysia.

Thailand

Thailand PISA 2012 data informed that perseverance (PERSEV) was the strongest predictor of mathematics performance at the student level. One standard deviation increase in PERSEV was associated with an increase in mathematics performance of about 10 points. One standard deviation increase in ATTSCHL contributes about eight points of increase in mathematics performance. Results revealed a negative relationship between mathematics interest (INTMAT) and mathematics performance, as well as ATTLNACT and mathematics performance in Thailand. One standard deviation increase in INTMAT and ATTLNACT was associated with a decrease in mathematics of about seven and five points, respectively. On the other hand, a negative relationship was also found between FAILMAT and mathematics performance, with one standard deviation increase in FAILMAT associated with a decrease in mathematics performance of about nine points. Although not significant at the student level, the school average openness to problem-solving (SOPENPS) was found to be positively associated with mathematics performance in Thailand. One standard deviation increase in SOPENPS was associated with an increase in mathematics performance of about 38 points. SATTSCHL was also found to be positively associated with mathematics performance with a one-point standard deviation increase in SATTSCHL as associated with an increase in mathematics performance of about 27 points. Compared to Indonesia and Malaysia, the proportions of variance explained at the school level and student level are not significantly different to Thailand, with 25 % of variance at the school level and 16 % of variance at the student level. However, the variance explained at student level in the Thai model was found more than twice the variance explained in the Indonesian model. The results informed that the influence of the significant student-level variables on students’ performance in mathematics literacy in the Thai model was stronger when compared to Indonesia. In addition, the variance explained at school level in the Thai model was found to be less than half when compared to the Malaysian model. The results indicate that the influence of the significant school-level variables on mathematics literacy in Thailand is smaller when compared to Malaysia.

Conclusion and discussion

The results of Programme for International Student Assessment (PISA) 2012 showed that Indonesia, Malaysia, and Thailand underperformed and were positioned in the bottom third out of 65 participating countries for mathematics, science, and reading literacies. The wide gap between these three countries and the top performing countries in the performance of PISA 2012 has prompted this study to examine the influence of students’ affective characteristics on their performance in mathematics literacy using a multilevel analysis.

This study has provided empirical evidence on how affective characteristics, in terms of social and emotional factors, are associated with the PISA 2012 mathematics performance in Indonesia, Malaysia, and Thailand. In essence, the inconclusive results of the multilevel analysis have demonstrated some interesting points for discussion.

The results could be seen with some contradictory interpretations. In Indonesian context, the lesser the students participated in mathematics-related activities and the lesser they perceived the importance of schools to their future career, the better the performance they reached. Similarly, in Malaysian context, the more the students tend to thrive when they form positive relationships with peers, feel happy and at ease at school, the lower the performance they reached; and the more the students participated in mathematics-related activities at or outside of school, the lower the performance they reached. Meanwhile, in the Thai context, the lesser the students enjoy and work hard in mathematics, the better their performance. The results provide insight in regards to the relevancy of the items, and fully capture the conceptual meaning of the undertaken affective variables in the Indonesian, Malaysian, and Thai context due to the possibility of item translation that bias from its original item. This is because the same question been translated into different languages could have generated different meaning depending on the different cultural background (Solano-Flores et al. 2009). In addition, such contradictory interpretation could be explained, with possible reason that competencies tested by PISA and referred to by the variables do not correspond well with content covered by teaching in these three countries.

The inconclusive results appeared at both student and school level. At the student level, attitudes towards learning outcomes was the only common variable that predicted increased scores for all the country models. Mathematics self-efficacy predicted increased scores in Indonesia and Malaysia, but no significant increase in the Thai model. Instrumental motivation predicted increased scores in Malaysia, but was not significant in the Indonesian and Thai models. Meanwhile, mathematics behaviour predicted decreased scores in the Indonesian and Malaysian models only. On the other hand, attitudes towards learning activities predicted decreased scores in the Indonesian and Thai models only. Self-responsibility in failing mathematics is only significant in the Thai model with predicted decrease in scores. Mathematics self-concept predicted a decrease in scores for the Indonesian model only. At the school level, the Indonesian model has the highest number of significant predictors of mathematics performance. School average of sense of belonging, openness to problem-solving, and mathematics self-efficacy predicted increase in scores, whereas school average mathematics behaviour, mathematics self-concept, and intrinsic motivation predicted decrease in scores. Meanwhile, school average attitudes towards learning activities and openness to problem-solving were the only two school-level variables with a predicted increase in scores in the Thai model. However, school average mathematics interest, mathematics behaviour, and attitudes towards learning outcomes were the only three significant school-level variables with predicted decreases in scores for the Malaysian model. The findings also revealed that Malaysia has the highest school variance, followed by Indonesia and Thailand. This indicates that the effects of school traits on variation in students’ mathematics performance are much greater in Malaysia than Indonesia and Thailand.

Clearly, the different relationship pattern between each undertaken student- and school-level variables and mathematics performance were identified. The cultural differences in the three countries might serve as a possible reason in explaining the different interpretations of the undertaken affective variables at both the student and school levels. According to Klieme (2013), cultural differences in education systems and school atmospheres may contribute to explaining the different patterns of relationships between students, school factors, and mathematics performance among these three countries. This is because the psychological characteristics of individuals are influenced by the cultures in which they live (Triandis 1995; Tweed and Lehman 2002). In this study, affective characteristics are not an exception. As such, the influence of sociopsychological factors on affective characteristics should be understood in consideration of cultures and contexts. In addition, the different medium of instruction used in teaching the mathematics subject in Malaysia, Thailand, and Indonesia could explain such results. The medium of instruction is the Thai language in Thailand, Indonesian language for Indonesia, and Malay or English language for Malaysia.

These hypothetical explanations of the results in the study, however, should be examined empirically or qualitatively in the follow-up research. Our understanding of affective characteristics based on multicultural perspective will be enhanced through a deeper investigation. Subsequently, such comprehensive understanding would serve for identifying ways of changing educational practices and systems for better educational outcomes. On the other hand, it is important to highlight that this study could also have not considered or included other affective-related variables which might associate with mathematics performance.

For policy implication, findings from this current study shed light on the importance of teachers to incorporate significant affective components into their classroom teaching or intervention programs in order to improve and enhance students’ understanding in mathematics. For instance, the Indonesian mathematics teachers should strike to strengthen students’ attitudes toward learning outcomes and self-efficacy in their teaching, as these affective factors strongly influence their mathematics performance. Meanwhile, the Malaysian teachers should enhance students’ instrumental motivation, their attitudes towards learning outcomes, self-efficacy, students’ perseverance; and Thai teachers should enhance their students’ attitudes towards learning outcomes. The results further suggest that there is room for Indonesian and Thai teachers to foster student learning by promoting self-confidence and connectedness to school and peers in mathematics classroom.

The findings further inform the challenges faced by the education stakeholders in the Southeast Asian societies that might need to strike a balance between the cognitive and affective domain of students’ mathematics learning. For the implication on research, the findings inform the needs for researchers to review and develop more context-based items that able to capture the conceptual meaning of the undertaken variables based on the three countries context. More investigation on other factors that might impact on mathematics performance are needed especially in Indonesian, Malaysian, and Thai context.

This study is not without limitations. The questionnaire items of affective characteristics in PISA 2012 might be too general and therefore might not fully capture students’ perception of their learning characteristics. In addition, translating items creates the potential to lose integrity in the equivalency of what is being asked from one country to other country (Ram 2007). Affective characteristics and aggregated school variables in this study only accounted for part of the variances in mathematics performance. Students’ mathematics performance depends on multiple factors that contribute to a large proportion of unexplained variance. For instance, teachers’ instructional practices (O’Dwyer et al. 2015), socio-economic composition of the schools (Rumberger and Palardy 2005) as well as student and school resources (Topҫu et al. 2014) within each of these three Southeast Asian countries should be examined in future studies.

Similar analysis could be conducted using data in future cycles of PISA or from other international studies such as trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) to cross-validate the findings. In relation to this, a number of questions are worthwhile for future studies: Why do students in Indonesia, Malaysia, and Thailand have a low level of performance but high levels of perseverance, mathematics interest, instrumental motivation, and mathematics behaviour? Why do the three Southeast Asian schools engage students’ bodies but not their hearts? What are the possible ways to improve mathematics performance from a cultural perspective?

Overall, this study represents a multilevel analysis in examining the relationships between the student and school level affective variables and mathematics performance in PISA 2012 across Indonesia, Malaysia, and Thailand. This study contributes to provide evidence-based policy making. Practically, the findings inform direction for future studies in providing empirical evidence on how students’ affective characteristics impact on their mathematics performance in the three countries. It is hoped that this study serves as not only a reference to provide knowledge and empirical evidence for the researchers, but aid in gaining a better understanding of the effects of policies on education outcomes.