 Research
 Open Access
 Published:
Affective characteristics and mathematics performance in Indonesia, Malaysia, and Thailand: what can PISA 2012 data tell us?
Largescale Assessments in Education volume 3, Article number: 3 (2015)
Abstract
Background
The results of the Programme for International Student Assessment (PISA) 2012 showed that Indonesia, Malaysia, and Thailand underperformed and were positioned in the bottom third out of 65 participating countries for mathematics, science, and reading literacies. The wide gap between these three countries and the top performing countries has prompted this study to examine the influence of students’ affective characteristics on their performance in mathematics literacy using a multilevel analysis. The purpose of this study is to examine the relationships among affective characteristicsrelated variables at the student level, the aggregated schoollevel variables, and mathematics performance by using the Programme for International Student Assessment (PISA) 2012 dataset.
Method
The data used for the analysis were retrieved from the official PISA website. The student samples from Indonesia, Malaysia and Thailand were 5, 622, 5, 192 and 6, 602, respectively. The data were analysed using descriptive statistics, and a hierarchical linear modeling (HLM) approach with the HLM version 7.0 computer programme.
Results
Different patterns of relationships were found between student and schoollevel variables and mathematics performance in the three countries. The common studentlevel variable is attitudes towards learning outcomes, which predicted an increase in scores for the Indonesian, Malaysian, and Thai models. At the student level, the strongest predictor on mathematics literacy performance was mathematics selfefficacy for both Indonesian and Malaysian models, and perseverance for the Thai Model. At the school level, school average mathematics selfefficacy was the strongest predictor of mathematics performance in the Indonesian model; average openness to problemsolving in the Thai model; and school average instrumental motivation, mathematics behaviour, and attitudes towards learning outcomes predicted a decrease in scores for the Malaysian model.
Conclusion
The inconclusive results of the multilevel analysis has demonstrated some interesting points for discussion, though the results could be attributed to the differences in education system and a diversity of cultural context in each individual country. This study contributes to providing evidencebased policy making in addition to informing the mathematics teachers the particular students’ affective characteristics, which should be strengthened to ensure better mathematics learning outcomes in Indonesia, Malaysia, and Thailand. Implications of the findings and limitations are discussed.
Background
Programme for International Student Assessment (PISA) is a 3year cycle of large scale international assessments organised by the Organisation for Economic Cooperation and Development (OECD) since 2000. The first PISA survey was launched in 2000, followed by the cycles of 2003, 2006, 2009, and recently in 2012. In each cycle, PISA has assessed three key domains of knowledge and skill, namely reading, mathematics, and science literacies. Each of the three domains is cyclically treated as a major domain. For PISA 2012, the focus was on mathematics literacy with two newly added domains: problemsolving and financial literacies. PISA offers insight for education policy and practice, which helps to monitor trends in students’ acquisition of knowledge, and skills across countries, economies and different demographic subgroups within each country. More specifically, PISA provides reliable empirical evidence to identify the strengths and weaknesses of the education systems and presents examples of good practices in the sense of benchmarking (Prensel et al. 2013).
Notably, the East Asian countries have a higher performance than their western counterparts in PISA 2012. ShanghaiChina, Hong KongChina, Japan, Korea, and ChinaTaipei are the top performing East Asian countries in mathematics, science, and reading literacies (OECD 2013). Literature shows that scholars attribute the high educational performance of students to the Asian model of learners with the focus on the Chinese ways of learning and teaching (e.g., Li 2004; Schneider and Lee 1990; Stevenson and Stigler 1992; Watkins and Biggs 1996; Wong 2004). For instance, students’ values and aspirations are shared with their parents (Schneider and Lee 1990; Yao 1985), students’ motivation for socioeconomic advancement or selfperfection are developed through education (Lee 1996; Salili et al. 2001), learning activities are undertaken at home (Schneider and Lee 1990; Stevenson and Stigler 1992), and teacherstudent relations are more respectable (Hau and Salili 1991; Schneider and Lee 1990; Stevenson and Stigler 1992).
Compared to the above mentioned East Asian countries, the five participating Southeast Asian countries showed a diversity of performance in mathematics, science, and reading literacies in PISA 2012. Singapore maintained its position as one of the top performers in mathematics, science, and reading literacies. Vietnam has shown a high performance in reading, mathematics, and science literacies and was positioned 17th out of 65 countries. In contrast, Indonesia, Malaysia, and Thailand ranked in the bottom third countries participating in PISA 2012. The mean scores of mathematics, science, and reading literacies in Indonesia, Malaysia, and Thailand are relatively lower than the corresponding OECD average in PISA 2012. This revealed the challenges faced by these three countries to enhance students’ performance in mathematics, science, and reading literacies.
The results have thus raised questions in regards to what PISA 2012 data can tell us about factors that might relate to performance in mathematics literacy in Indonesia, Malaysia, and Thailand. In relation to this, affective characteristics have gained prominence in education and psychology research due to its remarkable effect on schooling processes and outcome (McCoach et al. 2013; Hattie 2009). Affective is often referred, but not limited by the researchers as students’ emotions or feelings towards mathematics (Reyes 1984; Zan et al. 2006). Affective research in the 1960s and 1970s has often referred to mathematics anxiety and attitudes towards mathematics (Zan et al. 2006). Later, various characteristics such as beliefs, motivation, interest, and values were studied in mathematics education context (e.g., Grootenboer and Hemmings 2007; Zan et al. 2006). A voluminous of previous studies indicated that affective characteristics were associated with their mathematics performance such as mathematics anxiety (Ma and Xu 2004; Reyes 1984), students’ selfefficacy (Bandura 1997; Marsh 1987), and their perception of the classroom environment during mathematics lessons (Barth et al. 2004). Hattie (2009) emphasised that selfefficacy, selfconcept, motivation, engagement, and persistence were highly correlated with performance. Further, attitudes towards mathematics have often shown positive relationships in mathematics performance (Chow 2011; Kumar and Morris 2005; Wong 1993). Students who have shown positive attitudes towards mathematics have performed better in their mathematics performance (Chow 2011). A recent study by Kim, Park, and Cozart (2014) on affective factors revealed that students’ motivation and affective emotion, such as enjoyment and anxiety, played an important role in contributing to their mathematics performance.
Apart from empirical support, literature provides reasons behind why personality variables may have an effect on performance (O’Connor and Paunonen 2007). First, there are behavioural tendencies reflected in personality traits that can affect certain habits influencing academic performance, such as perseverance and conscientiousness. Second, cognitive ability reflects what an individual can do, and personality traits reflect what an individual will do. Wilkins and Ma (2003) emphasised the significance of affective in enabling daily mathematics knowledge application. On the other hand, the earlier study by Bloom (1976) indicated that affective characteristics accounted for 25 per cent of the variance in student performance. Reyes (1984) highlighted the importance of positive attitudes towards learning by creating a motivational learning environment, which was believed to increase performance level.
In a related vein, previous studies related to students’ affective characteristics and students’ performance using PISA dataset with multilevel analysis is considered rare in Indonesia, Malaysia, and Thailand despite their early participation since the first cycle of PISA in 2000 for Indonesia and Thailand, and 2009+ for Malaysia. The individual country report was more focused on the mean score and percentages of students’ performances by certain demographic characteristics such as gender and school location based on the secondary dataset (e.g., Ministry of Education 2013). It is important to highlight that the ignorance of using multilevel analysis in dealing with the PISA dataset could jeopardize the important empirical evidence such as a decomposition of variance of student performance by school and student level in line with the nested structure of the dataset (Raudenbush and Bryk 2002).
With reference to the Indonesian, Malaysian and Thailand have lower performance than the OECD average on mathematics in PISA 2012, the effort to undertake an investigation into the students’ affective characteristics is justified based on several common characteristics. First, Indonesia, Malaysia, and Thailand are the Southeast Asian neighbouring peer groups, being grouped as countries and economies with per capita Gross Domestic Product (GDP) less than USD 20,000 (Ministry of Education 2013; OECD 2014). Secondly, the three countries are categorised in the medium human development group of countries with the human development indicators (HDI) of 2012 of 0.773, 0.722, and 0.684, respectively (UNDP 2014). Specifically, Malaysia and Thailand have had good access to basic education since the eighties (Jimenez et al. 2012). Both countries had also achieved gross secondary rates of 68 and 76 %, respectively; and the tertiary gross enrolment rates of 36 and 45 %, respectively (Jimenez et al. 2012). Hence, it deserves to examine affectiverelated factors that might associate with students’ performance in mathematics literacy across Indonesia, Malaysia, and Thailand.
In addressing the research gap, the purpose of this study is to examine the relationships among affectiverelated variables at the student level, the aggregated schoollevel variables, and students’ performance in mathematics literacy. Two research questions were advanced in accordance to the purposes of this study.

(a)
Which student and schoollevel variables are associated with mathematics performance in Indonesia, Malaysia, and Thailand?

(b)
What is the proportion of variance explained at the student and school levels in Indonesia, Malaysia, and Thailand?
This study provides evidencebased insight for policy makers of the respective countries of the affective factors that affect quality of students’ performance in mathematics literacy, and to learn from others while enhancing its own strengths in the education system.
Theoretical underpinning
Bloom’s (1976) student learning model has identified students’ affective entry behaviours are related to student learning outcomes. According to Bloom (1976), the affective entry behaviours includes noncognitive characteristics such as the academic selfconcept of the students, and in general their attitudes towards particular subjects in school, such as reading and mathematics. In accordance to Bloom’s (1976) student learning model, educational and psychological theories of achievement, such as the social cognitive theory (Bandura 1989) has highlighted distinct noncognitive, socialemotional characteristics of students that impact on student performance. Wang, Haertel and Walberg (1993) found that students’ affectivemotivational attitudinal disposition was an important factor in influencing studentlearning outcomes based on 91 meta analyses. Bloom’s (1976) student learning model was further supported by voluminous studies that indicate students’ affective characteristics influenced their learning outcomes, including academic selfconcept (e.g., Marsh and Yeung 1997), selfefficacy (e.g., Pajares 1996), anxiety (e.g., Everson et al. 1994), and attitude towards learning (Fantuzzo et al. 2004). Therefore, investigating the affective variables and their relationship with students’ performance in mathematics literacy is undertaken in this study.
On the other hand, multilevel relationships among school and student characteristics and educational outcomes at both school and student levels should be taken into consideration due to the hierarchical structure of PISA dataset (Raudenbush and Bryk 2002). The argument involved is supported theoretically by the multilevel organisation theory (MOT). MOT explains that the interaction process can occur simultaneously at the lower (student level) and higher levels (school level). MOT specifies the relationships between the lower and higher levels by highlighting the topdown processes, referring to the direct effects from a higherlevel unit (school characteristics) to a lowerlevel unit (such as student performance) (Kozlowski and Klein 2000). In the school context, the higherlevel unit is referred to as school factors such as school types at school level whereas the lowerlevel unit refers to an educational outcome such as student performance. In this study, the studentlevel variables refer to the affective variables, whereas the schoollevel variables refer to the aggregated studentlevel variables at school level.
Student and schoollevel variables
Bloom’s (1976) student learning model and MOT are used to derive the student and schoollevel variables that are hypothesised to have relationships with the Malaysian students’ performance in mathematics literacy.
Affective characteristicsrelated measures in PISA 2012 can be categorised into three components: (1) student engagement with and at schools, (2) students’ drive and motivation, and (3) mathematics selfbeliefs, dispositions, and participation in mathematicsrelated activities. Table 1 shows the operational definitions of the measures for each component. For the ease of interpretation, measures of student learning characteristics were capitalised throughout the study. The schoollevel variables are referred to as the aggregates of studentlevel variables.
Sampling design and data sources
PISA used a twostage stratified design sampling (Willms and Smith 2005). In the first stage of sampling, schools having ageeligible students were sampled systematically with probabilities proportional to the school size, which is a function of the number of eligible students enrolled. A minimum of 150 schools was selected in each country. This was followed by randomly selecting a number of students around 15 years of age in the selected schools. Table 2 shows the sample demographic characteristics in Indonesia, Malaysia, and Thailand. The sample size ranged from 5197 to 6606. The gender ratio was fairly equal, except the Thai student sample. The number of Thai female students is higher than male students compared to Indonesia and Malaysia. One of the possible reasons in explaining the difference is due to the fact that most of the Thai male students are studying in the religious schools, such as the Buddhist or Islamic schools (Precharn, public communication, August 4, 2015). The religious schools were not selected in PISA 2012.
The data used for the analysis was retrieved from the official PISA website (http://www.pisa.oecd.org). Each index was then standardised, with the average score across OECD countries set at zero and the standard deviation set at one. A positive value on an index indicates that scores obtained in a particular country were higher than the OECD average, which in turn indicates that students in this country have more positive perceptions on the undertaken affective characteristics compared to the students from other OECD countries, and vice versa for an index with negative value. However, such interpretation needs to be handled with caution, as it is not consistent for all indexes. The interpretation should be relevant in how each undertaken affective characteristic has been operationalised as shown in Table 1. For example, positive value of the index of Mathematics anxiety indicates that students reported higher levels of anxiety towards mathematics than the OECD average and vice versa for the negative value. However, a positive value of high responsibility in failing in mathematics indicates students tend to attribute the responsibility for failure in solving mathematics problems to themselves; a negative value indicates students are more likely to see other individual or factors as responsible to their failure in mathematics (OECD 2013).
HLM analysis procedure
As students are nested within schools, multilevel analysis was conducted to examine the effects of a higherlevel unit such as school factors, and on a lowerlevel unit such as student factors (Hox 2010; Raudenbush and Bryk 2002). In fact, there have been numerous studies that have attempted to ascertain the amount of variance that can be attributed to the school input through multilevel modelling procedures (FitzGibbon and Kochan 2000; Teddlie et al. 2000). In a related vein, hierarchical linear modelling (HLM) is a statistical technique for analysing the hierarchical structure of PISA 2012 database. We used twolevel HLM to examine the relationships between student and schoollevel variables and mathematics performance using HLM 7.0 computer software. HLM is not only able to calculate using plausible values and handling missing values; it can also replicate the analysis across all of the plausible values and compute standard errors of the coefficients based on the full analysis (Willms and Smith 2005). The HLM2 module of HLM 7 program can handle missing data at level1 of the hierarchy. Observations with missing data will be deleted using listwise deletion at either the MDM creation stage or when the analysis is run. HLM assumes level2 data files to be completed. If any of the higherlevel files contain missing data, units with missing data will automatically be deleted when the MDM file is created (Raudenbush et al. 2011). In PISA 2012, the probabilities selection varied although the sample was chosen randomly. Each index was scaled using a weighted maximum likelihood estimate method with its multiple questions and responses. As such, student and school weight variables abbreviated as W_FSTUWT and W_FSCHWT, respectively in the database were incorporated into the analysis to ensure that each sampled student and school represented the correct number of students and schools in the population.
Centering is an important issue in multilevel analysis. According to Raudenbush and Bryk (2002), centering could be presented in the form of grand mean centering and group mean centering. Grand mean centering is achieved by subtracting the sample mean from each student score (\(x_{ij}  \bar{x}\)). Meanwhile, group mean centering is done by subtracting the mean of the school that the student attends from the scores for each student within that school (\(x_{ij}  \bar{x}_{j}\)). In this study, we centred all student and school variables around the grand mean except for gender, where the variable is included in the equation uncentered. The purpose of this is to reduce the multicollinearity among variables and bias in variance estimates so that a more meaningful interpretation can be made (Kreft and de Leeuw 1998).
The multilevel model building begins with a null model. The null model contains only the dependent variable, namely mathematics performance, and no dependent variables except an intercept. The null model is statistically equivalent to oneway random effects analysis of variance (Raudenbush and Bryk 2002). The null model has two purposes. First, to estimate the grand mean of mathematics performance with adjustments for clustering of students within schools and for different sample sizes across schools. Second, to estimate variance components available by decomposing the total variance in mathematics performance into variance attribute to students (within school variance) and variance attribute to schools (between school variance). Practically, the null model serves as the baseline model to compare with the results of the final model. The final model is developed by adding the undertaken student and schoollevel variables to the null model. The final model includes the student and schoollevel variables that demonstrated a statistically significant relationship with mathematics performance. In this study, separate analyses were conducted for the three countries using the above procedures. However, the models for these three countries were identical to maintain their comparability and to enable between countries comparisons to be made.
Results
Table 3 shows the results of the HLM analysis with the null model. Intraclass correlation coefficient (ICC) refers to the ratio between the amount of variance at the school level to the total amount of variance available at both student and school level. Table 3 shows that about 47 % of the total variance in mathematics performance is attributed to schools in Indonesia, about 35 % in Malaysia, and about 58 % in Thailand. The results indicated that variability of average mathematics performance between schools in these three Southeast Asian countries was sizeable. The results confirmed the need to investigate the relationships between student and schoollevel variables, and mathematics performance for each country using a multilevel approach. Building on the null model, the studentlevel variables were added into the null model. The nonsignificant studentlevel variable with the highest p value was first excluded using the backward elimination approach. The analysis was repeated until all nonsignificant studentlevel variables were excluded from the analysis. The same procedure was repeated for the schoollevel variables.
Indonesia
Table 4 shows that mathematics selfefficacy (MATHEFF) and attitudes towards learning outcomes (ATTSHCL) had statistically significant and positive effects on mathematics performance at the student level in Indonesia. MATHEFF was the strongest predictor of mathematics performance with one standard deviation increase in MATHEFF was associated with an increase in mathematics performance of about 13 points after controlling all other variables. Meanwhile, one standard deviation increase in ATTSCHL was associated with an increase in mathematics performance of about five points when controlling all other variables. The other three studentlevel variables: Anxiety in mathematics (ANXMAT), mathematics selfconcept (SCMAT), and mathematics behaviour (MATBEH) were found negatively related to mathematics performance. One point standard deviation increase in ANXMAT was associated with a decrease in mathematics performance of about 10 points. This was followed by one standard deviation increase in SCMAT and MATBEH, which were associated with a decrease in mathematics performance of about eight and five points, respectively after controlling other variables.
Variables that start with the capital “S” represent the schoollevel variables (aggregated from studentlevel variables),
At the school level, school average mathematics selfefficacy (SMATHEFF) is the strongest predictor that is positively associated with mathematics performance. One standard deviation increase in SMATHEFF was associated with an increase in mathematics performance of about 61 points. This indicated that schools with high mathematics selfefficacy have better average performances in mathematics compared to schools with lower mathematics selfefficacy. School average sense of belonging (SBELONG) and school average openness to problemsolving (SOPENPS) were also found to be positively associated with mathematics performance, with one standard deviation increase in SBELONG and SOPENPS being associated with an increase in mathematics performance of about 30 and 23 points, respectively. However, school average mathematics interest (SINTMAT) was found to be negatively associated with mathematics performance, indicated by one standard deviation increase in SINTMAT, and associated with a decrease of mathematics performance of about 66 points. Similarly, school average mathematics selfconcept (SSCMAT) and school average mathematics behaviour (SMATHBEH) were found to be negatively associated with mathematics performance. One standard deviation increase in SSCMAT was associated with a decrease in mathematics performance of about 36 points, whereas one standard deviation increase in SMATBEH was associated with a decrease in mathematics performance of about 26 points. Relative to the null model, the final model explained about 44 % of variance at the school level and 7 % of variance at the student level.
Malaysia
Similarly to Indonesia, MATHEFF was the strongest predictor of mathematics performance in Malaysia. One standard deviation increase in MATHEFF was associated with an increase in mathematics performance of about 25 points after controlling all other variables. This was followed by ATTSCHL and instrumental motivation (INSTMOT). One standard deviation increase in ATTSCHL and INSTMOT contributed to an increase in mathematics performance of about nine and eight points, respectively. However, MATBEH and BELONG were found to be negatively associated with mathematics performance in Malaysia. One standard deviation increase in MATHBEH and BELONG was associated with a decrease in mathematics performance of about 10 and 6 points, respectively after controlling other variables. It is worthy to highlight that three schoollevel variables: SMATBEH, school average instrumental motivation (SINSTMOT), and school average attitude towards learning outcomes (SATTSCHL) were also found to be negatively associated with mathematics performance in Malaysia. One standard deviation increase in SCMATBEH was associated with a decrease in mathematics performance of about 55 points after controlling all other variables. Similarly, one standard deviation increase in SINSTMOT and SATTSCHL was associated with decreases in mathematics performance of about 34 and 33 points, respectively. Overall, the final model explained about 58 % of variance at the school level and 17 % of variance at the student level in Malaysia.
Thailand
Thailand PISA 2012 data informed that perseverance (PERSEV) was the strongest predictor of mathematics performance at the student level. One standard deviation increase in PERSEV was associated with an increase in mathematics performance of about 10 points. One standard deviation increase in ATTSCHL contributes about eight points of increase in mathematics performance. Results revealed a negative relationship between mathematics interest (INTMAT) and mathematics performance, as well as ATTLNACT and mathematics performance in Thailand. One standard deviation increase in INTMAT and ATTLNACT was associated with a decrease in mathematics of about seven and five points, respectively. On the other hand, a negative relationship was also found between FAILMAT and mathematics performance, with one standard deviation increase in FAILMAT associated with a decrease in mathematics performance of about nine points. Although not significant at the student level, the school average openness to problemsolving (SOPENPS) was found to be positively associated with mathematics performance in Thailand. One standard deviation increase in SOPENPS was associated with an increase in mathematics performance of about 38 points. SATTSCHL was also found to be positively associated with mathematics performance with a onepoint standard deviation increase in SATTSCHL as associated with an increase in mathematics performance of about 27 points. Compared to Indonesia and Malaysia, the proportions of variance explained at the school level and student level are not significantly different to Thailand, with 25 % of variance at the school level and 16 % of variance at the student level. However, the variance explained at student level in the Thai model was found more than twice the variance explained in the Indonesian model. The results informed that the influence of the significant studentlevel variables on students’ performance in mathematics literacy in the Thai model was stronger when compared to Indonesia. In addition, the variance explained at school level in the Thai model was found to be less than half when compared to the Malaysian model. The results indicate that the influence of the significant schoollevel variables on mathematics literacy in Thailand is smaller when compared to Malaysia.
Conclusion and discussion
The results of Programme for International Student Assessment (PISA) 2012 showed that Indonesia, Malaysia, and Thailand underperformed and were positioned in the bottom third out of 65 participating countries for mathematics, science, and reading literacies. The wide gap between these three countries and the top performing countries in the performance of PISA 2012 has prompted this study to examine the influence of students’ affective characteristics on their performance in mathematics literacy using a multilevel analysis.
This study has provided empirical evidence on how affective characteristics, in terms of social and emotional factors, are associated with the PISA 2012 mathematics performance in Indonesia, Malaysia, and Thailand. In essence, the inconclusive results of the multilevel analysis have demonstrated some interesting points for discussion.
The results could be seen with some contradictory interpretations. In Indonesian context, the lesser the students participated in mathematicsrelated activities and the lesser they perceived the importance of schools to their future career, the better the performance they reached. Similarly, in Malaysian context, the more the students tend to thrive when they form positive relationships with peers, feel happy and at ease at school, the lower the performance they reached; and the more the students participated in mathematicsrelated activities at or outside of school, the lower the performance they reached. Meanwhile, in the Thai context, the lesser the students enjoy and work hard in mathematics, the better their performance. The results provide insight in regards to the relevancy of the items, and fully capture the conceptual meaning of the undertaken affective variables in the Indonesian, Malaysian, and Thai context due to the possibility of item translation that bias from its original item. This is because the same question been translated into different languages could have generated different meaning depending on the different cultural background (SolanoFlores et al. 2009). In addition, such contradictory interpretation could be explained, with possible reason that competencies tested by PISA and referred to by the variables do not correspond well with content covered by teaching in these three countries.
The inconclusive results appeared at both student and school level. At the student level, attitudes towards learning outcomes was the only common variable that predicted increased scores for all the country models. Mathematics selfefficacy predicted increased scores in Indonesia and Malaysia, but no significant increase in the Thai model. Instrumental motivation predicted increased scores in Malaysia, but was not significant in the Indonesian and Thai models. Meanwhile, mathematics behaviour predicted decreased scores in the Indonesian and Malaysian models only. On the other hand, attitudes towards learning activities predicted decreased scores in the Indonesian and Thai models only. Selfresponsibility in failing mathematics is only significant in the Thai model with predicted decrease in scores. Mathematics selfconcept predicted a decrease in scores for the Indonesian model only. At the school level, the Indonesian model has the highest number of significant predictors of mathematics performance. School average of sense of belonging, openness to problemsolving, and mathematics selfefficacy predicted increase in scores, whereas school average mathematics behaviour, mathematics selfconcept, and intrinsic motivation predicted decrease in scores. Meanwhile, school average attitudes towards learning activities and openness to problemsolving were the only two schoollevel variables with a predicted increase in scores in the Thai model. However, school average mathematics interest, mathematics behaviour, and attitudes towards learning outcomes were the only three significant schoollevel variables with predicted decreases in scores for the Malaysian model. The findings also revealed that Malaysia has the highest school variance, followed by Indonesia and Thailand. This indicates that the effects of school traits on variation in students’ mathematics performance are much greater in Malaysia than Indonesia and Thailand.
Clearly, the different relationship pattern between each undertaken student and schoollevel variables and mathematics performance were identified. The cultural differences in the three countries might serve as a possible reason in explaining the different interpretations of the undertaken affective variables at both the student and school levels. According to Klieme (2013), cultural differences in education systems and school atmospheres may contribute to explaining the different patterns of relationships between students, school factors, and mathematics performance among these three countries. This is because the psychological characteristics of individuals are influenced by the cultures in which they live (Triandis 1995; Tweed and Lehman 2002). In this study, affective characteristics are not an exception. As such, the influence of sociopsychological factors on affective characteristics should be understood in consideration of cultures and contexts. In addition, the different medium of instruction used in teaching the mathematics subject in Malaysia, Thailand, and Indonesia could explain such results. The medium of instruction is the Thai language in Thailand, Indonesian language for Indonesia, and Malay or English language for Malaysia.
These hypothetical explanations of the results in the study, however, should be examined empirically or qualitatively in the followup research. Our understanding of affective characteristics based on multicultural perspective will be enhanced through a deeper investigation. Subsequently, such comprehensive understanding would serve for identifying ways of changing educational practices and systems for better educational outcomes. On the other hand, it is important to highlight that this study could also have not considered or included other affectiverelated variables which might associate with mathematics performance.
For policy implication, findings from this current study shed light on the importance of teachers to incorporate significant affective components into their classroom teaching or intervention programs in order to improve and enhance students’ understanding in mathematics. For instance, the Indonesian mathematics teachers should strike to strengthen students’ attitudes toward learning outcomes and selfefficacy in their teaching, as these affective factors strongly influence their mathematics performance. Meanwhile, the Malaysian teachers should enhance students’ instrumental motivation, their attitudes towards learning outcomes, selfefficacy, students’ perseverance; and Thai teachers should enhance their students’ attitudes towards learning outcomes. The results further suggest that there is room for Indonesian and Thai teachers to foster student learning by promoting selfconfidence and connectedness to school and peers in mathematics classroom.
The findings further inform the challenges faced by the education stakeholders in the Southeast Asian societies that might need to strike a balance between the cognitive and affective domain of students’ mathematics learning. For the implication on research, the findings inform the needs for researchers to review and develop more contextbased items that able to capture the conceptual meaning of the undertaken variables based on the three countries context. More investigation on other factors that might impact on mathematics performance are needed especially in Indonesian, Malaysian, and Thai context.
This study is not without limitations. The questionnaire items of affective characteristics in PISA 2012 might be too general and therefore might not fully capture students’ perception of their learning characteristics. In addition, translating items creates the potential to lose integrity in the equivalency of what is being asked from one country to other country (Ram 2007). Affective characteristics and aggregated school variables in this study only accounted for part of the variances in mathematics performance. Students’ mathematics performance depends on multiple factors that contribute to a large proportion of unexplained variance. For instance, teachers’ instructional practices (O’Dwyer et al. 2015), socioeconomic composition of the schools (Rumberger and Palardy 2005) as well as student and school resources (Topҫu et al. 2014) within each of these three Southeast Asian countries should be examined in future studies.
Similar analysis could be conducted using data in future cycles of PISA or from other international studies such as trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) to crossvalidate the findings. In relation to this, a number of questions are worthwhile for future studies: Why do students in Indonesia, Malaysia, and Thailand have a low level of performance but high levels of perseverance, mathematics interest, instrumental motivation, and mathematics behaviour? Why do the three Southeast Asian schools engage students’ bodies but not their hearts? What are the possible ways to improve mathematics performance from a cultural perspective?
Overall, this study represents a multilevel analysis in examining the relationships between the student and school level affective variables and mathematics performance in PISA 2012 across Indonesia, Malaysia, and Thailand. This study contributes to provide evidencebased policy making. Practically, the findings inform direction for future studies in providing empirical evidence on how students’ affective characteristics impact on their mathematics performance in the three countries. It is hoped that this study serves as not only a reference to provide knowledge and empirical evidence for the researchers, but aid in gaining a better understanding of the effects of policies on education outcomes.
References
Bandura, A. (1989). Social cognitive theory. In R. Vasta (Ed.), Annals of child development: six theories of child development (Vol. 6, pp. 1–60). Greenwich: JAI Press.
Bandura, A. (1997). Selfefficacy: the exercise of control. NY: Freeman.
Barth, J. M., Dunlap, S. T., Dane, H., Lochman, J. E., & Wells, K. C. (2004). Classroom environment influences on aggression, peer relations, and academic focus. Journal of School Psychology, 42(2), 115–133.
Bloom, B. S. (1976). Human characteristics and school learning. New York: McGraw Hill.
Chow, T. C. F. (2011). Students’ difficulties, conceptions and attitudes towards learning algebra: an intervention study to improve teaching and learning (Unpublished doctoral dissertation). Australia: Curtin University.
Everson, H., Smodlaka, L., & Tobias, S. (1994). Exploring the relationship of test anxiety and metacognition on reading test performance: a cognitive analysis. Anxiety, Stress and Coping, 7, 85–96.
Fantuzzo, J., Perry, M., & McDermott, P. (2004). Preschool approaches to learning and their relationship to other relevant classroom competencies for lowincome children. School Psychology Quarterly, 19, 212–230.
FitzGibbon, C., & Kochan, S. (2000). School effectiveness and education indicators. In C. Teddlie & D. Reynolds (Eds.), The international handbook of school effectiveness research (pp. 257–282). London: Falmer Press.
Grootenboer, P., & Hemmings, B. (2007). Mathematics performance and the role played by affective and background factors. Mathematics Education Research Journal, 19(3), 3–20.
Hattie, J. A. C. (2009). Visible learning: a synthesis of over 800 metaanalyses relating to performance. New York: Routledge.
Hau, K. T., & Salili, F. (1991). Structure and semantic differential placement of specific causes: academic causal attributions by Chinese students in Hong Kong. International Journal of Psychology, 26(2), 175–193.
Hox, J. J. (2010). Multilevel analysis: techniques and applications. New York: Routledge.
Jimenez, E., Nguyen, V., Patrinos, H. A. (2012). Stuck in the middle? Human capital development and economic growth in Malaysia and Thailand (No. 6283). Policy Research Working Study. doi:10.1596/181394506283.
Kim, C. M., Park, S. W., & Cozart, J. (2014). Affective and motivational factors of learning in online mathematics courses. British Journal of Educational Technology, 45(1), 171–185.
Klieme, E. (2013). The role of large scale assessment in research on educational effectiveness and school development. In M. von Davier, E. Gonzalez, I. Kirsch, & K. Yamamoto (Eds.), The role of international largescale assessments: perspectives from technology, economy, and educational research (p. 115–147). New York: Springer.
Kozlowski, S. W., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, methods in organizations (pp. 3–90). San Francisco: JosseyBass.
Kreft, I., & de Leeuw, J. D. (1998). Introducing multilevel modelling. London: Sage Publications.
Kumar, D. D., & Morris, J. D. (2005). Predicting scientific understanding of prospective elementary teachers: role of gender, education level, courses in science, and attitudes toward science and mathematics. Journal of Science Education and Technology, 14(4), 387–391.
Lee, W. O. (1996). The cultural context for Chinese learners: conceptions of learning in the Confucian tradition. In D. A. Watkins & J. B. Biggs (Eds.), The Chinese learners (pp. 45–67). Hong Kong: Comparative Education Research Centre, University of Hong Kong.
Li, J. (2004). A Chinese culture model of learning. In L. Fan, N. Y. Wong, J. Cai, & S. Li (Eds.), How Chinese learn mathematics: perspectives from insiders (pp. 124–156). Singapore: World Scientific.
Ma, X., & Xu, J. (2004). The causal ordering of mathematics anxiety and mathematics performance: a longitudinal panel analysis. Journal of Adolescence, 27(2), 165–179.
Marsh, H. W. (1987). The bigfishlittlepond effect on academic selfconcept. Journal of Educational Psychology, 79(3), 280–295.
Marsh, H. W., & Yeung, A. S. (1997). Causal effects of academic selfconcept on academic achievement: structural equation models on longitudinal data. Journal of Educational Psychology, 89, 41–54.
McCoach, D. B., Gable, R. K., & Madura, J. P. (2013). Instrument development in the affective domain. New York: Springer.
Ministry of Education, MOE. (2013). Malaysian education blueprint 2013–2025: preschool to postsecondary education. Putrajaya: MOE.
O’Connor, M. C., & Paunonen, S. V. (2007). Big five personality predictors of postsecondary academic performance. Personality and Individual Differences, 43(5), 971–990.
O’Dwyer, L. M., Wang, Y., & Shields, K. A. (2015). Teaching for conceptual understanding: a crossnational comparison of the relationship between teachers’ instructional practices and student achievement in mathematics. Largescale Assessment in Education. doi:10.1186/s4053601400116.
OECD. (2013). PISA 2012 results: ready to learn: students’ engagement, drive and self beliefs (Vol. III). Paris: OECD Publishing. doi:10.1787/9789264201170en.
OECD. (2014). PISA 2012 results: what makes schools successful? Resources, policies and practices (Vol. IV). Paris: OECD Publishing.
Pajares, F. (1996). Selfefficacy beliefs in academic settings. Review of Educational Research, 66, 543–578.
Prensel, M., Kobarg, M., Schöps, K., & Rönnebeck, S. (2013). Research on PISA: research outcomes of the PISA research conference 2009. New York: Springer.
Ram, A. (2007). A multilevel analysis of mathematics literacy in Canada and Japan: the effects of sex differences, teacher support, and the school learning environment (Unpublished master thesis). Victoria, British Columbia, Canada: University of Victoria.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: applications and data analysis methods (2nd ed.). Thousand Oaks: Sage.
Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., Congdon, R. T., & Toit, Md. (2011). HLM 7: hierarchical linear and nonlinear modelling. Lincolnwood: Scientific Software International Inc.
Reyes, L. H. (1984). Affective variables and mathematics education. The Elementary School Journal, 84(5), 558–581.
Rumberger, R. W., & Palardy, G. J. (2005). Does segregation still matter? The impact of student composition on academic performance in high school. Teachers College Record, 107(9), 1999–2045.
Salili, F., Chiu, C. Y., & Lai, S. (2001). The influence of culture and context on students’ motivational orientation and performance. In F. Salili, C. Y. Chiu, & Y. Y. Hong (Eds.), Student motivation: the culture and context of learning (pp. 221–247). New York: Plenum.
Schneider, B., & Lee, Y. (1990). A model for academic success: the school and home environment of East Asian Students. Anthropology and Education Quarterly, 21, 358–377.
SolanoFlores, G., Backhoff, E., & ContrerasNino, L. A. (2009). Theory of test translation error. International Journal of Testing, 9(2), 78–91.
Stevenson, H. W., & Stigler, J. W. (1992). The learning gap: why our schools are failing and what we can learn from Japanese and Chinese education?. New York: Simon and Schuster.
Teddlie, C., Reynolds, D., & Sammons, P. (2000). The methodology and scientific properties of school effectiveness research. In C. Teddlie & D. Reynolds (Eds.), The international handbook of school effectiveness research (pp. 55–133). London: Falmer Press.
Topҫu, M. S., Arikan, S., & Erbilgin, E. (2014). Turkish students’ science performance and related factors in PISA 2006 and 2009. The Australian Educational Researcher. doi:10.1007/s1338401401579.
Triandis, H. C. (1995). Individualism and collectivism. Boulder: Westview.
Tweed, R. G., & Lehman, D. R. (2002). Learning considered within a cultural context: confusion and Socratic approaches. American Psychologist, 57, 89–99.
UNDP (United Nations Development Programme). (2014). Human development report, 2012 sustaining human progress: reducing vulnerabilities and building resilience. http://www.undp.org/.
Wang, M. C., Haertal, G. D., & Walberg, H. J. (1993). Toward a knowledge base for school learning. Review of Educational Research, 63, 249–294.
Watkins, D. A., & Biggs, J. (Eds.). (1996). The Chinese learner: cultural, psychological, and contextual influences. Hong Kong: Comparative Education Research Centre, University of Hong Kong.
Wilkins, J. L. M., & Ma, X. (2003). Modeling change in student attitude toward and beliefs about mathematics. The Journal of Educational Research, 97(1), 52–63.
Willms, J. D., Smith, T. M. (2005). A manual for conducting analyses with data from TIMSS and PISA. Montreal, Canada: United Nations Educational, Scientific, and Cultural Organization Institute for Statistics (UIS).
Wong, N. Y. (1993). The relationship among mathematics performance, affective variables and home background. Mathematics Education Research Journal, 4(3), 32–42.
Wong, N. Y. (2004). The CHC learner’s phenomenon: Its implications on mathematics education. In L. Fan, N. Y. Wong, J. Cai, & S. Li (Eds.), How Chinese learn mathematics: perspectives from insiders (pp. 503–534). Singapore: World Scientific.
Yao, E. (1985). A comparison of family characteristics of AsianAmerican and AngloAmerican high achievers. International Journal of Comparative Sociology, 26(3–4), 198–208.
Zan, R., Brown, L., Evans, J., & Hannula, M. S. (2006). Affect in mathematics education: an introduction. Educational Studies in Mathematics, 63(2), 113–121.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Thien, L.M., Darmawan, I.G.N. & Ong, M.Y. Affective characteristics and mathematics performance in Indonesia, Malaysia, and Thailand: what can PISA 2012 data tell us?. Largescale Assess Educ 3, 3 (2015). https://doi.org/10.1186/s405360150013z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s405360150013z
Keywords
 Affective characteristics
 Hierarchical linear modeling (HLM)
 Mathematics performance
 Programme for International Student Assessment (PISA)
 Southeast Asian countries