The relationship among personal achievement motives, school relational goal structures and learning outcomes: a multilevel analysis with PISA 2018 data

Abstract

The influence of achievement motives and relational goal structures embedded in schools on learning outcomes as well as their interaction has attracted much attention in educational research. In the secondary multi-level analysis of data from four areas of China (B–S–J–Z) and the United States in the PISA 2018 study, the relationship among students’ personal achievement motives (competitiveness, work mastery and fear of failure), competitive or cooperative goal structures in schools, reading achievement and reading self-concept was examined. Results revealed similar relationship across countries: both competitiveness and work mastery motives had positive influence on reading self-concept; competitive goal structure influenced reading achievement. Cross-level interactions were only observed in the Chinese (B–S–J–Z) sample, with the competitive goal structure negatively moderating the relationship between work mastery motive and learning outcomes as well as the relationship between competitiveness motive and self-concept. Implications for theory and educational practice were discussed.

Introduction

Achievement goals/motives and goal structures embedded in learning environment have attracted much attention in educational research (Roseth, Johnson, & Johnson, 2008; Bardach, Oczion, Pietschnig, & Lüftenegger, 2020; Senko, Hulleman, Harackiewicz, 2011; Dweck & Leggett, 1988; Elliot, 2005; Ames, 1992; Maehr & Midgley, 1991). Besides the importance of personal achievement goals/motives, the focus was also on the impact of goal structures on personal goals/motives or the interaction among personal goals/motives and goal structures (Roseth, Johnson, & Johnson, 2008; Bardach, et al., 2020). There are still open questions regarding the roles of personal motives/goals and goal structures in promoting achievement related outcomes.

In the PISA (Programme for International Student Assessment) 2018 project, constructs such as achievement motives and perception of relational goal structures in schools were measured. It provides an opportunity to investigate the relationships among goal structures, personal achievement motives, and achievement related outcomes in the framework of a large-scale assessment and with samples from different cultures or educational systems. Specifically, this paper aims at understanding how the Chinese (B–S–J–Z) and U.S. students’ reading performance and self-concept were influenced by their achievement motives and the school focus on competition and cooperation (relational goal structures). China (B–S–J–Z) and the United States were selected, for they are countries representing collectivistic and individualistic cultures. Previous research also indicated the discrepancy between Asian students and students in western countries regarding the relationship between achievement motivation and learning outcomes (Pintrich, 2003; Khajavy et al., 2018; Bardach, Oczion, Pietschnig, & Lüftenegger, 2020).

Personal achievement motives

Achievement motives were defined as stable dispositions rooted in personality (Diseth & Martinsen, 2003) and the latent capacity to anticipate affect in different situations (Christophersen & Rand, 1982). It was differentiated between the motive for success and the motive to avoid failure (Atkinson & Feather, 1966; Gjesme, 1983; Heckhausen, 1986) and the motive for success was further differentiated between competitiveness motive and work mastery motive (Helmreich et al., 1978; Elliot & McGregor, 2001). Achievement motives were assessed in PISA 2018 with 3 constructs: competitiveness, work mastery motive, and fear of failure (OECD, 2019a, 2019b). Competitiveness was defined as the dispositional motive to outperform others, while work mastery was conceptualized as the dispositional desire to master tasks (Elliot & McGregor, 2001; Helmreich & Spence, 1978). Fear of failure is a general tendency to avoid potential mistakes and failures and was applied in PISA as a dispositional variable for avoidance motivation (OECD, 2019a, 2019b).

The dimensions were similar to the three dimensions of achievement goals (Wolters, 2004): performance-approach (demonstrating their ability relative to others), mastery-approach (attainment of success in competence judgment) and performance-avoidance goals (avoiding looking incompetent in comparison with others). Unlike achievement motives, which are measured on a more general level and represent the dispositional traits of the individuals (Elliot, Conroy, Barron & Murayama, 2010), achievement goals were defined as cognitive representations individuals have for directing their behaviour and pursuing a specific end status (Elliot & McGregor, 2001).

Although achievement goals and motives are labelled differently and measure motivation from different perspectives, research showed that they had close relationship with each other (Diseth & Kobbeltvedt, 2010; Elliot & Church, 1997) and had similar effect on achievement related outcomes. General findings from ample research about their influence on learning and achievement showed that mastery motive is associated with adaptive outcomes while competiveness motive might be correlated with maladaptive behaviour but higher achievement (OECD, 2019a, 2019b; Busato, Prins, Elshout, & Hamaker, 2000). Fear of failure was correlated with self-protective behaviour thus can have negative effect on learning (Covington, 1992; Kaye et al., 2008). In the current study we use “motive” instead of “goal” in the following sections to represent the three motivational variables in the PISA study, nevertheless the research findings from achievement goal theory were also included in the review.

Relational goal structures

Students’ personal motives might be influenced by the contextual factors such as the goal or value that emphasized by teachers or schools in early or middle adolescence, which are referred as goal structures (Meece, Anderman, & Anderman, 2006a; Roseth, Johnson, & Johnson, 2008). Goal structures can be considered as contextual (classroom goal structure) or relational variables (relational goal structure). Classroom goal structure has been defined as salient messages embedded in the learning environment which has influence on personal goals and behaviours (Kaplan et al., 2002); while relational goal structures describe the social interdependency or shared common goals among students in the learning context (Johnson & Johnson, 1974; Roseth, Johnson, & Johnson, 2008). In the latter perspective, distinctions among cooperative, competitive and individualistic goal structures were made (Deutsch, 1949; Johnson & Johnson, 1989, 2005). Positive interdependence (i.e. cooperative goal structures) happens when students perceive that they can reach their goals cooperatively with others; negative interdependence (i.e., competitive goal structures) exists when students perceive that they can obtain their goals when others fail to obtain theirs. It could also be the case of no interdependence (i.e., individualistic goal structures), when students perceive that they can reach their goal regardless of others’ attainment of goals. The perception of cooperation or competition in schools is not only related to student level variables such as personal goals or experiences, but also related to classroom- or school-level variables such as teachers’ teaching practice in grouping, group work and evaluation (Roseth, Johnson, & Johnson, 2008). Based on the results of the meta-analysis (Roseth, Johnson, & Johnson, 2008), researchers concluded that the relational goal structures might be more proximal predictors for achievement than classroom goal structures.

In the PISA 2018 study, two constructs were parallel assessed which belong to the module of assessment, evaluation and accountability of school environment (OECD, 2019a, 2019b): perception of competiveness and cooperation in schools. Although they were scales of student perception, the items can actually reflect the school climate (see “Appendix”). They were generally mentioned as “school climate” in the OECD report (OECD, 2019a, 2019b). In the current study they were considered as indices for competitive and cooperative goal structures.

Relationship among student achievement motives and school goal structures

Personal and environmental factors interact with each other and work together to influence learning motivation, strategies and achievement related outcomes. It is of importance to know whether the goal structures influence personal motives and how this “match” or “mismatch” situation influences learning outcomes (Murayama & Elliot, 2009).

Most studies related to the interaction between personal achievement goals and goal structures were based on the achievement goal theory and the according classroom goal structures (Lau & Nie, 2008; Schwinger & Stiensmeier-Pelster, 2011; Murayama & Elliot, 2009). There were generally positive relationships among achievement motives and the corresponding goal structures according to a meta-analysis (Bardach, Oczion, Pietschnig, & Lüftenegger, 2020): between mastery approach goal structure and students’ personal mastery approach goal as well as performance goal structure and the according personal performance goal. However, they also found the “cross-relationships”: positive association between performance approach goal structure and performance avoidance goal as well as between performance goal structure and mastery approach goal (Bardach, Oczion, Pietschnig, & Lüftenegger, 2020; Kim, Schallert, & Kim, 2010). Achievement motives might be relatively stable in comparison with achievement goals but the results can be similar.

Furthermore, since the competitive motive indicates the focus on comparing with others and demonstrating own ability, it is meaningful to include the social perspective (relationship with others) into consideration. Collaborative goal structure was not investigated in the above mentioned research, however it might reduce the emphasis on the comparison with others thus abate the competitiveness motive. It was also considered as one element of mastery goal structure (Meece, Anderman, & Anderman, ). In the TARGET (Task, Authority, Recognition, Grouping, Evaluation, and Timing) system, a system emphasizing mastery goal structure, the instructional practice promoting peer collaboration was also included as an important dimension (Ames, 1992). Moreover, a longitudinal study showed that shared collaborative goal structure can positively influence individual mastery oriented goals and negatively influence individual performance-avoidance goals (Summers, 2006). In summary, it can be hypothesized that the competition focus in schools is positively associated with competitiveness motive and fear of failure, collaboration focus is positively related to work mastery motive and negatively related to fear of failure, and there can be cross-relationships.

Influence of achievement motives and relational goal structures on achievement and academic self-concept

Personal mastery motives were generally proved to be related to adaptive motivational, emotional, and behavioural outcomes, such as deep-learning strategies (Greene, Miller, Crowson, Duke, & Akey, 2004; Wolters, 2004), higher learning motivation and self-efficacy, and positive attitude towards school and general wellbeing (Ames, 1992; Urdan, 1997). Findings regarding the effect of competitiveness motives are mixed. They might have positive influence on achievement (Harackiewicz et al., 2002) but were also related to maladaptive behaviour such as self-handicapping (Midgley et al., 1996; Senko & Dawson, 2017). Fear of failure or avoidance motives were generally considered maladaptive and correlated with low performance and low motivation (e.g., Church, Elliot, & Gabel, 2001; Wolters, 2004). Research based on multiple goal perspective provided evidence that students with high mastery and high performance goals have higher achievement; students with dominant mastery goals tend to have lower level of achievement than the multiple-goal group (Pintrich, 2000).

Regarding the effect of relational goal structures, empirical evidence generally showed that cooperative goal structure was related to positive outcomes in comparison with competitive goal structure: it was positively related to intrinsic motivation (Nicholls, 1996) and mastery goals (Summers, 2006), could reduce performance-avoidance goals (Summers, 2006), had a closer relationship with achievement and peer relationship, and enhanced the positive interaction between achievement and peer relationship (Roseth, Johnson, & Johnson, 2008). Competitive goal structure was related to more disruptive behaviour (Kaplan, Gheen, & Midgley, 2002), less help-seeking and less self-efficacy (Urdan, Midgley, & Anderman, 1998). There was however also evidence that competitive goal structure could entail the most comprehensive classroom communication, thus yields better achievement (Vu et al., 2021).

Regarding the interaction and its influence on learning there were different hypotheses proposed. For example, the “match” theory proposed that optimal learning outcomes would be expected when there is congruence between personal goals and goal structures of the learning environment (Bretz & Judge, 1994; Eccles et al., 1993; Higgins, 2000; Vansteenkiste et al., 2004). The “mismatch” theory provided different combinations and possible effects on learning: the influence of the personal motives would be vitiated (e.g., personal mastery goals may have a weaker positive influence on outcomes in the context of a performance-based goal structure), or mitigated (e.g., personal performance-avoidance goals may not be as problematic in the context of a mastery goal structure), or exacerbated (e.g., personal performance-avoidance goals may have a particularly negative influence on outcomes in the context of a performance-approach goal structure) by a “mismatched” goal structure (Murayama & Elliot, 2009). In Murayama and Elliot’s study (2009), they have found cross-level interactions in predicting math self-concept with Japanese students: personal performance-avoidance goal was a negative predictor of math self-concept, and this relation was particularly strong in classrooms with a high performance-approach goal structure; similar effect was also observed by the relationship among performance-approach goals, goal structure and math self-concept.

In PISA 2018 project, different outcome variables were measured. Since reading is the focus of this test cycle, reading performance and reading self-concept were included in the analysis as important achievement related outcomes. Based on the previous research it is of our interest whether they can be influenced by personal motives, relational goal structures as well as their different types of cross-level interactions.

Motivational research across cultures

Although culture was considered important in motivational research (Pintrich, 2003), the cultural difference regarding achievement motives and goal structures was underresearched. General findings in previous literature showed that the effect of pursuing mastery goals is relatively consistent across cultural and ethnic groups (King et al., 2017); Asian students might be more influenced by performance goals (Grand & Dweck, 2001); fear of failure might also play different roles for Asian students. Several studies pointed to the advantages of avoidance motives for students in (more) collectivist cultural contexts (e. g., Khajavy et al., 2018). As for goal structures, students from Asian countries or with Asian background might value collaboration instead of competition due to the collectivist culture (Bardach, Oczion, Pietschnig, & Lüftenegger, 2020). In the meta-analysis focusing on the interaction among personal achievement goals and goal structures, world region or culture worked also as a moderator. In this study, China (B–S–J–Z) and the United States were selected in order to represent collectivism and individualism cultures, which were proved to be influential regarding the relationship between motivation and learning (Bardach, Oczion, Pietschnig, & Lüftenegger, 2020). They were also countries from East Asia and North America thus can represent somehow the different world regions.

The present study

Study aims

The present study aims to investigate the relationship among students’ personal achievement motives, school relational goal structures, and learning outcomes (reading achievement and reading self-concept). The data from China (B–S–J–Z) and the United States was included in the analysis to check the potential differences between different cultures/school systems.

Research questions

The following research questions were proposed:

• 1. What is the relationship among personal achievement motives (competitiveness, work mastery, fear of failure) and relational goal structures (school focus on competition or cooperation)?

• 2. Can students’ personal achievement motives (competitiveness, work mastery, fear of failure) explain the variance of their reading achievement and reading self-concept?

• 3. Can students’ perceived relational goal structures at school level explain the variance of their reading achievement and self-concept?

• 4. Can the relational goal structures moderate the relationship between one of the personal achievement motives and reading achievement or self-concept?

• 5. Are there differences between China (B–S–J–Z) and the United States regarding the relationship among personal achievement motives, relational goal structures and learning outcomes (reading achievement and self-concept)?

Methods

PISA 2018 survey

The present study is a secondary analysis of the student data collected in the PISA 2018 project, the international student assessment that aims to provide information about the ability of 15-year-old students to face the challenges in their future lives. In PISA 2018, the focus is on reading literacy and 84 countries and economies participated in the study.

PISA has applied the complex sample and assessment design such as stratification, two-stage clustered sampling and rotated booklet design of tests (OECD, 2009). Multiple imputation approach and plausible value methods were employed. Ten plausible values were used as proxy for students’ performance in PISA 2018 (OECD, 2018). Besides, scales were constructed from items using IRT scaling method, specifically the method using the generalized partial credit model (OECD, 2018).

Analysis method and data preparation

The data was directly downloaded from the public user files on the OECD website and data preparation was completed in R (such as country and variable selection, school track combination, data format transformation, etc.). Missing data was excluded in the analysis. Imputation was not applied for the multi-level analysis, for the percentages of missing value from Chinese (B–S–J–Z) and U.S. samples were less than 15% and imputation would not be necessary (Arbuckle, 1996). R package “intsvy” was applied for conducting descriptive and correlational analysis with consideration of weights and plausible values. Aggregated variables of relational goal structures at school level were group means of the perception variables “PERCOMP” and “PERCOOP”. Intraclass correlation coefficient ICC1 was applied to compute the group-level variance and ICC2 was calculated to check the reliability of group-level variables (Shrout & Fleiss, 1979). ICC2 higher than .4 was considered acceptable in this research (Fleiss, 2011). The R package “misty” was applied for calculating ICC1 and ICC2.

The multi-level regressions were conducted in Mplus 8 (Muthén & Muthén, 2017). The cluster and stratification options in Mplus were employed and only school-level weight was applied for the estimation based on the simulation results from Mang and her colleagues (Mang et al., 2021). The data imputation command in Mplus enables estimation with 10 plausible values for reading achievement avoiding bias in standard errors and χ² estimates (Muthén, 2013; Rutkowski et al., 2010).

In order to check the cross-level interactions, the student-level motivation predictors were group-mean centered and the school-level goal structure variables were grand-mean centered. The following random-slop models were chosen for checking the cross-level interactions:

$$\begin{aligned} & {\text{Level}}\,{1}\,\left( {\text{student - level}} \right):\,{\text{Y}}_{{{\text{ij}}}} = \, \beta_{{0{\text{j}}}} + \, \beta_{{{\text{1j}}}} \left( {{\text{X}}_{{{\text{ij}}}} - {\text{ X}}_{{\text{j}}} } \right) \, + {\text{ r}}_{{{\text{ij}}}} \\ & {\text{Level}}\,{2}\,\left( {\text{school - level}} \right):\,\beta_{{0{\text{j}}}} = \, \gamma_{00} + \, \gamma_{{0{1}}} \left( {{\text{W}}_{{\text{j}}} - {\text{W }}} \right) \, + {\text{ u}}_{{0{\text{j}}}} \\ & {\text{Level}}\,{2}\,\left( {\text{school - level}} \right):\,\beta_{{{\text{1j}}}} = \, \gamma_{{{1}0}} + \, \gamma_{{{11}}} \left( {{\text{W}}_{{\text{j}}} - {\text{W }}} \right) \, + {\text{ u}}_{{{\text{1j}}}} \\ \end{aligned}$$

In the three equations X represents one of the personal motivation variables, W is one the school-level goal structure variables, Y is one of the outcome variables. The effect of W on the slop β_1j represents the cross-level interaction effect γ₁₁.

There were altogether six possible interactions between three motive variables and two goal structure variables for each outcome variable, which were checked one after another. Only the models with significant interaction effect were selected. They were compared with models without interaction and the null models based on Akaike’s Information Criteria (AIC) and Bayesian Information Criteria (BIC). The results of best models were selected for further reporting.

In order to visualize the cross-level interaction, simple effect plots and Johnson-Neyman plots were created using R and Mplus. In the J–N plots X-axis depicts the range of the moderator, Y-axis depicts the adjusted slop of the explanatory variable on the outcome variable, and the curved lines above and below the main line represent the 95% confidence bands around the adjusted effect. With these plots we can better observe the adjusted main effect given a range of the moderator.

Samples

In this study, data from China (B–S–J–Z) and the United States was analysed and the following section provides specific information about the sample in each country. Descriptive information can be seen in Table 1.

Table 1 Descriptive information of ESCS, achievement motives, perceived goal structures and outcome variables

Chinese sample (B–S–J–Z sample)

Only students from the four provinces/municipalities Beijing, Shanghai, Jiangsu, and Zhejiang participated in the PISA 2018 study, which was the sample our analysis was based. Original participants were 12,058 Chinese students, including 5775 female students and 6283 male students. There were only 19 students having immigrant background, so the scale for immigration status was not included as a predictor for outcomes. There was one school with only one student, which was eliminated from the multilevel regression analysis. Since there were missing data for different key variables in this analysis, there were finally 11,700 students included in the multilevel regression analysis (missing rate around 3%).

Within the 11,700 students there were 5618 female students and 6082 male students. Only 12 students in lower secondary vocational school system, so lower secondary vocational school track was combined with other students in the vocational track. There were altogether 2053 students in vocational schools, 4146 in lower secondary and 5501 higher secondary schools.

US sample

Original participants were 4838 U.S. students. There were 2 schools, in one school only 1 student was sampled, in the other only 2 students were sampled. They were excluded in the regression analysis. Given missing data of key variables, there were finally 4170 students included in the multilevel regression analysis (missing around 14%).

Within the 4170 students there were 2091 female students and 2079 male students. 306 students were in lower secondary schools, and 3864 were in higher secondary schools. Regarding immigration status, there were 220 first generation, 656 s generation, and 3294 students without immigration background.

Measures/indicators

Student level explanatory variables: student personal achievement motives

Students’ responses were used as manifest indicators of a latent variable representing the underlying trait, which is the scale scores for different constructs (OECD, 2018). These individual scores, Warm’s Mean Weighted Likelihood Estimates (WLEs), were transformed to a mean of zero and a standard deviation of one across the OECD countries.

Three scales were constructed in the PISA 2018 project which belongs to the module 9 (Dispositional & school-focused variables): COMPETE (competitiveness), WORKMAST (work mastery) and GFOFAIL (fear of failure). COMPETE was composed of 3 items, collecting information about students' competitiveness achievement motive. It applied a 4-point Likert scale ranging from "strongly disagree" to "strongly agree". WORKMAST was composed of 4 items, informing us about the mastery achievement motive of students. It was measured by a 4-point Likert scale ranging from "strongly disagree" to "strongly agree". GFOFAIL consisted of 3 items, informing us about students' general fear of failure. A 4-point Likert scale was applied, ranging from "strongly disagree" to "strongly agree". Reliability of the three constructs was higher than .75 for the two countries. RMSD (root mean square deviation, item fit index) showed no item differences of the three scales among the two countries (< .3). The items of these variables were listed in the “Appendix”. Since these variables were considered as individual variables, school-level variables would be less meaningful thus were not used as predictors in regressions in this study.

School level explanatory variables: relational goal structures

In PISA 2018 there were two scales measuring students’ perception of school environment. PERCOOP measured students’ perception of school goal structure, whether there was more value on cooperation, while PERCOMP measured students’ perception school focus on competition. The two scales consisted of 4 items each with reliability higher than .8 across countries. The 4-point Likert scale ranging from "not at all true" to "extremely true" was applied. The items were listed in “Appendix”. RMSD also showed no item differences of the two scales among the two countries (< .3).

Given the reflective nature of the perception variables for the school climate and the predictive power of the reflective measures for outcomes (Lüdtke, et al. 2009; Wallace et al., 2016), school-level variables were generated from the two perception variables “PERCOMP” and “PERCOOP”. They were group mean at school level and represented the school climate of focusing on competition or cooperation. They were labelled as “COMPFO” and “COOPFO” accordingly in the following sections. The reliability of the aggregated school-level variables was also calculated. For Chinese (B–S–J–Z) students, the ICC2 was .60 for PERCOMP and .66 for PERCOOP respectively; for U.S. students, ICC2 was .47 for PERCOMP and .47 for PERCOOP respectively. They were not high but considered as acceptable (Fleiss, 2011).

Outcome variables: reading achievement and self-concept

Students’ general reading performance was defined as a set of competences that enable students to write information, present texts for a purpose. The assessment measures besides reading fluency also the process of locating, understanding, evaluating and reflecting information (OECD, 2019a, 2019b). It contained various types of texts and tasks over a range of difficulty levels. Due to the study design students’ achievement was estimated with ten plausible values.

Self-concept in reading was measured by the scale SCREADCOMP, consisting of three items with 4-point Likert scale ranging from "strongly disagree" to "strongly agree". The items were listed in “Appendix”. The scale captured students’ general perception of their competence in reading and had a reliability higher than .75. RMSD also showed no item differences of the two scales among the two countries (< .3).

Control variables: demographic variables

Information about gender, school track (vocational or academic track, lower or higher secondary), economic, social and cultural status (ESCS) and immigration status were also included in the analysis. In PISA 2018 project, the variable for school track was PROGN, which was a school-level variable. The index of social economic and cultural status is ESCS (economic, social and cultural status), IMMIG is the variable for immigration status. For the variable PROGN, the different tracks were combine into vocational/academic track, lower or higher secondary schools. ESCS in PISA is a composite score built by the indicators parental education, highest parental occupation, and home possessions. Immigration status was categorised as native (students who have at least one parent born in the country), second generation (students born in the country of assessment but their parents are born in another country) and first generation (both the students and their parents are born in another country).

Results

School level variance

For Chinese (B–S–J–Z) sample, the intraclass correlation coefficients ICC1 for outcomes were checked. The ICC1 for reading achievement was .46, for self-concept in reading was .09. The ICC1 for the perception of competition in schools was .04 and for cooperation was .06. For the U.S. sample, the ICC1 for achievement was .16, for self-concept in reading was .02, for perception of competition and cooperation in schools were both .03. School-level variance for outcomes was relatively smaller for the U.S. sample. However based on the design of PISA research, multi-level analysis is still necessary.

The relationship between personal achievement motives and relational goal structures

Regarding RQ1, correlations among personal achievement motives and relational goal structures at student and school level were presented in Table 2. The correlations at student level were population correlations without centering, for the school level correlations group means were generated and centered with grand mean. From the results we can see, there were positive correlations between work mastery motive and cooperative goal structure, as well as between competitiveness motive and competitive goal structure across two countries. At student level as well as school level, most of the “cross-correlations” (such as the correlation between competitiveness motive and cooperative goal structure) were lower than the “paired correlations” (such as the correlation between competitiveness motive and competition goal structure). However, there was one exception: the correlation between competitiveness motive and cooperation goal structure was higher than the correlation between competitiveness motive and competition goal structure for Chinese (B–S–J–Z) students at student level. For fear of failure besides the expected positive correlation with competition focus and negative correlation with cooperation focus there was also one unexpected positive correlation for U.S. students: cooperation focus in school was positively related to mean fear of failure at school level.

Table 2 Population correlations and between-school correlations among personal achievement motive and relational goal structures

Results of model comparison

Chinese (B–S–J–Z) sample

For predicting reading achievement, six models with possible cross-level interactions were checked one after another. Only the model with COMPFO*WORKMAST was significant thus was selected for the comparison with other models. Three nested models were compared: null model M₀ without predictors, M_c11 with three motivational variables and two contextual variables but without interactions, and M _c12 with the interaction COMPFO*WORKMAST. The results of model comparison were shown in Table 3. From the results we can conclude that M12 is the best model fitting the data.

Table 3 Comparison of models for prediction of reading achievement and self-concept

For predicting reading self-concept, the interaction COMPFO*WORKMAST and COMPFO*COMPETE were significant. They were then selected and compared with other models. Results in Table 3 showed that M_C22 and M_C23 were both better models compared with null model and the model without interaction.

U.S. sample

For predicting reading achievement and self-concept using the U.S. sample, no interaction was significant. Thus the best model is the model without interaction M1. Specific model information can be found in Table 3.

The influence of personal achievement motives and goal structures on reading achievement and self-concept

Based on the results from model comparisons, the estimates of the best model for each country were presented in the Tables 4 and 5. Regarding RQ2, the results showed that competitiveness was important for achieving a higher score in China (B–S–J–Z), work mastery motive and fear of failure can influence achievement in the United States. For predicting reading self-concept, the results were similar between countries: both work mastery and competitiveness motives can influence self-concept, fear of failure can negatively influence reading self-concept for Chinese (B–S–J–Z) students.

Table 4 Regression for reading achievement and self-concept: Chinese B–S–J–Z sample

Table 5 Regression for reading achievement and self-concept: USA sample

As for RQ3, only competence focus in schools influenced the achievement of Chinese (B–S–J–Z) students but both goal structures had positive relationship with reading self-concept; For U.S. students both goal structures positively influence reading achievement.

There were also similar effect of demographic variables across countries observed. ESCS played an important role in predicting achievement and self-concept. Girls in the two countries had better achievement. Chinese (B–S–J–Z) students in higher secondary schools tracks had higher reading achievement but lower reading self-concept. In the United States, students in higher secondary schools had both higher achievement and self-concept. In the United States, the first generation performed worse in reading tests and had lower self-concept.

Cross-level interactions

Regarding RQ 4, cross-level interactions were only observed for Chinese (B–S–J–Z) students. For Chinese (B–S–J–Z) students the negative interactions between competitive goal structure and personal competitiveness and work mastery motives were observed. In schools with less focus on performance and competition, the work mastery motive can predict achievement and self-concept more and the competitiveness motive can predict achievement more; in schools with more focus on performance and competition, the influence of competitive and work mastery motives was reduced. Plots for simple effect were generated for the three cross-level interactions (see Figs. 1, 2, 3). Furthermore, the Johnson-Neyman plots for the three cross-level interactions were created (see Figs. 4, 5, 6) in order to check the confidence band of the adjusted effect. From Fig. 4 we can observe that the effect of work mastery motive on achievement was significantly larger than zero for the schools with less competition focus but the positive relationship was not significant for the schools with higher rating of competition focus in schools.

Fig. 1

The relationship between work mastery motive and reading achievement for Chinese B–S–J–Z students in schools with different level of competition focus

Fig. 2

The relationship between work mastery motive and reading self-concept for Chinese B–S–J–Z students in schools with different level of competition focus

Fig. 3

The relationship between competitiveness motive and reading self-concept for Chinese B–S–J–Z students in schools with different level of competition focus

Fig. 4

The Johnson-Neyman plot for the effect of competition focus in schools on the relationship between work mastery motive and reading achievement. “COMPFO” is the centered variable school focus on competition. “Slop” in this figure represents the adjusted effect of work mastery motive on reading achievement, and the curve lines represent 95% CI of this adjusted effect

Fig. 5

The Johnson-Neyman plot for the effect of competition focus in schools on the relationship between work mastery motive and reading self-concept. “COMPFO” is the centered variable school focus on competition. “Slop” in this figure represents the adjusted effect of work mastery motive on reading self-concept, and the curve lines represent 95% CI of this adjusted effect

Fig. 6

The Johnson-Neyman plot for the effect of competition focus in schools on the relationship between competitiveness motive and reading self-concept. “COMPFO” is the centered variable school focus on competition. “Slop” in this figure represents the adjusted effect of competitiveness motive on reading self-concept, and the curve lines represent 95% CI of this adjusted effect

Discussion and Conclusion

We set out to explore the relationship among personal achievement motives, school relational goal structures, and the achievement related outcomes with the PISA data from China (B–S–J–Z) and the United States. Similar results across the two countries were found: (1) There were positive correlations among the two achievement motives competitiveness and work mastery motives and the goal structure variables perception of cooperation and competition at both student- and school-levels; fear of failure was positively related to school competition focus. (2) Work mastery and competitiveness motives positively associated with reading self-concept, and fear of failure negatively associated with self-concept. (3) The competition focus of schools influenced reading achievement positively. There were also differences between the two countries: (1) Cross-level interactions can only be observed in the Chinese (B–S–J–Z) sample, with the competition focus of schools negatively moderating the relationship between work mastery motive and outcomes, and the relationship between competitiveness and reading self-concept. (2) The role of the three motive variables in explaining reading achievement is different: competitiveness motive predicted reading achievement for Chinese (B–S–J–Z) students while work mastery motive and fear of failure are significant explanatory variables for U.S. students; (3) For U.S. students the cooperation focus of schools was also an important factor in explaining the variance of reading achievement. (4) Moreover, both goal structures predicted self-concept for Chinese (B–S–J–Z) students while they were not significant predictors for U.S. students. In the following section the results were discussed.

Relationship between personal achievement motives and relational goal structures

In general there were positive relationships between one personal motive and the corresponding goal structure, which was in line with the previous literature (Meece, Anderman, & Anderman, ). The positive connection between mastery motives and cooperative goal structure was also consistent with the previous literature (Ames, 1992; Summers, 2006). Students would “adapt” to the school environment and their personal motives were enhanced when they fit the goal structures. Specifically, focus of competition in schools enhanced the competitiveness motive, while cooperation focus in schools created a positive learning environment and enhanced the work mastery motive. However, the “cross-relationship”, as was described in the meta-analysis (Bardach, Oczion, Pietschnig, & Lüftenegger, 2020), was lower but also found.

Fear of failure had no according goal structure. On one side, it was positively related to competitive goal structure at individual level and at school-level for U.S. students, which was consistent with previous findings (Bardach, Oczion, Pietschnig, & Lüftenegger, 2020). The focus on interpersonal comparison leads to more fear of failure. On the other side, fear of failure was negatively associated with cooperation focus of schools for Chinese (B–S–J–Z) students, which indicates that the fear of failure can be reduced if students feel support from cooperation goal structure. This was also consistent with previous findings (Summers, 2006). It is however unexpected that the mean fear of failure at school level correlated positively with cooperation focus in schools for U.S. students. It implies that in schools with more cooperation focus in the United States the students had generally higher fear of failure. Considering the positive relationship between the competition and cooperation focus of schools in the United States (r = .2***, SE = .09), the higher fear of failure can also be due to the competition focus or the combined effect of the two focuses.

Effect of personal achievement motives

The positive effect of work mastery goal on reading achievement and self-concept is consistent with previous literature (Greene, Miller, Crowson, Duke, & Akey, 2004; Wolters, 2004; Ames, 1992; Urdan, 1997). The average effect of mastery motive on reading achievement in Chinese (B–S–J–Z) sample was not significant, however from the J–N plot (see Fig. 4) we can see that for schools with lower competition focus there was still significant positive association between mastery motive and reading achievement.

Competitiveness motive associated with reading achievement for Chinese (B–S–J–Z) students but not for U.S. students. In previous studies the effect of competitiveness motive is also two-fold: It can have positive effect on learning (Busato, Prins, Elshout, & Hamaker, 2000; Niepel et al., 2014) but might also entail self-handicapping (Hulleman et al., 2010; Senko & Dawson, 2017). Evidence from cross-cultural studies showed that the performance goals are important for predicting achievement (Grand & Dweck, 2001). The results of Chinese (B–S–J–Z) sample supported the positive effect of competitiveness motive while the effect was not that strong for U.S. students.

The negative effect of fear of failure on self-concept is in line with previous research (Martin, 2002). The positive effect of general fear of failure on achievement in U.S. sample is however not consistent with previous findings (e.g., Church et al., 2001; Wolters, 2004). The positive connection between fear of failure and achievement is more like cultural characteristics of Asian students (Khajavy et al., 2018; Luo et al., 2011). But in the current study it was instead observed in U.S. sample. Fear of failure could lead to achievement success when students with high fear of failure have at the same time strong motive for approaching success, who can be described as “overstriver” (Martin & Marsh, 2003). Stronger social desirability in Asian samples (the tendency of not being an “overstriver”) might lead to biased rating on fear of failure (He & van de Vijver, 2012). Since we did not explore whether they are multiple-goal goers (Linnenbrink & Pintrich, 2003), it can also be a result of multiple-goal pursuing. More research is needed to understand the phenomenon.

Effect of relational goal structures in schools

The positive effect of cooperative goal structure on reading achievement for U.S. students and on self-concept for Chinese (B–S–J–Z) students was consistent with previous findings (Nichols, 1996; Roseth, Johnson, & Johnson, 2008). However for Chinese (B–S–J–Z) students it is not that important for enhancing achievement and for U.S. students it did not influence self-concept.

The role of competitive goal structure in the literature was two-fold. On one hand, it was shown that perceived competitive goal structure related to less engagement and lower self-efficacy and is less favourable (Urdan, 1997; Vu et al., 2021); on the other hand, there was evidence that it could entail the most comprehensive classroom communication, which can lead to better achievement (Vu et al., 2021). Our findings supported the latter hypothesis. For students from both countries there was positive association between competition goal structure and achievement and there was further positive influence of competition goal structure on self-concept for Chinese (B–S–J–Z) students.

Interactions among achievement motives and relational goal structures

The model including interaction variables were not the best model predicting achievement and self-concept in reading for U.S. samples. We only found the cross-level interaction for Chinese (B–S–J–Z) students.

If the positive influence of mastery and competitiveness motives on achievement and self-concept is desirable, the interactions can be described as “a dampening pattern” as defined in the previous literature (Lau & Nie, 2008). It was also depicted in another literature as vitiation effect of goal structures on the effect of personal motives (Murayama & Elliot, 2009). For Chinese (B–S–J–Z) students the competition focus of schools weakened the positive effect of work mastery and competitiveness motives on learning outcomes. This seems to be inconsistent with previous findings (Lau & Nie, 2008; Murayama & Elliot, 2009). In the study with Singapore students, Lau and Nie have not found any moderation effect of performance goal structure on the relationship between personal mastery or performance goal and achievement outcomes. Murayama and Elliot found a positive moderation effect of performance goal structure on the relationship between personal performance goal and self-concept. It could be due to the instrument and sample differences. As was elaborated in the introduction session, the relational goal structure was adopted in this current analysis and the measurement was also not the same with contextual goal structures. The two studies focused on math self-concept instead of reading, which can also be a reason for the discrepancy. Moreover, there were still ample differences of educational systems among China, Singapore and Japan, although they were considered culturally close to each other (Silver, Hu & Iino, 2002).

Cultural differences

Although there are general findings across two countries, differences can still be observed. For example, the role of the three achievement motives in explaining the variance of reading achievement is different: competitiveness motive predicted reading achievement for Chinese (B–S–J–Z) students while work mastery motive and fear of failure were significant explanatory variables for U.S. students. Work mastery motive had only influence on achievement for Chinese students in the schools with less focus on competition. Except for the students in Chinese schools with strong focus on competition, the results are consistent with previous literature about the role of work mastery motive across cultures (King et al., 2017). The focus on competition or the pressure from the educational system however hindered the positive function of work mastery motive in Chinese schools. The important role of competitiveness in predicting achievement for Chinese students is also in line with previous literature (Grand & Dweck, 2001). The highly selective educational system and the competition in the entrance examination for the secondary high school in China might be a reason, in which it is not possible to achieve without competitiveness motive (OECD, 2016). Moreover, fear of failure influenced the reading performance of U.S. students, which was also reported in the OECD analysis (OECD, 2019a, 2019b). Considering the selective college entrance examinations in the United States, it is understandable that certain level of fear of failure or anxiety can promote performance. This effect is more the consequence of educational system rather than the culture influence.

Furthermore, for U.S. students the cooperation focus of schools was an important factor in explaining the variance of reading achievement but the two goal structures cannot explain the variance of self-concept. On the contrary, both goal structures can explain the variance of self-concept for Chinese students but cooperation focus cannot explain the variance of reading achievement. It can be speculated that the cooperative goal structure or school climate in Chinese schools functioned more like a self-concept promotor (more positive feedback from others in the cooperative climate), while for US students they profited from this cooperative climate in speaking of improving their reading achievement (more communications, collaborative reading activities, etc.). We need to collect more information on the operationalization of the cooperative goal structure in the two educational systems in order to understand the results.

Implication and Limitation

Our study contributes to understanding the effect of personal achievement motives and relational goal structures on achievement and self-concept under the framework of a large scale assessment. It provided also insight on the joint influence of personal motives and goal structures on achievement related outcomes. Moreover, cultural general and specific findings can enrich the cross-cultural motivational research, which is an important but under-researched aspect (Bardach et al., 2020). The findings contribute to identify the culture-independent motivational mechanisms and help us understand cultural differences regarding motivational dynamics.

The results offered further implications for the school practice. For example, cooperative goal structure is in general positive in enhancing students’ achievement and self-concept and is worth being implemented in school practice. The focus on competition might also enhance achievement and it can be combined with the cooperation focus.

Limitations were partly related to the design of the PISA study. PISA employed a cross-sectional design and self-reported instruments, therefore causal conclusions cannot be made and response styles or social desirability might bias the results (He & van de Vijver, 2012). Although it should also be mentioned that for the measurement of goal structure the predictive power of student rating was considered higher than teacher rating (Bardach et al., 2020) and the use of aggregated level-2 variables to reflect the climate (reference shift) can better predict outcomes in general (Wallace et al., 2016). Moreover, the PISA study sampled students at school level instead of classroom level. There might be between-classroom variations of goal structures which cannot be identified with this sampling strategy. For a more precise estimation, information about the class-level goal structures should be collected.

Furthermore, large scale assessments provide opportunities for verifying hypotheses and improving theories. However there might also be discrepancy between the framework in previous literature in a specific area and the chosen framework in a large scale assessment such as PISA. The different operationalisation of motives and goal structures in PISA 2018 were not identical with previous literature about achievement goals. Therefore some results cannot be directly compared and the effect of the mastery goal structure was not covered in this current study. Moreover, multiple-goal perspective (Pintrich, 2000) and the coexistence of cooperation and competition focus was beyond the research scope of this current study. However they were also important aspects and can be integrated in the future analysis and studies. Additionally, the school number sampled in China (B–S–J–Z) was more than twice as that in the United States, it might influence the statistic power of the estimation of cross-level interactions and reduce the comparability of the results from the two countries. A future study with larger sample in the United States might provide more precise information for the cross-level interactions. Future research might also consider using qualitative approaches to research on the specific ways of implementing cooperation and competition goal structures in Chinese and U.S. secondary schools, the process they interact with student achievement motives, and their effect on learning outcomes.

Appendix: Items of relevant scales implemented in the PISA 2018 project

COMPETE: competitiveness motive

• ST181 How much do you agree with the following statements about yourself?

• ST181Q02HA I enjoy working in situations involving competition with others.

• ST181Q03HA It is important for me to perform better than other people on a task.

• ST181Q04HA I try harder when I’m in competition with other people.

WORKMAST: work mastery motive

• ST182 How much do you agree with the following statements about yourself?

• ST182Q03HA I find satisfaction in working as hard as I can.

• ST182Q04HA Once I start a task, I persist until it is finished.

• ST182Q05HA Part of the enjoyment I get from doing things is when I improve on my past performance.

• ST182Q06HA If I am not good at something, I would rather keep struggling to master it than move on to something I may be good at.

GFOFAIL: fear of failure

• ST183 How much do you agree with the following statements?

• ST183Q01HA When I am failing, I am afraid that I might not have enough talent.

• ST183Q02HA When I am failing, I worry about what others think of me.

• ST183Q03HA When I am failing, this makes me doubt my plans for the future.

PERCOMP: perception of competition in schools

• ST205 Think about your school: how true are the following statements?

• ST205Q01HA Students seem to value competition.

• ST205Q02HA It seems that students are competing with each other.

• ST205Q03HA Students seem to share the feeling that competing with each other is important.

• ST205Q04HA Students feel that they are being compared with others.

PERCOOP: perception of cooperation in schools

• ST206 Think about your school: how true are the following statements?

• ST206Q01HA Students seem to value cooperation.

• ST206Q02HA It seems that students are cooperating with each other.

• ST206Q03HA Students seem to share the feeling that cooperating with each other is important.

• ST206Q04HA Students feel that they are encouraged to cooperate with others.

SCREADCOMP: self-concept in reading

• ST161 How much do you agree with the following statements?

• ST161Q01HA I am a good reader.

• ST161Q02HA I am able to understand difficult texts.

• ST161Q03HA I read fluently.