Measuring student well-being in adolescence: proposal of a five-factor integrative model based on PISA 2018 survey data

Abstract

Much debate exists concerning the factorial dimensionality of student well-being. We contribute to this debate by drawing on PISA 2018 data from a total of 61,722 students, 2528 schools, and nine countries. For our investigation, we test multiple associated measurement models for convergent, discriminant, and concurrent validity, model fit, and measurement invariance. For the PISA 2018 sample, we found very little variation in student well-being both between schools and between countries suggesting that variation in student well-being exists predominantly within respective school contexts. In addition, our findings support the notion that student life-satisfaction should be split into separate general and school-based factors in a Bi-Dimensional Life-Satisfaction Model. Moreover, results suggest that PISA 2018 data can be used to successfully measure a Four-Factor Hedonic Model of Student Well-being, tackling both cognitive (general life-satisfaction and satisfaction with school factors) and affective features (positive and negative affect). Finally, we propose a Five-Factor Integrative Model of Student Well-being that supports a broader conceptualization of student well-being that includes life-satisfaction, both positive and negative affect, and eudaemonia. We tested this model and found that it met the requirements for scalar invariance across male and female gender groups. However, for all 36 inter-country tests of measurement invariance, metric invariance was only reached nine times (25%), and scalar invariance was reached once (2.8%). Implications and recommendations for follow up research are provided.

Introduction

A growing body of research has focused on adolescence and well-being, with investigations ranging from positive education (Morrish et al., 2018), quality of life Keith and Schalock (1994), family (Hawkins et al., 2007; McFarlane et al., 1995), social support (Yarcheski et al., 1994), life-satisfaction (Huebner, 1994), race Brenner et al. (2018), and spirituality King and Benson (2005) to name a few. Maintaining and sustaining the well-being of children and young people are among the priorities of the United Nations Educational, Scientific, and Cultural Organization’s (UNESCO) strategy on education (UNESCO, 2016). The Organization for Economic Co-operation and Development (OECD, 2015) identifies two distinct perspectives for the conceptualization and measurement of children's well-being: the (1) developmental and (2) child-rights perspectives. The developmental perspective stresses that sound child well-being implies sound adult well-being in the future. In contrast, the child-rights perspective emphasizes the idea that children are human beings and experience well-being in the ‘‘here-and-now’’ (p. 144). Notwithstanding these different approaches, it is broadly recognized that childhood is a unique and critical phase for the prosperity and development of future societies (OECD, 2015). Hence, there has been rising interest over the past decade in measuring students’ well-being and comparing the efforts of countries in promoting students' quality of life and overall development (Borgonovi & Pál, 2016). Measuring 15 year-old students' well-being, the target population of the Programme for International Student Assessment (PISA) surveys, serves well to this end because this period of early adolescence is a critical transition period of socio-emotional development (American Psychological Association, 2002). This paper explores the factorial structure of student well-being for this stage of development by drawing on the 2018 PISA data.

Literature review

The purpose of the literature review is four-fold. First, it provides operational definitions for how the PISA 2018 program defines aspects of student well-being in terms of life-satisfaction, positive and negative affect, and eudaemonia. Second, it provides a historical review of how well-being has been conceptualized and operationalized since becoming popular in the 1960s. Third, it gives an account of different measurement approaches to student well-being in PISA. Fourth, it provides a rationale for the current investigation.

Definitions

The OECD defines well-being as a dynamic state identified by students experiencing the opportunity and ability to actualize their social and personal goals, including psychological, cognitive, material, physical, and social capabilities required to live a fulfilling and happy life (Borgonovi & Pál, 2016). The PISA 2015 survey included variables reflecting the dimensions of students' psychological, cognitive, material, physical, and social well-being. However, it has been argued that the dimensionality of well-being was not clear as some of the factors were not entirely relevant to student well-being, and some were not unidimensional (Govorova et al., 2020a). Thus, in the subsequent PISA 2018 cycle, the composition of the dimensions and the indicators were adjusted to better manifest student well-being (Govorova et al., 2020b). By these recent changes, the current study employs the latest PISA 2018 cycle to explore the factor structure of the psychological domains of student well-being, including life-satisfaction, positive affect (PA), negative affect (NA), and eudaemonia.

Life-satisfaction, PA, and NA are the three components of Diener and Ryan's (2009) Tripartite Model of Subjective well-being, which manifests well-being as an overall evaluation of an individual's quality of life. This subjective well-being model is also called the hedonic model of well-being because it involves satisfaction, pleasure, enjoyment, comfort, painlessness, and ease with a particular focus on the self and the minimization of negative feelings and maximization of positive ones (Huta & Waterman, 2014).

On the other hand, Eudaemonia aims to pursue ‘‘a good life’’ by hoping for the best within us, excellence, and virtue (Huta & Waterman, 2014). Hence, eudaemonic well-being refers to the quality of life derived from developing an individual’s longer-term capacities (Ryan & Deci, 2000) and their application to actualize personally meaningful objectives (Waterman et al., 2010). Some researchers suggest that eudaemonic well-being should be treated separately from the dichotomy of affective and cognitive measures of subjective well-being because it is a theoretically distinct construct and its underlying principles are different (Tsurumi et al., 2021).

However, most scholars believe that hedonic and eudaemonic well-being are complementary psychological functions and individuals need both to flourish (Huta, 2015). Waterman et al. (2010) suggested that although subjective well-being and eudaemonia constitute aspects of a single core well-being concept, they are empirically and conceptually distinguishable. The findings of Rudolf (2020) and Seon and Smith-Adcock (2021), who both used the PISA 2018 survey data, revealed that eudaemonia was strongly associated with better subjective well-being among adolescents. Moreover, Strelhow et al. (2020) suggested evidence that both hedonic and eudaemonic well-being can be the two main components of an integrative model of well-being.

Approaches for measuring well-being

The conceptualization and operationalization of well-being have gone through many changes since the 1960s (Diener, 1984), and instruments have varied in length and exhibited various psychometric properties. Nevertheless, hedonic, eudaemonic, and integrated models of well-being are generally the most common.

The early focus of hedonic well-being was on PA and NA (Bradburn & Caplovitz, 1965). This line of research helped in understanding how PA and NA were negatively correlated. However, during this period, scholars argued that the sole focus on positive and negative emotions, without regard for one's quality of life or general assessment of one’s life, was somewhat myopic (Brenner, 1975). These criticisms led to the conceptualization of Subjective Well-Being (SWB) (Schueller & Seligman, 2010; Tov, 2018), which along with the presence of pleasure and absence of displeasure, included cognitive evaluations such as judgements of one's overall satisfaction with life (Diener, 2000; Kahneman et al., 1999; Shin & Johnson, 1978).

Eudaemonic well-being's modern conceptualizations are influenced by Aristotle's ideas of a virtuous, authentic, and balanced life (Adler & Seligman, 2016; Allport, 1961; Bradburn, 1969; Diener et al., 2018; Ryan & Deci, 2001; Ryff, 1989; Seligman, 2011; Steger, 2016). The fulfillment of one's highest nature, authenticity, and self-actualization, often used synonymously, are the most consistently and agreed-upon descriptions of eudaemonia (Allport, 1961; Annas, 1993; Frankl, 1963; Haybron, 2016; Norton, 1976; Rogers, 1961; Ryff, 2016; Waterman, 1990, 1993). Concurrently, eudaemonia has also been conceptualized as trait-like and expanding over one's lifespan (Ryff, 1989). Other definitions include a sense of balance and harmony (Delle Fave et al., 2011), the fulfillment of one's basic psychological needs for autonomy and intrinsic motivation (Deci & Ryan, 1995), the use of strength of character to achieve engagement and meaning in life (Seligman, 2002), a psychological state marked by positive emotion and social and psychological function (Keyes, 2002), the state of mind in opposition to mental disorders (Huppert & So, 2013). This myriad of conceptualizations has led to disagreements in the field (see, for example, Kashdan et al., 2008; Keyes & Annas, 2009; Ryan & Deci, 2000; Sheldon, 2016; Waterman, 2008). Among these debates, two theories have dominated—psychological well-being (PWB) (Ryff, 1989) and self-determination theory (SDT) (Ryan & Deci, 2001).

The dimensions of eudaemonia, according to PWB, are self-acceptance, positive relations with others, autonomy, environmental mastery, and personal growth over the lifespan (Ryff, 1989). Ryff's (2013) The PWB Scales is the most common measure in this area and has been translated into over 30 languages.

On the other hand, SDT defines eudaemonia as a way of living motivated by the fulfillment of one's most authentic or highest nature, or nature-fulfillment (see Ryan et al., 2008; also see Greek root, ‘‘daemon’’, ‘‘an attendant power or spirit’’; Māori, replete ‘‘mana’’: Merriam-Webster, 2023) and conceives eudaemonia as neither a mental state nor outcome. Its practical framework allows the study of social-contextual factors that support and thwart well-being (Ryan & Deci, 2017). According to SDT, two eudaemonic processes articulate the fulfillment of one's nature. First, human motivation, the importance of autonomy, and internalization are seen as the drivers of behavior. Second, autonomy, competence, and relatedness are seen as the mediators between social-contextual factors and well-being.

The distinctions between hedonic and eudaemonic well-being are not without controversy. Studies have found significant correlations between the two constructs, and proponents of the associated two-factor model also report that components of both hedonic and eudaemonic well-being load substantially on both factors (Waterman, 1993). More recently, therefore, Seligman's (2011) PERMA model and Keyes' (2005) Mental Health Continuum (MHC) have attempted to integrate hedonic and eudaemonic well-being.

The PERMA model was conceptualized with the aims of positive psychology in mind—individual flourishing and not just the absence of relief from distress (Seligman & Csikszentmihalyi, 2000). Its five factors include positive emotion (or a pleasant life), engagement and flow, relationships, meaning, and accomplishment (PERMA). MHC incorporates subjective well-being (SWB; Keyes, 2002, 2005, 2007) and psychological well-being (PWB) (Ryff, 1989), as discussed above. The PERMA-profile model also includes positive social functioning constituting a five-factor model that captures elements of functioning in society and related aims and challenges (Keyes, 2005).

There have been various measurement models and scales with sound psychometric properties that focus on measuring well-being among adolescents. The constructs measured by these instruments include empathy, connectedness, self-efficacy, adaptability, initiative, conscientiousness, social competence, optimism, emotional self-regulation, and mindfulness included in the Child and Adolescent Wellness scale (Copeland et al., 2010); engagement, perseverance, optimism, connectedness, and happiness in the EPOCH Measure of Adolescent Well-Being (Kern et al., 2016); physical, anxiety, mood, and self/others in the PGI (patient-generated index) Well-Being scale (Verma et al., 1983); and belief in self, belief in others, emotional competence, and engaged living in the Social and Emotional Health Survey (Furlong et al., 2014). While these various approaches have been useful, the OECD's PISA program has also made significant contributions to measuring adolescent well-being.

Measuring student well-being in PISA

Over the last two decades, the OECD's PISA programme has played an increasingly important role in shaping national education systems and policies. New assessment domains have been incorporated into PISA in recent cycles, including well-being, further expanding their interest in the educational and psychological research community (Hernández-Torrano & Courtney, 2021). Addressing student well-being in PISA has been justified due to its intrinsic value for the well-being of the general population, its influence as a determinant of adult well-being, and its role as a driver of educational outcomes in school contexts (Schleicher, 2015).

PISA defines well-being broadly as the ‘‘quality of people’s lives and standard of living’’ (OECD, 2019a, p. 259). The measurement of student well-being in PISA debuted in the 2015 cycle. However, only a few well-being focused items were added to the student questionnaire limiting the conclusions drawn from that survey cycle. As of 2018, PISA incorporated a separate questionnaire that included multiple questions that reflected a more exhaustive conception of student well-being. The current PISA well-being conceptual framework accounts for two types of well-being indicators: objective/material (e.g., household income) and subjective/psychological (e.g., life-satisfaction). The framework also distinguishes between three different dimensions: self, school environment, and out-of-school environment (OECD, 2019a). Yet, there exists a lack of scholarship justifying the particular distinction between student well-being in the school and out-of-school environment.

Due to the number of well-being indicators and dimensions included in the PISA questionnaires, there is currently no consensus on the best measurement approach for student well-being. A diversity of approaches has been recently proposed and used in the literature. One of them is rooted in the framework for analyzing student well-being proposed by Borgonovi and Pál (2016). This framework refers to well-being as the psychological, cognitive, material, social, and physical capabilities students need to live a happy and fulfilling life (OECD, 2017). To date, studies testing the PISA 2018 framework are sparse and have provided contradictory evidence about its construct validity. For example, Govorova et al. (2020a) used a latent variable approach (i.e., CFA) to assess a five-dimensional model based on this well-being structure. The results demonstrated that only the cognitive and material dimensions achieved appropriate model fit, and the psychological, social, and physical dimensions did not exhibit construct solidity. In a subsequent study based on the same PISA 2018 data, Govorova et al. (2020b) used a psychometric network approach to examine an updated model of student well-being that included three dimensions: psychological, cognitive, and social. The authors concluded that these three dimensions constitute a solid construct of student well-being, with resilience, fear of failure, and sense of belonging playing the most central roles in the network.

An alternative approach to incorporating student well-being in PISA studies is to include particular questions in the PISA survey that attempt to capture a general or broad sense of student well-being. The most common approach has been to use the single item of overall life-satisfaction (OLS) as a cognitive indicator of subjective well-being (e.g., Marquez & Long, 2021; Marquez & Main, 2020; Tang, 2019; Yin-Nei Cho, 2019). Also included in the 2015 cycle, this item asks students to rate how satisfied they are with their lives these days from 0 (not at all satisfied) to 10 (completely satisfied). The simple method has been justified due to its brevity, simplicity, and robustness (OECD, 2019b). In addition, the single item presents a moderate to high correlation with other multidimensional life-satisfaction scales (e.g., sense of belonging in school) (Marquez & Main, 2020). However, the use of a single measure for life-satisfaction has inherent limitations (see “construct under-representation”, AERA et al., 2014, p. 12). Life-satisfaction is complex, and a global measure can mask distinctions between the different life domains (Weber & Huebner, 2015). Consequently, the OECD (2019a) recommended including additional items in the recent PISA survey to account for students' judgments of their satisfaction with specific domains in life, such as family and school. This is more in line with the conception that student well-being is better understood as a multifaceted construct (e.g., Ben-Arieh et al., 2014; Hernández-Torrano, 2020).

PISA has now incorporated affective features into the measurement of subjective well-being. In the PISA 2018 survey, students were asked how often they experience certain positive and negative states on a scale from 1 to 4 (1 = never, 2 = rarely, 3 = sometimes, 4 = always). This has provided valuable opportunities for secondary analyses based on the PISA 2018 data specifically. For example, Rodríguez et al. (2020) assessed student PA using three items asking students how often they felt happy, cheerful, and animated. Other studies have also successfully assessed NA using the four available items that ask how often students feel scared, miserable, afraid, and sad (e.g., Rudolf, 2020; Seon & Smith-Adcock, 2021).

Including the eudaemonic items in the most recent PISA cycle has also provided valuable insights into adolescent well-being. Often, such research has focused on two popular dimensions of eudaemonic well-being: sense of belonging and meaning in life. For example, Montt and Borgonovi (2018) used the sense of belonging scale in PISA 2015 to proxy student well-being. This scale is built from nine four-category Likert-type questions in the student questionnaire (e.g., ‘‘I make friends easily at school’’). Other studies have used the six four-point Likert-scale items included in the PISA 2018 student questionnaire (e.g., ‘‘I feel part of the school’’) to create a composite score for sense of belonging (Rodríguez et al., 2020; Seon & Smith-Adcock, 2021). However, the sense of belonging scale was not intended to measure student well-being in PISA 2012, and it was retained in the PISA cycles of 2015 and 2018 to measure the ‘‘social’’ sub-construct of well-being (Clarke, 2020), which is outside the more specific scope of this study, and thus, we did not include the sense of belonging scale in our measurement models. In a recent PISA 2018-based study, leaning on the three eudaemonic items, Seon and Smith-Adcock (2021) explored the mediating role that meaning in life had on the relationship between bullying victimization and subjective well-being (an example eudaemonic item in this case: "My life has clear meaning or purpose").

Some scholars have criticized the overall OECD's measurement approach to student well-being in PISA. Cefai and Cavioni (2015) argue that PISA promotes a narrow view of education with too much focus on cognitive and academic aspects and less attention to emotional and affective dimensions. In addition, PISA has been rebuked for promoting an individual approach to life-satisfaction and well-being that is more established in Western traditions, excluding and overshadowing interdependent conceptions of well-being widespread in collectivist cultures (Rappleye et al., 2019). Moreover, previous studies have expanded the conception of child and adolescent well-being to include, for example, tendency to try new things (Huebner, 1994) and quality of same- and opposite-sex friendships (Gilligan & Huebner, 2007). There is, and likely will continue to be, much debate about the appropriate conceptualization and measurement of student well-being. As PISA applies more nuanced approaches to measuring student well-being, more research will attempt to understand the phenomenon itself. Therefore, ongoing, careful studies that serve to explore the factorial dimensionality of student well-being provide an essential starting point.

Rationale for current investigation

As explained, the PISA survey includes multiple questions about student well-being, and there is a general debate among researchers in the field as to how the various factors associated with this focal area should be conceived (e.g., Ryan & Deci, 2001; Strelhow et al., 2020). In addition, researchers may be interested in only one or perhaps multiple outcomes in a single study, and general guidelines for such research are lacking. Therefore, this study contributes to the body of literature concerned with the general measurement of student well-being in PISA by systematically testing multiple measurement models in different steps. For step 1, we test the Life-Satisfaction model and explore the possible distinction between general life-satisfaction and satisfaction with school factors. Thereafter, for step 2, we look to extend this initial model to include the factors of PA and NA in a hedonic model. For step 3, we test the inclusion of meaning in life factor in a broad, integrated five-factor model of student well-being that includes both hedonic and eudaemonic features.

Methodology

General approach

The methodological approach employed here is an adaptation of the general approach employed by Waterman et al. (2010) who implemented a multi-step evaluation system for a single-factor eudaemonic model. For each of the three steps, we undertook a traditional examination of model fit and also employ an examination of discriminant and concurrent validity of the model under investigation. Because of the nested structure of the data, with students nested in schools and schools nested in countries, we undertake an initial examination of the variance components for the well-being variables of interest to determine the general framework for analysis of the study. Finally, we also include an examination of measurement invariance for our proposed final model in terms of both gender and country.

Research questions

Given the goals of the current study, we propose the following five research questions:

• RQ1: To what degree do the PISA well-being variables vary (a) between schools and (b) between countries?

• RQ2: Step 1: What is the most appropriate life-satisfaction model viz-á-viz (a) discriminant validity, (b) concurrent validity, and (c) model fit?

• RQ3: Step 2: What is the most appropriate hedonic model viz-á-viz (a) discriminant validity, (b) concurrent validity, and (c) model fit?

• RQ4: Step 3: What is the most appropriate integrative model viz-á-viz (a) discriminant validity, (b) concurrent validity, and (c) model fit?

• RQ5: Is the proposed integrative model invariant to country and student gender?

Participants

Participants in the study included students in the PISA 2018 survey that completed all of the life-satisfaction, PA, NA, and eudaemonia survey items. Only nine (of the 80) countries administered the ten-item life-satisfaction battery together with the other well-being focused questions included in this study. All the other 71 participating countries used just one item to measure overall life-satisfaction. Thus, the sample of this study consists of participants from nine countries, as described in Table 1.

Table 1 Breakdown of Participating Students by Country and School

Instruments

The instruments used in the current study were fielded in the 2018 PISA cycle. Here, we make use of the scale designed to measure Satisfaction with General Life (10 items), the Positive and Negative Affect scales (4 items each), and the Eudaemonia scale (3 items). To examine the concurrent validity of the factors at the three steps, we include the additional item, Overall Life-Satisfaction. Details for the presentation of these items and respective response options are provided in Table 2.

Table 2 Descriptive statistics and variance components for well-being related variables

Statistical procedures

Variance components

To answer RQ1, with regards to variance components, all items of interest were examined viz-a-viz the degree to which they varied (1) between schools and (2) between countries. For this analysis, separate three-level null models were run with the assistance of the R lme4 package (Bates et al., 2015). According to Snijders and Bosker (1999), there is no need to undertake multilevel modeling when ICC estimates are below 0.05. In addition, it is also broadly acknowledged that under low ICC conditions, between-level fit indices become less useful (Hsu et al., 2016). Therefore, the results of this initial investigation determine the general framework for the analysis of consequent research questions in the current study.

Discriminant and concurrent validity and model fit

For this study, RQ2, RQ3, and RQ4 include an analysis of (a) discriminant validity, (b) concurrent validity, and (c) model fit for the Life-Satisfaction, Hedonic, and Integrated models. For the current study, bi-factor and higher-order models were not included as research suggests that factors associated with quality of life tend not to be orthogonally related and often exhibit moderately strong correlations (Strelhow et al., 2020). A description of how the three statistical approaches were applied in the current study will now be provided.

Discriminant validity

Discriminant validity was examined by way of the following five criteria: (i) the existence of minimum item-factor loadings (with > 0.50, acceptable), (ii) the Cronbach’s alpha reliability coefficients (Cronbach, 1951) for each factor (with > 0.70, acceptable), (iii) the heterotrait-monotrait criteria (HTMT_0.85; Kline, 2011), (iv) the average variance extracted (AVE_0.50) for each factor, and (v) the average variance extracted-shared variance criterion (AVE-SV; Fornell & Larcker, 1981). Given that these series of tests can be considered increasingly conservative (Rönkkö & Cho, 2022), at a minimum, we decided that criteria (i), (ii), and (iii) should be met for a model to be accepted. Tests for discriminant validity were undertaken with the assistance of the R semTools package (Jorgensen et al., 2021).

Concurrent validity

Concurrent validity was examined by way of each factor's bivariate correlation with the single Life-Satisfaction item. For this, separate additional confirmatory models were tested for the sole purpose of estimating single correlation coefficients (r) with interpretations by way of Cohen (1992; 0.10 = small, 0.30 = medium, and 0.50 = large). An examination of 95% confidence intervals for this relationship is also conducted. Interpretations were based on theory, with positive correlations expected. It should also be noted that the single Life-Satisfaction item was, like the various well-being related items in the current study, scattered across the large Module 9 in the PISA student survey which measured various student dispositional and school-focused variables. Therefore, the relative position of the item itself was similar to the other focal variables in the current study.

Model fit

In terms of model fit, competing models were examined (i) in terms of general model fit and (ii) nested fit (see Fan & Sivo, 2005; Hu & Bentler, 1999, for simulation studies suggestive of the utility of model fit cutoffs below). For general model fit, the badness-of-fit measures include the Chi-square (χ²) and degrees of freedom (df) (though considered sensitive to sample size, so non-significance is not necessary; Fan et al., 1999), Standardized Root Mean Square Residual (SRMR) (less than 0.08; Hu & Bentler, 1999), and Root Mean Square Error of Approximation (RMSEA) (less than 0.08; Byrne, 2001). The goodness of fit measures include the Comparative Fit Index (CFI) (above 0.90; Cheung & Rensvold, 2002), the Tucker-Lewis Index (TLI) (above 0.90; Cheung & Rensvold, 2002), and Gamma Hat (above 0.90, a powerful index that is not affected by sample size) (Fan & Sivo, 2007). Where models were nested (step 1), direct comparisons of model efficiency and consistency were made in accordance with the AIC and BIC criteria, respectively (Burnham & Anderson, 2004).

Measurement Invariance

For RQ5, measurement invariance was checked within a multiple-group confirmatory factor analysis (MGCFA) framework for the integrative measurement model. In order to make valid comparisons between groups (correlations between groups), it is necessary to demonstrate measurement invariance. For these tests, configural (equivalent item-factor structure), metric (equivalent item-factor loadings), and scalar invariance (equivalent item-factor loadings and intercepts) between (1) gender groups and (2) all possible country combinations were undertaken. We assess the configural model to each condition’s data, whereby CFI should be no smaller than 0.95 and RMSEA should be no larger than 0.05. The acceptance of a more restrictive model was based on the change of CFI and RMSEA. Specifically, for configural to metric models, the cutoff point for CFI was 0.02, and RMSEA was 0.03. For metric to scalar models, cut off points for CFI and RMSEA were both 0.01 (see Rutkowski & Svetina, 2014). For configural, metric, and scalar invariance to be met, models would need to meet the requirements (to 2 decimal places) for both CFI- and RMSEA-based statistics.

All models were examined with the assistance of the R lavaan package (Rosseel, 2012). For all tests, statistical significance was set at p < 0.05. Note that for all of the measurement models, maximum likelihood (ML) estimation was used as the data was assumed to be continuous (Robitzsch, 2020).

Use of sample weights

Sample weights were applied to all CFA models. The analysis drew upon the PISA “sentate weights” variable (SENWT) to ensure that each of the nine countries made equal contributions to the results of the analysis. This was done so that the findings of the study could be seen as equally applicable to all nine participating countries. Because of missing data, the sum of all senate weights for each country did not reach 5000. Therefore, the senate weights for each country were multiplied by a constant such that the resultant sum of all student senate weights for the respective countries came to 5000. The constant for each country was estimated in accordance with Eq. 1:

$$Country\; Senate\; Weight \;Constant_{i}= \frac{5000}{\sum_{i=1}^{I}SENW{T}_{i}}$$

(1)

Additional country-level data (Additional file 1) and the R code for all analysis (Additional file 2) is available as supplementary material.

Results

RQ1: Between-school and between-country variance

Table 2 presents the descriptive statistics for the variables of interest for the study. Specific to RQ1, the between-school and between-country variance components for each variable are presented in the final two columns.

Between-school variance ranged between 0.016 and 0.044 (M = 0.028, SD = 0.008, for 22 variables). This meant that, on average, only 2.8% of the variation in the items could be attributed to between-school effects. Similarly, with the exception of the single NA item, Afraid, the between-country variance was also quite low, ranging between 0.012 and 0.078 (M = 0.042, SD = 0.033). This meant that, on average, only 4.2% of the variance in items could be attributed to country effects. Because of the generally low levels of between-group effects (under 5%), it was decided that single-level confirmatory factor analysis would be used as a general analytical framework for the current study.

Satisfaction with life models

A total of two satisfaction with life models were proposed. The first represented a congeneric model, defined here as the Uni-Dimensional Life-Satisfaction Model (Fig. 1). The second represented a two-factor model, defined here as the Bi-Dimensional Life-Satisfaction Model (Fig. 2).

Fig. 1

Uni-dimensional life satisfaction model. Note. All item-factor loadings are standardized and statistically significant, p <0 .001

Fig. 2

Bi-dimensional life satisfaction model. Note. All item-factor loadings are standardized and statistically significant, p < 0.001

RQ2(a, b): Discriminant and concurrent validity

Table 3 presents the results for RQ2 for discriminant and concurrent validity. Both models pass the minimum requirements for discriminant validity. In terms of concurrent validity, in the two-factor model, the Overall Life-Satisfaction item was slightly more associated with the Satisfaction with Life factor at r = 0.430 (p < 0.001; lower = 0.417, upper = 0.443) than with the Satisfaction with School Factor at r = 0.409 (p < 0.001; lower = 0.397, upper = 0.422). Finally, as both models met the requirements for discriminant validity, both models were examined in terms of overall model fit for RQ2(c).

Table 3 Discriminant and concurrent validity for the uni- and bi-dimensional life-satisfaction models

RQ2(c): Model fit

Table 4 provides the test results for model fit for the two competing Satisfaction with Life models. Only the Bi-Dimensional Life-Satisfaction Model exhibited an overall satisfactory fit to the data (RMSEA = 0.107). Therefore, nested model fit tests were not undertaken. Consequently, it was proposed that, in step 2, the proposed hedonic model would include separate satisfaction with general life and satisfaction with school factors.

Table 4 Model Fit for the Uni- and Bi-dimensional life-satisfaction models

Hedonic model

To extend the Bi-Dimensional Life-Satisfaction Model, we proposed including factors pertaining to both PA and NA in a Four-Factor Hedonic Model. This included a total of 19 items and the following four factors: Satisfaction with Life, Satisfaction with School, PA, and NA (Fig. 3).

Fig. 3

Four-factor hedonic well-being model. Note. All item-factor loadings are standardized and statistically significant, p < 0.001

RQ3(a, b): Discriminant and concurrent validity

Table 5 presents the results for RQ3 for discriminant and concurrent validity. The four-factor Hedonic model passes the minimum requirements for discriminant validity. In terms of concurrent validity, all correlations are positive except for the relationship between NA and the single item OLS (r = − 0.407, p < 0.001, L = − 0.420, U = − 0.393).

Table 5 Discriminant validity and concurrent validity for the four-factor hedonic model

RQ3(c): Model fit

Table 4 provides the test results for model fit for the four-factor hedonic model. Results suggest that the Four-Factor Hedonic Model exhibited a satisfactory fit to the data. Therefore, it was proposed that, in step 3, the integrative model would include a total of five factors.

Integrative model

The proposed Integrative model included a total of 22 items and the following five factors: Satisfaction with Life, Satisfaction with School, PA, NA, and Eudemonia (Fig. 4).

Fig. 4

Five-factor integrative well-being model. Note. All item-factor loadings are standardized and statistically significant, p <0 .001

RQ4(a, b): Discriminant and concurrent validity

Table 6 presents the results for RQ4 for discriminant and concurrent validity. The Five-Factor Integrative Model passes the minimum requirements for discriminant validity. In terms of concurrent validity, all correlations are positive except for the relationship between NA and the single item OLS (r = − 0.406; p < 0.001, L = − 0.420, U = − 0.393).

Table 6 Discriminant validity and concurrent validity for the five-factor integrative model

RQ4(c): Model fit

Table 4 provides the results for the tests for model fit for the Five-Factor Integrative model. Results suggest that the Five-Factor Integrative Model exhibited a satisfactory fit to the data. Therefore, consequent measurement invariance tests were undertaken on that final proposed model. It should be noted that, compared to the four-factor hedonic model, the five-factor integrative model has slightly better CFI, TLI, RMSEA, SRMR, and Gamma Hat fit indices.

Measurement invariance

Results for the RQ5, the country-by-country measurement invariance tests of the integrative model, are presented in Table 7. For gender, the configural invariance test resulted in CFI = 0.950 and RMSEA = 0.050, while the metric invariance test resulted in CFI = 0.001 and RMSEA = 0.001, and the scalar invariance test resulted in CFI = 0.005 and RMSEA = 0.001 therefore meeting the requirements for scale invariance. However, for all 36 inter-country tests of measurement invariance, metric invariance was only reached nine times (25%), and scalar invariance was reached once (2.8%).

Table 7 Country-by-Country Tests of Measurement Invariance

Discussion

Much debate exists concerning the factorial structure/dimensionality of student well-being. We contribute to this debate by drawing on PISA 2018 data from a total of 61,722 students, 2528 schools, and nine countries. For our investigation, we test multiple associated measurement models for convergent, discriminant, and concurrent validity, model fit, and measurement invariance. Single-level measurement models were tested in the current study because of the low between-school and between-country effects. Uni-dimensional and bi-dimensional life-satisfaction models were first compared, and the bi-dimensional model was considered the preferable life-satisfaction model. The four-factor hedonic model (inclusive of satisfaction with life and school, and PA and NA factors) and the five-factor integrative model (with the addition of eudaemonia) yielded satisfactory results. However, due to its comparative sound psychometric properties, we prefer the Five-Factor Integrative Model of Student Well-being that supports a broader conceptualization of student well-being that includes life-satisfaction, both positive and negative affect, and eudaemonia. Invariance tests were performed on the final integrative model. While configural, metric, and scalar invariance tests yielded satisfactory results for gender, metric invariance was attained only 25% of the time, and scalar invariance was only attained 2.8% of the time for the country-by-country comparisons.

Variance between schools and countries in student well-being

The results of the current study show that there exists very little variation in student well-being both between schools and between countries. On average, only 2.8% of the variation in the items could be explained by between-school effects, while only 4.2% of the variance in items could be attributed to country effects. This suggests that variation in student well-being exists predominantly at the individual student level. Although we included only nine countries in our analyses, our results are in line with that of Govorova et al., (2020a, 2020b), who analyzed the whole PISA sample and found that only 5–9% of the variance in student well-being was attributable to school-level effects, which was consistent across countries. Accordingly, they suggested that school-level interventions do not have a strong influence on the well-being of adolescents. The weak influence of schools on student well-being was attributed by Govorova et al., (2020a, 2020b) to a lack of policies and tools for sustaining students' well-being and the limited time allocated to non-academic and socio-emotional aspects of education at schools. While our findings support little difference between the average level of well-being between schools, further research should be undertaken to understand this phenomenon.

Life-satisfaction models

Unidimensional life-satisfaction models assume that a single aggregate score can represent children's overall happiness, and this approach has been common in studies devoted to understanding student life-satisfaction in PISA (Govorova et al., 2020a, 2020b; OECD, 2019b; Seon & Smith-Adcock, 2021). However, over the past two decades, there has been a shift from unidimensional to multidimensional measurement of life-satisfaction because global measures can often mask nuances relating to essential sub-dimensions of life-satisfaction (Clarke, 2020). We argue that the measurement of adolescents' life-satisfaction requires a more multi-dimensional approach incorporating children's evaluations of school life alongside various aspects of their lives outside school. Understanding how each of these aspects pertains to educational outcomes is essential. As an original contribution of the current study, we unearth and make the case for different elements of the life-satisfaction scale used in PISA 2018. Three items regarding students’ evaluations of school life gathered under a subdimension called ‘‘Satisfaction with School’’, and all the other items regarding students' assessment of their lives outside school (self, family, friends, and environment) came together under the dimension of ‘‘Satisfaction with General Life’’. We believe that the application of the Bi-Dimensional Life-Satisfaction model has the potential to enhance the implications drawn from the results of further analyses of the PISA life-satisfaction survey. One example might be the discernment of effects associated with higher levels of student life-satisfaction at school, as opposed to at home. Effects for which governmental policy may be able to regulate.

The findings in the study are relevant to future studies intending to measure and integrate findings related to student well-being. Our research supports the concurrent validity of the Satisfaction with General Life (r = 0.43, 95% CIs: 0.42, 0.44) and Satisfaction with School (r = 0.41, 95% CIs: 0.40, 0.42) factors given the medium-size correlations with the single OLS item. While the dimension of ‘‘satisfaction with school’’ includes three items focusing on school factors, the dimension of ‘‘satisfaction with life’’ compositely measures student’ perceptions of the factors outside school, such as friends, family, environment, and self. We would also note that our analysis of the correlation confidence intervals suggests students' Satisfaction with Life is slightly more aligned with their Overall Life-Satisfaction. This certainly makes sense given that satisfaction with friends, family, living environment, and self constitutes the predominant part of adolescent life.

Four-factor hedonic well-being model

The results of this study support the discriminant and convergent validity of a four-factor hedonic model of student well-being using PISA 2018 data. Therefore, we argue that PISA data can be used to successfully measure student well-being from a hedonic perspective tackling both cognitive (general life-satisfaction and satisfaction with school factors) and affective features (PA and NA). This further extends previous research that has limited the measurement of student well-being to cognitive facets (e.g., Marquez & Long, 2021; Marquez & Main, 2020; Tang, 2019; Yin-Nei Cho, 2019). In line with subjective well-being models, discriminant validity results in this study (i.e., low AVE, HTMT, and shared variance scores) provides supporting evidence that PA and NA are (negatively) related and independent constructs. This has been demonstrated to be particularly true when PA and NA refer to a long-time span (Diener & Emmons, 1984; OECD, 2019b), such as in the case of PISA items, which does not confine the experience of the positive and negative emotions to a specific timeframe. Interestingly, the single OLS item correlation with PA is slightly higher than with the other dimensions of the hedonic well-being model, including the satisfaction with life and school factors. This may be since both the single OLS and the PA items assess students' cognitive and affective accounts of their life in general. In contrast, the satisfaction with life and school factors address students’ cognitive evaluations of specific life domains. Moreover, experiencing positive emotions results in tangible benefits, including better marriages, fewer sleep problems, higher income, and a longer life span, which can overall contribute to a more positive evaluation of the overall quality of life as a whole (Lyubomirsky et al., 2005). Nevertheless, cross-country variability has been observed in the relationship between OLS and PA in children and adolescents depending on the level of industrialization (Casas et al., 2020). Therefore, additional research is needed to advance the understanding of the relationship between PA (and NA) and OLS in other contexts.

Five-factor integrative well-being model

Previous researchers emphasized the need to integrate the hedonic and eudemonic components to evaluate the well-being of children and adolescents appropriately (Gallagher et al., 2009; Sarriera & Bedin, 2017). Our Five-Factor Integrative Model of Student Well-being, which fitted the data best, supports a broader conceptualization of student well-being that integrates cognitive (satisfaction with life and satisfaction with school), affective (PA and NA), and eudaemonic student evaluations. Our results suggest that students' eudemonic evaluations have significant correlations with all the cognitive (satisfaction with life and satisfaction with school) and affective (PA and NA) components of hedonic well-being, corroborating the findings of the previous studies (Gallagher et al., 2009; Linley et al., 2009; Strelhow et al., 2020). This integrative model is indicative of a multidimensional perspective of well-being where all those hedonic and eudemonic evaluations are part of single core well-being construct (Ben-Arieh et al., 2014; Strelhow et al., 2020). However, the relatively low levels of correlations (between 0.418 and 0.291) between eudemonic well-being and the components of hedonic well-being imply that the two well-being perspectives are still distinct from each other and there are divergence points that need to be considered, which reiterate the contentions of Keyes (2002) and Waterman et al. (2010) that the hedonic and eudemonic evaluations of well-being are conceptually and empirically distinguishable.

In the Integrative Model, eudemonia has the lowest level of correlation with NA (r = − 0.27) while it has relatively higher levels of correlations with satisfaction with life (0.39), satisfaction with school (0.37), and PA (0.47). These results are in line with Jia et al.'s (2021) findings that eudemonia is more strongly associated with the positive aspects of subjective well-being (life-satisfaction and PA) in adolescents. This implies that eudaemonic well-being provides more concrete pathways to positive cognitive and affective evaluations, and, thus, contributes to the happiness of adolescents (DeHaan & Ryan, 2014).

Measurement invariance

In order to interpret (1) differences in factorial means by groups, and (2) correlations in factors between groups, scalar invariance needs to be demonstrated. The invariance tests herein resulted in sequentially satisfactory configural, metric, and scalar invariance for gender, but it was not the case for country-by-country invariance analysis results. Therefore, direct comparisons between group means for factors can only be made for: Hong Kong and the UAE, and Hong Kong and Bulgaria. Certainly, more work could be done to understand the lack of strong invariance for other country combinations.

On a speculative note, the general lack of measurement invariance across cultures may be the result of inherent differences in the way that questions pertaining to well-being are interpreted by young people (Hernández-Torrano, 2020). Alternatively, although the OECD declared (n.d., p.1) that they applied stringent quality-assurance mechanisms in translation, sampling and data collection, and that, as a result, PISA assessments have a high level of reliability and validity, there may still be challenges in the translation of the items in other languages because of translators' wording preferences and their different interpretations in other languages and cultures (see Bray et al., 2020, for example on related PISA constructs). In the future, researchers could conduct further country-by-country comparisons of specific words, emotions, and concepts used in the battery of questions designed to measure student well-being. While no small task, cross-cultural alignment between how these words are interpreted by adolescents may be important to making more viable cross-country group comparisons.

Limitations, future research, and conclusion

An important limitation of our study is that the model that we propose only applies to the nine countries that (1) used the well-being questionnaire and (2) provided feedback for the 10 items on life-satisfaction (items on PA, NA, and eudaimonia are included in the general student questionnaire distributed internationally). Given the limited sample of countries administering the full well-being questionnaire in PISA 2018, more research is needed to make more broader generalizations to populations in other contexts. In addition, alternative measurement models can be tested by other researchers, including the remaining 71 countries that preferred to use only a single item to measure overall life-satisfaction (OLS). The results of the current study suggest that using 10 items to measure life-satisfaction is statistically preferable to using the single OLS item. Therefore, it would be helpful for countries to administer the more comprehensive life-satisfaction survey given the increase in the importance of student well-being around the world.

Another limitation is related to the content validity of the scales used by PISA. None of the scales used by PISA to measure different aspects of well-being is all-encompassing. There are various approaches to measuring different aspects of well-being in the literature, and PISA preferred to use brief scales or even single items (e.g., OLS) to measure those structures. Our results revealed that the ten-item life-satisfaction scale measures life-satisfaction better than the single item OLS. As an original contribution of the current study, we proposed a two-dimension model of life-satisfaction consisting of satisfaction with life and satisfaction with school, which fitted the data well. However, the current life-satisfaction survey does not comprehensively measure all aspects of life-satisfaction such as satisfaction with friends, family, living environment, and self (Huebner, 1994). There are only one or two items regarding each dimension (self, family, friends, and environment) in the current life-satisfaction scale, which cannot provide a detailed assessment of each aspect.

Researchers are also advised to be cautious while interpreting the ‘‘eudaemonic well-being’’ concept using the PISA data. We used the three-item ‘‘meaning in life’’ scale, which provides limited insight into eudaemonic well-being. This scale, which is called ‘‘eudaemonia’’ by the PISA (OECD, 2019a), is not comprehensive and does not cover other aspects of "eudaemonia" such as authenticity, purpose, the pursuit of excellence, self-discovery, perceived development of one's best potentials, and involvement in and enjoyment of activities as personally expressive (Waterman et al., 2010). Besides, the absence of some vital domains like empathy, self-efficacy, adaptability, emotional self-regulation, among others, from the other adolescent well-being scales like PGI (Verma et al., 1983), the child and adolescent wellness scale (Copeland et al., 2010); the social and emotional health survey (Furlong et al., 2014), and the EPOCH (Kern et al., 2016) is a limitation of PISA's conceptualization of well-being among young people that could be addressed in subsequent cycles.

PISA has been criticized for not paying attention to affective domains and focusing too much on cognitive aspects (Cefai & Cavioni, 2015). Therefore, the inclusion of more comprehensive scales in subsequent PISA cycles would be helpful for understanding both the hedonic and eudaemonic well-being of adolescents. Potential expansion to such domains, in addition to others (e.g., Rappleye et al., (2019) Interdependent Happiness Scale, and PISA’s Sense of Belonging Scale (OECD, 2019a), would promote a wider view of education with balanced attention to affective aspects of students' lives in addition to the more prevalent cognitive variables.