Articles

DIF as a pedagogical tool: analysis of item characteristics in ICILS to understand what students are struggling with

International large-scale assessments like international computer and information literacy study (ICILS) (Fraillon et al. in International Association for the Evaluation of Educational Achievement (IEA), 2015)...

Authors: Jeppe Bundsgaard

Trends in gender gaps: using 20 years of evidence from TIMSS

Potential differences in achievement between female and male students have always been an interesting topic in educational research, as well as having policy and economic implications. This study provides an o...

Authors: Sabine Meinck and Falk Brese

Generational status, immigrant concentration and academic achievement: comparing first and second-generation immigrants with third-plus generation students

Immigrants and their children are the fastest-growing demographic group in the United States, and schools are often the first social institution young immigrants engage with on a sustained basis. As such, the ...

Authors: Margarita Pivovarova and Jeanne M. Powers

Indicators of (in)tolerance toward immigrants among European youth: an assessment of measurement invariance in ICCS 2016

Promoting tolerance is an important goal of European education policies focused on education for democratic citizenship and human rights. In this article, we argue that cross-cultural comparability must be emp...

Authors: Maria Magdalena Isac, Laura Palmerio and M. P. C. (Greetje) van der Werf

Predictors of individual performance changes related to item positions in PISA assessments

Item position effects have been a common concern in large-scale assessments as changing the order of items in booklets may have an undesired effect on test performance. If every test taker would be affected by...

Authors: Qian Wu, Dries Debeer, Janine Buchholz, Johannes Hartig and Rianne Janssen

Relationships among locus of control, learned helpless, and mathematical literacy in PISA 2012: focus on Korea and Finland

The purpose of this research is to gather empirical evidence for attribution theory (Weiner in J Educ Psychol 71(1):3–25. https://​doi.​org/​10.​1037/​0022-0663.​71.​1.​3

Authors: Jihyun Hwang

The standard setting process: validating interpretations of stakeholders

Stakeholders’ interpretations of the findings of large-scale educational assessments can influence important decisions. In the context of educational assessment, standard-setting remains an especially critical...

Authors: Nele Kampa, Helene Wagner and Olaf Köller

Sex differences in variability across nations in reading, mathematics and science: a meta-analytic extension of Baye and Monseur (2016)

A recent study by Baye and Monseur (Large Scale Assess Educ 4:1–16, 2016) using large, international educational data sets suggest that the “greater male variation hypothesis” is well supported. Males are often o...

Authors: Helen Gray, Andrew Lyth, Catherine McKenna, Susan Stothard, Peter Tymms and Lee Copping

Instructional quality: catalyst or pitfall in educational systems’ aim for high achievement and equity? An answer based on multilevel SEM analyses of TIMSS 2015 data in Flanders (Belgium), Germany, and Norway

In the Trends in Mathematics and Science Study (TIMSS) 2015, the educational systems from the Dutch speaking part of Belgium (Flanders), Germany, and Norway included scales capturing three dimensions of instru...

Authors: Kim Bellens, Jan Van Damme, Wim Van Den Noortgate, Heike Wendt and Trude Nilsen

lsasim: an R package for simulating large-scale assessment data

This article provides an overview of the R package lsasim, designed to facilitate the generation of data that mimics a large scale assessment context. The package features functions for simulating achievement dat...

Authors: Tyler H. Matta, Leslie Rutkowski, David Rutkowski and Yuan-Ling Liaw

Introduction to the special issue on Results, Methodological Aspects, and Advancements of the Programme for the International Assessment of Adult Competencies (PIAAC)

Authors: Anja Perry, Débora B. Maehler and Beatrice Rammstedt

Socio-economic gaps in subject interest: the mediating role of parental cognitive involvement

This paper examines the mediating role of parental literacy and numeracy involvement before the child entered school in the relationship between family socio-economic status (SES) and students’ interest in rea...

Authors: Daniel H. Caro

School students’ beliefs about abilities and perspective-taking over time

Beliefs about one’s own abilities and perspective-taking are essential for appropriate behavior in professional and social life, cooperative learning, and situations were conflict is occurring. The social comp...

Authors: Anett Wolgast

The TIMSS 2019 Item Equivalence Study: examining mode effects for computer-based assessment and implications for measuring trends

TIMSS 2019 is the first assessment in the TIMSS transition to a computer-based assessment system, called eTIMSS. The TIMSS 2019 Item Equivalence Study was conducted in advance of the field test in 2017 to exam...

Authors: Bethany Fishbein, Michael O. Martin, Ina V. S. Mullis and Pierre Foy

The role of reading motivation, self-efficacy, and home influence in students’ literacy achievement: a preliminary examination of fourth graders in Abu Dhabi

This study aimed to identify motivation and home influence factors that predict reading literacy achievement of grade 4 students in Abu Dhabi It drew on the Progress in International Reading Literacy Study (P...

Authors: Guang Yang, Masood Badri, Asma Al Rashedi and Karima Almazroui

Measuring the signaling value of educational degrees: secondary education systems and the internal homogeneity of educational groups

By providing high-quality, internationally comparable data on the cognitive skills of working-age adults, the Programme for the International Assessment of Adult Competencies (PIAAC) offers great potential for...

Authors: Jan Paul Heisig

Effect size measures for multilevel models: definition, interpretation, and TIMSS example

Effect size reporting is crucial for interpretation of applied research results and for conducting meta-analysis. However, clear guidelines for reporting effect size in multilevel models have not been provided...

Authors: Julie Lorah

Automatically analyzing text responses for exploring gender-specific cognitions in PISA reading

The gender gap in reading literacy is repeatedly found in large-scale assessments. This study compared girls’ and boys’ text responses in a reading test applying natural language processing. For this, a theore...

Authors: Fabian Zehner, Frank Goldhammer and Christine Sälzer

On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs

This paper extends a recent study by Kaplan and Su (J Educ Behav Stat 41: 51–80, 2016) examining the problem of matrix sampling of context questionnaire scales with respect to the generation of plausible values o...

Authors: David Kaplan and Dan Su

Promoting reading attitudes of girls and boys: a new challenge for educational policy? Multi-group analyses across four European countries

Numerous studies have investigated the relationships between various student, home and contextual factors and reading achievement. However, the relationship between such factors and reading attitudes has been ...

Authors: Sonja Nonte, Lea Hartwich and Ariane S. Willems

How long is the shadow? The relationships of family background to selected adult outcomes: results from PIAAC

Ongoing interest in the relationships between family background and adult outcomes is motivated by concerns regarding the intergenerational transmission of advantage/disadvantage. Currently all countries are f...

Authors: Henry Braun

The dependence on mathematical theory in TIMSS, PISA and TIMSS Advanced test items and its relation to student achievement

This paper discusses a framework for analyzing the dependence on mathematical theory in test items, that is, a framework for discussing to what extent knowledge of mathematical theory is helpful for the studen...

Authors: Arne Hole, Liv Sissel Grønmo and Torgeir Onstad

Personality, cognitive skills and life outcomes: evidence from the Polish follow-up study to PIAAC

There is a growing literature providing evidence on the importance of non-cognitive skills for life outcomes. However, to date there is limited evidence on the gains from incorporating such measures into large...

Authors: Marta Palczyńska and Karolina Świst

The sensitivity of TIMSS country rankings in science achievement to differences in opportunity to learn at classroom level

Fair comparisons of educational systems in large-scale assessments can be made only if the differences in curricula have little impact on the outcomes. This study investigated the sensitivity of science achiev...

Authors: Stephan Daus and Johan Braeken

Teachers’ participation in professional development concerning the implementation of new technologies in class: a latent class analysis of teachers and the relationship with the use of computers, ICT self-efficacy and emphasis on teaching ICT skills

The increasing availability of new technologies in an ever more digitalized world has gained momentum in practically all spheres of life, making technology-related skills a key competence not only in professio...

Authors: Kerstin Drossel and Birgit Eickelmann

Conditioning factors of test-taking engagement in PIAAC: an exploratory IRT modelling approach considering person and item characteristics

A potential problem of low-stakes large-scale assessments such as the Programme for the International Assessment of Adult Competencies (PIAAC) is low test-taking engagement. The present study pursued two goals...

Authors: Frank Goldhammer, Thomas Martens and Oliver Lüdtke

Patterns of students’ computer use and relations to their computer and information literacy: results of a latent class analysis and implications for teaching and learning

Previous studies have shown that there is a complex relationship between students’ computer and information literacy (CIL) and their use of information and communication technologies (ICT) for both recreationa...

Authors: Jeppe Bundsgaard and Julia Gerick

The use of test scores from large-scale assessment surveys: psychometric and statistical considerations

Economists are making increasing use of measures of student achievement obtained through large-scale survey assessments such as NAEP, TIMSS, and PISA. The construction of these measures, employing plausible va...

Authors: Henry Braun and Matthias von Davier

Performance decline in low-stakes educational assessments: different mixture modeling approaches

In low-stakes educational assessments, test takers might show a performance decline (PD) on end-of-test items. PD is a concern in educational assessments, especially when groups of students are to be compared ...

Authors: Marit K. List, Alexander Robitzsch, Oliver Lüdtke, Olaf Köller and Gabriel Nagy

Beyond the mean estimate: a quantile regression analysis of inequalities in educational outcomes using INVALSI survey data

The number of studies addressing issues of inequality in educational outcomes using cognitive achievement tests and variables from large-scale assessment data has increased. Here the value of using a quantile ...

Authors: Antonella Costanzo and Marta Desimoni

Same but different? Measurement invariance of the PIAAC motivation-to-learn scale across key socio-demographic groups

Data from the Programme for the International Assessment of Adult Competencies (PIAAC) revealed that countries systematically differ in their respondents’ literacy, numeracy, and problem solving in technology-...

Authors: Julia Gorges, Tobias Koch, Débora B. Maehler and Judith Offerhaus

Skills, earnings, and employment: exploring causality in the estimation of returns to skills

Ample evidence indicates that a person’s human capital is important for success on the labor market in terms of both wages and employment prospects. However, unlike the efforts to identify the impact of school...

Authors: Franziska Hampf, Simon Wiederhold and Ludger Woessmann

PIAAC: a new design for a new era

As the largest and most innovative international assessment of adults, PIAAC marks an inflection point in the evolution of large-scale comparative assessments. PIAAC grew from the foundation laid by surveys th...

Authors: Irwin Kirsch and Mary Louise Lennon

Assuming measurement invariance of background indicators in international comparative educational achievement studies: a challenge for the interpretation of achievement differences

Large-scale cross-national studies designed to measure student achievement use different social, cultural, economic and other background variables to explain observed differences in that achievement. Prior to ...

Authors: Heike Wendt, Daniel Kasper and Matthias Trendtel

Coverage of the migrant population in large-scale assessment surveys. Experiences from PIAAC in Germany

European countries, and especially Germany, are currently very much affected by human migration flows, with the result that the task of integration has become a challenge. Only very little empirical evidence o...

Authors: Débora B. Maehler, Silke Martin and Beatrice Rammstedt

A structural equation modeling approach for examining position effects in large-scale assessments

Position effects may occur in both paper–pencil tests and computerized assessments when examinees respond to the same test items located in different positions on the test. To examine position effects in large...

Authors: Okan Bulut, Qi Quo and Mark J. Gierl

The relevance of the school socioeconomic composition and school proportion of repeaters on grade repetition in Brazil: a multilevel logistic model of PISA 2012

The paper extends the literature on grade repetition in Brazil by (a) describing and synthesizing the main research findings and contributions since 1940, (b) enlarging the understanding of the inequity mechan...

Authors: Maria Eugénia Ferrão, Patrícia Mota Costa and Daniel Abud Seabra Matos

Degrees of competency: the relationship between educational qualifications and adult skills across countries

Educational qualifications and literacy skills are highly related. This is not surprising as it is one aim of educational systems to equip individuals with competencies necessary to take part in society. Becau...

Authors: Natascha Massing and Silke L. Schneider

School-level predictors for the use of ICT in schools and students’ CIL in international comparison

The increasing relevance of information and communication technologies (ICT) and society’s transition towards an information or knowledge society have led to the emergence of new challenges for schools and sch...

Authors: Julia Gerick, Birgit Eickelmann and Wilfried Bos

The PIAAC longitudinal study in Germany: rationale and design

In Germany, the respondents who had participated in the 2012 survey of the Programme for the International Assessment of Adult Competencies (PIAAC) were re-approached for the panel study PIAAC-L. PIAAC-L aims ...

Authors: Beatrice Rammstedt, Silke Martin, Anouk Zabal, Claus Carstensen and Jürgen Schupp

Evaluating the risk of nonresponse bias in educational large-scale assessments with school nonresponse questionnaires: a theoretical study

Survey participation rates can have a direct impact on the validity of the data collected since nonresponse always holds the risk of bias. Therefore, the International Association for the Evaluation of Educati...

Authors: Sabine Meinck, Diego Cortes and Sabine Tieck

Personality, competencies, and life outcomes: results from the German PIAAC longitudinal study

The present paper investigates the power of personality to predict important life outcomes in the context of the Programme for the International Assessment of Adult Competencies (PIAAC). On the most global lev...

Authors: Beatrice Rammstedt, Daniel Danner and Clemens Lechner

Home resources as a measure of socio-economic status in Ghana

In large scale international assessment studies, questionnaires are typical used to query students’ home possessions. Composite scores are computed from responses to the home resource questionnaires and are us...

Authors: Emmanuel Adu-tutu Bofah and Markku S. Hannula

Proficient beyond borders: assessing non-native speakers in a native speakers’ framework

English language proficiency is considered a basic skill that students from different language backgrounds are expected to master, independent of whether they are native or non-native speakers. Tests that meas...

Authors: Johanna Fleckenstein, Michael Leucht, Hans Anand Pant and Olaf Köller

Comparing DIF methods for data with dual dependency

The current study compared four differential item functioning (DIF) methods to examine their performances in terms of accounting for dual dependency (i.e., person and item clustering effects) simultaneously by...

Authors: Ying Jin and Minsoo Kang

The relation between teachers’ emphasis on the development of students’ digital information and communication skills and computer self-efficacy: the moderating roles of age and gender

Teachers’ integration of information and communication technology (ICT) has been widely studied, given that digital competence is considered to be a crucial outcome of twenty first century education. In this c...

Authors: Fazilat Siddiq and Ronny Scherer

Measurement of job motivation in TEDS-M: testing for invariance across countries and cultures

The paper presents the challenges of cross-country and cross-cultural research on the motivation to become a mathematics teacher based on data from the “Teacher Education and Development Study in Mathematics (...

Authors: Christin Laschke and Sigrid Blömeke

The relationship between students’ use of ICT for social communication and their computer and information literacy

This study investigates the relationship between students’ use of information and communication technology (ICT) for social communication and their computer and information literacy (CIL) scores. It also exami...

Authors: Meral Alkan and Sabine Meinck

Subject-specific strength and weaknesses of fourth-grade students in Europe: a comparative latent profile analysis of multidimensional proficiency patterns based on PIRLS/TIMSS combined 2011

In 2011 the Progress in International Reading Literacy Study (PIRLS) and the Trends in International Mathematics and Science Study (TIMSS) were conducted at fourth grade in a number of participating countries ...

Authors: Heike Wendt and Daniel Kasper

A statistical analysis of the characteristics of the intended curriculum for Japanese primary science and its relationship to the attained curriculum

This study statistically investigates the characteristics of the intended curriculum for Japanese primary science, focusing on the learning content. The study used the TIMSS 2011 Grade 4 Curriculum Questionnai...

Authors: Kenji Matsubara, Yasuhito Hagiwara and Yuji Saruta