Articles

Automatically analyzing text responses for exploring gender-specific cognitions in PISA reading

The gender gap in reading literacy is repeatedly found in large-scale assessments. This study compared girls’ and boys’ text responses in a reading test applying natural language processing. For this, a theore...

Authors: Fabian Zehner, Frank Goldhammer and Christine Sälzer

Effect size measures for multilevel models: definition, interpretation, and TIMSS example

Effect size reporting is crucial for interpretation of applied research results and for conducting meta-analysis. However, clear guidelines for reporting effect size in multilevel models have not been provided...

Authors: Julie Lorah

On imputation for planned missing data in context questionnaires using plausible values: a comparison of three designs

This paper extends a recent study by Kaplan and Su (J Educ Behav Stat 41: 51–80, 2016) examining the problem of matrix sampling of context questionnaire scales with respect to the generation of plausible values o...

Authors: David Kaplan and Dan Su

Promoting reading attitudes of girls and boys: a new challenge for educational policy? Multi-group analyses across four European countries

Numerous studies have investigated the relationships between various student, home and contextual factors and reading achievement. However, the relationship between such factors and reading attitudes has been ...

Authors: Sonja Nonte, Lea Hartwich and Ariane S. Willems

How long is the shadow? The relationships of family background to selected adult outcomes: results from PIAAC

Ongoing interest in the relationships between family background and adult outcomes is motivated by concerns regarding the intergenerational transmission of advantage/disadvantage. Currently all countries are f...

Authors: Henry Braun

The dependence on mathematical theory in TIMSS, PISA and TIMSS Advanced test items and its relation to student achievement

This paper discusses a framework for analyzing the dependence on mathematical theory in test items, that is, a framework for discussing to what extent knowledge of mathematical theory is helpful for the studen...

Authors: Arne Hole, Liv Sissel Grønmo and Torgeir Onstad

Personality, cognitive skills and life outcomes: evidence from the Polish follow-up study to PIAAC

There is a growing literature providing evidence on the importance of non-cognitive skills for life outcomes. However, to date there is limited evidence on the gains from incorporating such measures into large...

Authors: Marta Palczyńska and Karolina Świst

The sensitivity of TIMSS country rankings in science achievement to differences in opportunity to learn at classroom level

Fair comparisons of educational systems in large-scale assessments can be made only if the differences in curricula have little impact on the outcomes. This study investigated the sensitivity of science achiev...

Authors: Stephan Daus and Johan Braeken

Teachers’ participation in professional development concerning the implementation of new technologies in class: a latent class analysis of teachers and the relationship with the use of computers, ICT self-efficacy and emphasis on teaching ICT skills

The increasing availability of new technologies in an ever more digitalized world has gained momentum in practically all spheres of life, making technology-related skills a key competence not only in professio...

Authors: Kerstin Drossel and Birgit Eickelmann

Conditioning factors of test-taking engagement in PIAAC: an exploratory IRT modelling approach considering person and item characteristics

A potential problem of low-stakes large-scale assessments such as the Programme for the International Assessment of Adult Competencies (PIAAC) is low test-taking engagement. The present study pursued two goals...

Authors: Frank Goldhammer, Thomas Martens and Oliver Lüdtke

Patterns of students’ computer use and relations to their computer and information literacy: results of a latent class analysis and implications for teaching and learning

Previous studies have shown that there is a complex relationship between students’ computer and information literacy (CIL) and their use of information and communication technologies (ICT) for both recreationa...

Authors: Jeppe Bundsgaard and Julia Gerick

The use of test scores from large-scale assessment surveys: psychometric and statistical considerations

Economists are making increasing use of measures of student achievement obtained through large-scale survey assessments such as NAEP, TIMSS, and PISA. The construction of these measures, employing plausible va...

Authors: Henry Braun and Matthias von Davier

Performance decline in low-stakes educational assessments: different mixture modeling approaches

In low-stakes educational assessments, test takers might show a performance decline (PD) on end-of-test items. PD is a concern in educational assessments, especially when groups of students are to be compared ...

Authors: Marit K. List, Alexander Robitzsch, Oliver Lüdtke, Olaf Köller and Gabriel Nagy

Beyond the mean estimate: a quantile regression analysis of inequalities in educational outcomes using INVALSI survey data

The number of studies addressing issues of inequality in educational outcomes using cognitive achievement tests and variables from large-scale assessment data has increased. Here the value of using a quantile ...

Authors: Antonella Costanzo and Marta Desimoni

Same but different? Measurement invariance of the PIAAC motivation-to-learn scale across key socio-demographic groups

Data from the Programme for the International Assessment of Adult Competencies (PIAAC) revealed that countries systematically differ in their respondents’ literacy, numeracy, and problem solving in technology-...

Authors: Julia Gorges, Tobias Koch, Débora B. Maehler and Judith Offerhaus

Skills, earnings, and employment: exploring causality in the estimation of returns to skills

Ample evidence indicates that a person’s human capital is important for success on the labor market in terms of both wages and employment prospects. However, unlike the efforts to identify the impact of school...

Authors: Franziska Hampf, Simon Wiederhold and Ludger Woessmann

PIAAC: a new design for a new era

As the largest and most innovative international assessment of adults, PIAAC marks an inflection point in the evolution of large-scale comparative assessments. PIAAC grew from the foundation laid by surveys th...

Authors: Irwin Kirsch and Mary Louise Lennon

Assuming measurement invariance of background indicators in international comparative educational achievement studies: a challenge for the interpretation of achievement differences

Large-scale cross-national studies designed to measure student achievement use different social, cultural, economic and other background variables to explain observed differences in that achievement. Prior to ...

Authors: Heike Wendt, Daniel Kasper and Matthias Trendtel

Coverage of the migrant population in large-scale assessment surveys. Experiences from PIAAC in Germany

European countries, and especially Germany, are currently very much affected by human migration flows, with the result that the task of integration has become a challenge. Only very little empirical evidence o...

Authors: Débora B. Maehler, Silke Martin and Beatrice Rammstedt

A structural equation modeling approach for examining position effects in large-scale assessments

Position effects may occur in both paper–pencil tests and computerized assessments when examinees respond to the same test items located in different positions on the test. To examine position effects in large...

Authors: Okan Bulut, Qi Quo and Mark J. Gierl

The relevance of the school socioeconomic composition and school proportion of repeaters on grade repetition in Brazil: a multilevel logistic model of PISA 2012

The paper extends the literature on grade repetition in Brazil by (a) describing and synthesizing the main research findings and contributions since 1940, (b) enlarging the understanding of the inequity mechan...

Authors: Maria Eugénia Ferrão, Patrícia Mota Costa and Daniel Abud Seabra Matos

Degrees of competency: the relationship between educational qualifications and adult skills across countries

Educational qualifications and literacy skills are highly related. This is not surprising as it is one aim of educational systems to equip individuals with competencies necessary to take part in society. Becau...

Authors: Natascha Massing and Silke L. Schneider

Evaluating the risk of nonresponse bias in educational large-scale assessments with school nonresponse questionnaires: a theoretical study

Survey participation rates can have a direct impact on the validity of the data collected since nonresponse always holds the risk of bias. Therefore, the International Association for the Evaluation of Educati...

Authors: Sabine Meinck, Diego Cortes and Sabine Tieck

Personality, competencies, and life outcomes: results from the German PIAAC longitudinal study

The present paper investigates the power of personality to predict important life outcomes in the context of the Programme for the International Assessment of Adult Competencies (PIAAC). On the most global lev...

Authors: Beatrice Rammstedt, Daniel Danner and Clemens Lechner

The PIAAC longitudinal study in Germany: rationale and design

In Germany, the respondents who had participated in the 2012 survey of the Programme for the International Assessment of Adult Competencies (PIAAC) were re-approached for the panel study PIAAC-L. PIAAC-L aims ...

Authors: Beatrice Rammstedt, Silke Martin, Anouk Zabal, Claus Carstensen and Jürgen Schupp

School-level predictors for the use of ICT in schools and students’ CIL in international comparison

The increasing relevance of information and communication technologies (ICT) and society’s transition towards an information or knowledge society have led to the emergence of new challenges for schools and sch...

Authors: Julia Gerick, Birgit Eickelmann and Wilfried Bos

Home resources as a measure of socio-economic status in Ghana

In large scale international assessment studies, questionnaires are typical used to query students’ home possessions. Composite scores are computed from responses to the home resource questionnaires and are us...

Authors: Emmanuel Adu-tutu Bofah and Markku S. Hannula

Proficient beyond borders: assessing non-native speakers in a native speakers’ framework

English language proficiency is considered a basic skill that students from different language backgrounds are expected to master, independent of whether they are native or non-native speakers. Tests that meas...

Authors: Johanna Fleckenstein, Michael Leucht, Hans Anand Pant and Olaf Köller

Comparing DIF methods for data with dual dependency

The current study compared four differential item functioning (DIF) methods to examine their performances in terms of accounting for dual dependency (i.e., person and item clustering effects) simultaneously by...

Authors: Ying Jin and Minsoo Kang

The relation between teachers’ emphasis on the development of students’ digital information and communication skills and computer self-efficacy: the moderating roles of age and gender

Teachers’ integration of information and communication technology (ICT) has been widely studied, given that digital competence is considered to be a crucial outcome of twenty first century education. In this c...

Authors: Fazilat Siddiq and Ronny Scherer

Measurement of job motivation in TEDS-M: testing for invariance across countries and cultures

The paper presents the challenges of cross-country and cross-cultural research on the motivation to become a mathematics teacher based on data from the “Teacher Education and Development Study in Mathematics (...

Authors: Christin Laschke and Sigrid Blömeke

The relationship between students’ use of ICT for social communication and their computer and information literacy

This study investigates the relationship between students’ use of information and communication technology (ICT) for social communication and their computer and information literacy (CIL) scores. It also exami...

Authors: Meral Alkan and Sabine Meinck

Subject-specific strength and weaknesses of fourth-grade students in Europe: a comparative latent profile analysis of multidimensional proficiency patterns based on PIRLS/TIMSS combined 2011

In 2011 the Progress in International Reading Literacy Study (PIRLS) and the Trends in International Mathematics and Science Study (TIMSS) were conducted at fourth grade in a number of participating countries ...

Authors: Heike Wendt and Daniel Kasper

Context factors and student achievement in the IEA studies: evidence from TIMSS

The present study investigates what factors related to the school context influence student achievement on TIMSS mathematics tests across countries. A systematic review of the literature on PIRLS, TIMSS, and I...

Authors: Elisa Caponera and Bruno Losito

A statistical analysis of the characteristics of the intended curriculum for Japanese primary science and its relationship to the attained curriculum

This study statistically investigates the characteristics of the intended curriculum for Japanese primary science, focusing on the learning content. The study used the TIMSS 2011 Grade 4 Curriculum Questionnai...

Authors: Kenji Matsubara, Yasuhito Hagiwara and Yuji Saruta

Determinants of country differences in effects of parental education on children’s academic achievement

In a previous study, the total, direct and indirect effects of parental education on reading, mathematics and science achievement have been estimated for Grade 4 pupils of 37 countries that participated in PIR...

Authors: Kajsa Yang Hansen and Jan-Eric Gustafsson

Assessment of fit of item response theory models used in large-scale educational survey assessments

Latent regression models are used for score-reporting purposes in large-scale educational survey assessments such as the National Assessment of Educational Progress (NAEP) and Trends in International Mathemati...

Authors: Peter W. van Rijn, Sandip Sinharay, Shelby J. Haberman and Matthew S. Johnson

Who likes to learn new things: measuring adult motivation to learn with PIAAC data from 21 countries

Despite the importance of lifelong learning as a key to individual and societal prosperity, we know little about adult motivation to engage in learning across the lifespan. Building on educational psychologica...

Authors: Julia Gorges, Débora B. Maehler, Tobias Koch and Judith Offerhaus

Introduction to special issue on quasi-causal methods

Authors: Leslie Rutkowski

Causal inference with large-scale assessments in education from a Bayesian perspective: a review and synthesis

This paper reviews recent research on causal inference with large-scale assessments in education from a Bayesian perspective. I begin by adopting the potential outcomes model of Rubin (J Educ Psychol 66:688-701,

Authors: David Kaplan

Causal inferences with large scale assessment data: using a validity framework

To answer the calls for stronger evidence by the policy community, educational researchers and their associated organizations increasingly demand more studies that can yield causal inferences. International la...

Authors: David Rutkowski and Ginette Delandshere

Is computer availability at home causally related to reading achievement in grade 4? A longitudinal difference in differences approach to IEA data from 1991 to 2006

Research on effects of home computer use on children’s development of cognitive abilities and skills has yielded conflicting results, with some studies showing positive effects, others no effects, and yet othe...

Authors: Monica Rosén and Jan-Eric Gustafsson

Introduction to instrumental variables and their application to large-scale assessment data

In the social sciences, estimating causal effects is particularly difficult. Gold standards are set by randomized experiments in many cases expensive, unenforceable for ethical and practical reasons. Recent re...

Authors: Artur Pokropek

Does non-participation in preschool affect children’s reading achievement? International evidence from propensity score analyses

While expectations are high for early childhood education to support students’ reading literacy, research findings are inconclusive. The purpose of the study is to estimate the effect of preschool non-partici...

Authors: Nina Hogrebe and Rolf Strietholt

A multilevel analysis of Swedish and Norwegian students’ overall and digital reading performance with a focus on equity aspects of education

Influence of external factors in general, and socioeconomic background factors in particular, on traditional reading performance has been extensively researched and debated. While traditional reading is well i...

Authors: Maria A. Rasmusson

Gender differences in variability and extreme scores in an international context

This study examines gender differences in the variability of student performance in reading, mathematics and science. Twelve databases from IEA and PISA were used to analyze gender differences within an intern...

Authors: Ariane Baye and Christian Monseur

TIMSS data in an African comparative perspective: Investigating the factors influencing achievement in mathematics and their psychometric properties

Relationships among motivational constructs from the 2011 Trends in International Mathematics and Science Study (TIMSS 2011) were investigated for eight-graders in all the five participating African countries,...

Authors: Emmanuel Adu-tutu Bofah and Markku S. Hannula

Affective characteristics and mathematics performance in Indonesia, Malaysia, and Thailand: what can PISA 2012 data tell us?

The results of the Programme for International Student Assessment (PISA) 2012 showed that Indonesia, Malaysia, and Thailand underperformed and were positioned in the bottom third out of 65 participating countr...

Authors: Lei Mee Thien, I. Gusti Ngurah Darmawan and Mei Yean Ong

Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

In this paper, we consider a two-level multidimensional item response model that examines country differences in extreme response style (ERS) as a possible cause for the achievement-attitude paradox in PISA 20...

Authors: Yi Lu and Daniel M. Bolt

Teaching for conceptual understanding: A cross-national comparison of the relationship between teachers’ instructional practices and student achievement in mathematics

Educators, researchers and policy-makers worldwide continue to struggle to understand the changes needed for improving educational outcomes and attainment for students, particularly in content areas such as ma...

Authors: Laura M O’Dwyer, Yang Wang and Katherine A Shields