Skip to main content

Table 1 Comparing the Sample Representation and Measurement of (Low) Literacy of LEO, PIAAC, and NEPS

From: On the comparability of adults with low literacy across LEO, PIAAC, and NEPS. Methodological considerations and empirical evidence

  LEO PIAAC NEPSa
Sample representation
 Target population Residential population, living in private households in Germany, aged 18 to 64 Residential population, living in private households in Germany, aged 16 to 65 Residential population, living in private households in Germany, aged 24 to 69
 Sampling design Random route procedure, add-on to the AES Registry-based two-stage stratified and clustered random sampling design Registry-based two-stage stratified and clustered random sampling design
 Sample sizeb 8,436 5,465 8,480
 Response rate 52% (AES), 43% LEO (add-on) 55% 73% (wave 3)/77% (wave 5)c
 Weighting Design weights are adjusted to parameters of official statistics Design weights are adjusted to parameters of official statistics and replicate weights (for more details on the exact weighting procedure in PIAAC, see OECD 2013a) Design weights are adjusted to parameters of official statistics
Conceptual equivalence
 Focus Reading and writing
Lower end of the ability spectrum
Reading
Entire ability spectrum
Reading
Entire ability spectrum
 Cognitive requirements   55% access and identify, 29% integrate and interpret, 16% evaluate and reflect 43% finding information in text, 27% drawing text-related conclusions, and 30% reflecting and assessing
 Level Letter, word, sentence and text Text Text
 Medium Audio and print Digital and print Print
 Text contexts Work related, personal Work-related, personal, society and community, education, and training  
 Text functions   Description, narration, exposition, argumentation, instruction, records Informational, instructional, advertising, commenting or argumenting, literacy
 Text format Continuous, non-continuous Continuous, non-continuous, and mixed texts; single and multiple texts Continuous and single
 Item format Closed (writing and reading)
Open-constructed (writing)
Closed
Open-constructed (e.g. highlighting)
Closed
Test design
 Delivery mode Paper–Pencil-Assessment Computer-Based-Assessment (default) or Paper–Pencil-Assessment Paper–Pencil-Assessment
 Test assembly Booklet design with 72 items in total, 10 items for all respondents, additional 20 items out of three booklets for respondents who did not reach minimum number of correct answers Adaptive testing
2/3 of the respondents were administered 20 items in two stages from 58 items in total
One booklet for all respondents, 32 items in total per respondent
 Administration Background questionnaire was administrated before assessment of competencies
No time restriction, on average 25 min
Background questionnaire was administrated before assessment of competencies
No time restriction, on average 30 min
Background questionnaire was administrated after assessment of competencies
28-min time restriction
Scaling
 Scaling model
 Missing responses
 Estimator
1-PL model
Not treated as incorrect
5 Plausible Values
2-PL model
Some types of missing responses were treated as incorrect
10 Plausible Values
1-PL model
Not treated as incorrect
Warm’s mean weighted likelihood estimation (WLE)
Standard setting
 Number of levels 5 proficiency levels with respondents at alpha-level 3 and below as low-literate 5 proficiency levels with respondents at level 1 and below as low-literate 2 proficiency levels with respondents at level 1 as low-literate and no further distinction for literates
 Approach A priori Post hoc A priori
 Response probability 62% 67% 67%
  1. aNEPS Starting Cohort Adults (SC6), wave 3 and 5.
  2. bNote that these numbers refer to the number of participants who participated in the assessment.
  3. cAs the respective waves only includes panelists and no first respondents, it rather indicates respondent’ willingness to continue participating in panel. The initial response rate for NEPS respondents is lower, with response rates around 30% (Hammon et al. 2016).