An IERI – International Educational Research Institute Journal
Table 1 Comparing the Sample Representation and Measurement of (Low) Literacy of LEO, PIAAC, and NEPS
| Â | LEO | PIAAC | NEPSa |
|---|---|---|---|
Sample representation | |||
 Target population | Residential population, living in private households in Germany, aged 18 to 64 | Residential population, living in private households in Germany, aged 16 to 65 | Residential population, living in private households in Germany, aged 24 to 69 |
 Sampling design | Random route procedure, add-on to the AES | Registry-based two-stage stratified and clustered random sampling design | Registry-based two-stage stratified and clustered random sampling design |
 Sample sizeb | 8,436 | 5,465 | 8,480 |
 Response rate | 52% (AES), 43% LEO (add-on) | 55% | 73% (wave 3)/77% (wave 5)c |
 Weighting | Design weights are adjusted to parameters of official statistics | Design weights are adjusted to parameters of official statistics and replicate weights (for more details on the exact weighting procedure in PIAAC, see OECD 2013a) | Design weights are adjusted to parameters of official statistics |
Conceptual equivalence | |||
 Focus | Reading and writing Lower end of the ability spectrum | Reading Entire ability spectrum | Reading Entire ability spectrum |
 Cognitive requirements |  | 55% access and identify, 29% integrate and interpret, 16% evaluate and reflect | 43% finding information in text, 27% drawing text-related conclusions, and 30% reflecting and assessing |
 Level | Letter, word, sentence and text | Text | Text |
 Medium | Audio and print | Digital and print | |
 Text contexts | Work related, personal | Work-related, personal, society and community, education, and training |  |
 Text functions |  | Description, narration, exposition, argumentation, instruction, records | Informational, instructional, advertising, commenting or argumenting, literacy |
 Text format | Continuous, non-continuous | Continuous, non-continuous, and mixed texts; single and multiple texts | Continuous and single |
 Item format | Closed (writing and reading) Open-constructed (writing) | Closed Open-constructed (e.g. highlighting) | Closed |
Test design | |||
 Delivery mode | Paper–Pencil-Assessment | Computer-Based-Assessment (default) or Paper–Pencil-Assessment | Paper–Pencil-Assessment |
 Test assembly | Booklet design with 72 items in total, 10 items for all respondents, additional 20 items out of three booklets for respondents who did not reach minimum number of correct answers | Adaptive testing 2/3 of the respondents were administered 20 items in two stages from 58 items in total | One booklet for all respondents, 32 items in total per respondent |
 Administration | Background questionnaire was administrated before assessment of competencies No time restriction, on average 25 min | Background questionnaire was administrated before assessment of competencies No time restriction, on average 30 min | Background questionnaire was administrated after assessment of competencies 28-min time restriction |
Scaling | |||
 Scaling model  Missing responses  Estimator | 1-PL model Not treated as incorrect 5 Plausible Values | 2-PL model Some types of missing responses were treated as incorrect 10 Plausible Values | 1-PL model Not treated as incorrect Warm’s mean weighted likelihood estimation (WLE) |
Standard setting | |||
 Number of levels | 5 proficiency levels with respondents at alpha-level 3 and below as low-literate | 5 proficiency levels with respondents at level 1 and below as low-literate | 2 proficiency levels with respondents at level 1 as low-literate and no further distinction for literates |
 Approach | A priori | Post hoc | A priori |
 Response probability | 62% | 67% | 67% |