Skip to main content

Table 4 Science ability cut score VCs and G coefficients of the reliability analyses

From: The standard setting process: validating interpretations of stakeholders

Effect df MC items Effect df CMC items
Round 1 Round 2 Round 1 Round 2
VC % VC % VC % VC %
\({\hat{\sigma }}^{2} \left( \text{i} \right)\) 16 0.179 47.4 0.063 48.1 \({\hat{\sigma }}^{2} \left( \text{i} \right)\) 12 0.045 41.7 0.043 41.7
\({\hat{\sigma }}^{2} \left( \text{e} \right)\) 8 0.105 27.8 0.069 1.5 \({\hat{\sigma }}^{2} \left( \text{e} \right)\) 8 0.000 0.0 0.000 0.0
\({\hat{\sigma }}^{2} \left( {\text{ie}} \right)\) 128 0.094 24.8 0.066 50.4 \({\hat{\sigma }}^{2} \left( {\text{ie}} \right)\) 96 0.026 24.1 0.000 0.0
       \({\hat{\sigma }}^{2} \left( {\text{c}:\text{e}} \right)\) 27 0.012 11.1 0.017 16.6
       \({\hat{\sigma }}^{2} \left( {{\text{ic}}:{\text{e}}} \right)\) 324 0.025 23.1 0.043 41.7
\({\hat{\sigma }}^{2} \left( {{\text{X}}_{\text{ie}} } \right)\)   0.378 100 0.198 100 \({\hat{\sigma }}^{2} \left( {{\text{X}}_{{{\text{ie}}\left( {\text{ce}} \right)}} } \right)\)   0.108 100 0.103 100
Φ   0.63   0.48     0.42   0.42  
  1. \({\hat{\sigma }}^{2} \left( {{\text{X}}_{\text{ie}} } \right)\) and \({\hat{\sigma }}^{2} \left( {{\text{X}}_{{{\text{ie}}\left( {\text{ce}} \right)}} } \right)\) are total scores of variance, \(\hat{\sigma }^{2} \left( e \right)\) = variance of experts, \(\hat{\sigma }^{2} \left( i \right)\) = variance of items, \(\hat{\sigma }^{2} \left( {ie} \right)\) = variance of the interaction between experts and items, \(\hat{\sigma }^{2} \left( {ce} \right)\) = variance of the cut scores that are nested in experts and \(\hat{\sigma }^{2} \left( {{\text{ic}}:{\text{e}}} \right)\) = variance of interaction between the items and the cut scores that are nested in experts
  2. Φ index of dependability, VC variance components, df degrees of freedom