Matched Question Analysis

This report is grouped at the question level so the practitioner can make informed adjustments to their class. This is the most import analysis file for assessment and pedagogical improvement.

In all matched analysis files you will find the raw disaggregated learning types as well as columns labeled 'gamma', 'alpha', 'mu', and 'flow'. These correspond to "corrected" measurements of the learning types when factoring in the number of students guessing. $\hat \gamma$ (gamma) is corrected positive learning, $\hat \alpha$ (alpha) is corrected negative learning, $\hat \mu$ (mu) is corrected pretest stock knowledge (corrected retained plus corrected negative learning), and flow is the corrected pretest/posttest delta ($\hat \gamma-\hat\alpha$). Formally, the following equations are used to find the corrected values:

where $\hat{\text{pl}}$ (positive learning), $\hat{\text{rl}}$ (retained learning), $\hat{\text{zl}}$ (zero learning), and $\hat{\text{nl}}$ (negative learning) refer to the raw learning type values and $n$ is the number of answer options. It is important to use these corrected values as the raw scores can be sensitive to the percent of the class guessing. Smith and Wagner 2018 details this adjustment. Gamma gain ( $\hat \gamma/(1-\hat\mu)$ ) was introduced by Smith and White 2020.

If you are new to this analysis, focus your attention on $\hat \gamma$ (gamma). In simple terms, this is the proportion of students who learned the material (as opposed to answered the question correct). Higher is better, but comparing different questions can be problematic as they can be at different levels of difficulty. The gamma gain ( $\hat \gamma/(1-\hat\mu)$ ) estimate is the proportion of students who learned the material that didn't already know the material.

​