Assessment Disaggregation

A tool to disaggregate pre- and post-test responses into value-added learning scores (Walstad and Wagner 2016) and adjust them for guessing (Smith and Wagner 2018).

What this Does

In the Spring of 2016, Walstad and Wagner released a paper suggesting that the pretest/posttest delta is insufficient in assessing learning outcomes. In 2018, Smith and Wagner showed that this disaggregation should be adjusted using the probability of guessing correct. In 2020, Smith and White expanded the analysis with adjusted gain estimators. As this analysis of learning is extremely useful to both research and assessment, this analysis is common. However, performing such a disaggregation and adjustment is time intensive, especially if the questions appear in a different location (or order) on the pretest and posttest.

This software automatically performs this analysis from raw exam files (currently supporting Akindi, Scantron, Canvas Quizzes, Quick Key, and ZipGrade). Questions can be in any order as the the optional map file can describe how the same question can appear in in different locations.

Main Software Interface

The user of this software simply clicks pretest button and selects the pretest file, and clicks on the posttest button and selects the posttest file. The user can then (optionally) select an assessment map file if the questions are not in the same order.