Assessment Disaggregation

Disaggregates pre- and post-test answers into value-added learning scores (Walstad and Wagner 2016), adjusts for guessing (Smith and Wagner 2018), and computes gain scores (Smith and White 2021).

What this Does

In the Spring of 2016, Walstad and Wagner released a paper suggesting that the pretest/posttest delta is insufficient in assessing learning outcomes. In 2018, Smith and Wagner showed that this disaggregation should be adjusted using the probability of correctly guessing. In 2021, Smith and White expanded the analysis with adjusted gain estimators. As this analysis of learning is extremely useful to both research and assessment, this analysis is common. However, performing such a disaggregation and adjustment is time intensive, especially if the questions appear in a different location (or order) on the pretest and posttest.

This software automatically performs this analysis from raw exam files (currently supporting Akindi, Scantron, Canvas Quizzes, Quick Key, Moodle, Google Forms quizzes, ZipGrade, and some Blackboard systems). Questions can be in any order as the optional map file can describe how the same question can appear in different locations.

Main Software Interface

The user of this software simply clicks pretest button and selects the pretest file, and clicks on the posttest button and selects the posttest file. The user can then (optionally) select an assessment map file if the questions are not in the same order.