Assessing Assessors

An interesting piece in the Chronicle (via Rationally Speaking) on "Who's assessing the assessors' assessors?"

How do we know that outcomes assessment tests measure accurately something over and beyond the instructors grades?

We don't know that the outcomes-assessment tool reliably indicates student achievement. We can't merely assume without reason that it measures learning outcomes, and, by the same reasoning that justified outcomes assessment to start with, we need some other means of assessment to determine student success or failure. Once we use that new tool, then we can see how accurate outcomes assessment was. Let's call this new procedure outcomes-assessment assessment.

And onwards to infinite regress.

Interesting "practical" counterpoint in the comments (Manyul Im):

The article suffers from a mischaracterization of the motivation for assessment. It's not radical doubt about the role or effectiveness of grading as a measuring tool for learning outcomes that motivates assessment. It's just the desire to provide a second-level check on the effectiveness of such tools. That's a fair institutional structure to catch, prevent, correct, or improve ongoing systems of measurement. Think of it as an extra quality check mechanism. There's nothing epistemologically suspect about checking quality twice. And that doesn't lead us down some path of infinite quality checks. For example, I don't need a spellcheck for the spellcheck for the spellcheck, etc. I just need spellcheck sometimes.

Interesting conversation.