Friday, November 9, 2012

The perfect grading strategy, or: How to benefit from your own research

Grading exams is one of the burdens of being a teacher, especially when it is a large course. My biggest course has approximately 120 students, which means that right now 120 exams consisting of 6 essay questions each are sitting on my desk.
What is the right grading strategy? There seem to be two strategies.
One is to do one exam at a time, and grade all six questions. There are several problems with this approach. The main problems is the risk that your standards start drifting: if students are bad on average you may become more forgiving, or more stringent if they do well. The second problem is the change of mindset necessary after each question, which is mentally taxing and time-consuming.
The second strategy is the one I think most people use, which is to grade all question 1's, then shuffle the whole stack of exams, grade all question 2's, etc. This seems to be the perfect solution that solves the problems of strategy 1: your standards are less likely to drift, and if they do the effect is more randomly distributed. You also don't have to shift mindsets, because you grade the same question over and over again.
So isn't strategy 2 the perfect method? Unfortunately not. When reading and grading answers that are all similar but not the same it becomes increasingly difficult to separate them in your mind. Each new version of the answer needs more mental focus to keep it apart from the previous answers from other exams. That is why I often use this as an example of a case where interruption is beneficial instead of harmful: an interruption helps clearing the mind of previous answers, reducing interference. But regular interruptions do not, of course, speed up the process of grading, and as we all know interruptions often lead to more distraction.
The solution is so simple that I wonder why I haven't thought of it before. Isn't it obvious: grade TWO questions per exam, so first all questions 1 and 2, then all question 3 and 4, etc. Alternating two questions delivers just the amount of memory interference needed to disentangle the current answer from previous answers. Keeping two answer standards in mind is still doable, and the standard drift is also reasonably under control.
I am now halfway through with grading, and although there still a lot of work to do, I can report that it works very well, and I can recommend it to everyone in the same situation.

Sunday, January 29, 2012

Seven plus or minus two: hard to eradicate

The website of "de Volkskrant", a Dutch newspaper, had an interesting bit of news: ToDo lists don't work. The article referred to a blog of the Harvard Business Review by Daniel Markovitz. One of the main claims of that article is that our brain cannot handle more than seven choices, and will therefore be overwhelmed when a ToDo list is longer than seven items.
Seven plus or minus two has been a staple of Cognitive Psychology since Miller published a paper on the limitations of short-term memory in the fifties, which led to the naive theory (not endorsed by Miller himself!) that human short-term memory consists of seven (plus or minus two) slots in which items can be stored. But even though cognitive psychologists have abandoned this theory for decades, it still floats around in the applied domain as a serious limitation of human cognition.
To get back to the original topic, I always find ToDo lists quite useful. From a multitasking perspective they perform a very useful service, helping you keeping track of your uncompleted goals. So now this blog claims that these lists do not work, because your brain can only handle seven uncompleted goals.
The cited research for this claim is a set of studies that show that people want choices, but not too many of them. Being able to choose between 3 brands of detergent is nice, but choosing between 20 brands is ridiculous. So, what's the threshold? Seven. Really seven? In many situations I'd rather not choose at all, that is why I like house brands in supermarkets. Maybe in a restaurant I prefer some more choices on the menu than just seven.
But even if this were all true, the generalization from choosing between similar candidates for a purchase and which item to do next on a ToDo list is pretty far-fetched. And the idea that your brain will overheat if the length of your ToDo lists exceeds the threshold is pretty ludicrous.
To conclude: I think ToDo lists are pretty useful as a memory aid. But maybe the lesson is to be really, really suspicious about any type of psychological advice that involves the number Seven. Plus or Minus Two.