Should a machine mark your essay?

News broke last week of new software for grading essays that will, one supposes, revolutionize the way students are evaluated.

Let’s hope not.

The details so far are scant, but the idea is not new and typically machine grading involves algorithms that guess at the quality of the essay based on whether the essay looks like a good essay, not whether it really is or not.  This is an enormous problem, of course, because it is quite common to see essays that are superficially strong -- good grammar, rich vocabulary -- but lack any real insight (this is common among first year students who, presumably, sailed through high school by bamboozling their teachers). Similarly some very strong essays---with striking originality and deep insight---have a surprising number of technical errors that would likely lead a computer algorithm to conclude it was bad.

A machine cannot recognize the more subtle aspects of writing well. Can the software recognize wit or daring? Can it tell when phrasing is especially apt or clever?

Still worse than all these, the computer cannot possibly know how the students answers have related to what was done elsewhere in the course. Did a student’s answer present an original idea? Or did it just rehash what the prof said in class?

Related Posts

Five basic kitchen supplies every student should have

You don’t need a Le Creuset Dutch oven to whip up a delicious meal for one (or, say, eight). A few key items are enough to launch your cooking career—and some of them may even last you a lifetime.

Five easy, tasty recipes for students on a budget

Tempted to order pizza—again? Maybe try one of these recipes instead; they’re cheap and cheerful, with bragging rights guaranteed.

Worst of all, the new software is being touted for its ability to provide instant feedback and allow for immediate revision. While these elements may be appealing to students, they will, one suspects, encourage one of the worst tendencies among student writers: the urge to simply tinker rather than genuinely rewrite. Even if students don’t focus on figuring out what the computer program is looking for (which, let’s face it, they will), students will be led to seek the smallest possible change to get the biggest possible increase in the grade. Students do this with human graders already, but a good human grader can point out that tinkering is not revision, and can choose not to reward such strategies.

Looking further ahead, if this kind of software becomes standard for exams, it’s easy to see how administrators will push for its use for course assignments too. And with professors "freed" from the burden of grading, there will be little reason to keep class sizes down. Huge classes, no access to your professors---and your essays marked by HAL 9000 from the Space Odyssey series.

And while the instant feedback might be gratifying, something really profound will then have been lost. Because, while I often dislike grading, I spend a lot of time on it, because the most important part of grading an essay is not assigning the actual grade, but making incisive comments. And the ideal comments speak to the specific content as well as to the course content, and even to the needs and personality of the student. Again, this is, quite literally, the human element that the computer cannot reproduce.

And if that’s the revolution you want, well, you know that you can count me out.