Ontario’s auditor general wrong on teaching

Prof. Pettigrew: student evaluations won’t help
Professor Gary McCarron’s Communications 110 class
Prof. Gary McCarron's Communications class at Simon Fraser U. (Simon Hayter)

A recent report from the Ontario Auditor General Jim McCarter has got people talking about student teaching evaluations again. Hoo boy.

McCarter is concerned that evidence of teaching ability is not being taken into account when it comes to granting tenure and promotion to faculty. It’s a legitimate concern in theory. The problem is that this report takes student evaluations as a key method by which quality teaching should be measured. That’s trouble.

As the report rightly points out, the research on the usefulness of student evaluations is a subject of much disagreement. In fact, it’s actually even more hotly contested than the AG’s report admits. The Canadian Association of University Teachers (CAUT) insists, for instance, that such surveys cannot be taken as a measure of teaching effectiveness.

CAUT may be trying to protect the jobs of its members. Still, student evaluations, from the outset suffer from a basic flaw which is that they often fail to meet a very basic standard for any evaluation. That is, an evaluator should be qualified to evaluate. More specifically, the evaluator should be an expert on the subject, should be motivated to take the evaluation seriously, and should be a disinterested third party.

Students are none of these things.

Students, especially first year students are not experts when it comes to evaluating university teaching. How could they be?

Similarly, what motivation does a student have to take the exercise seriously? They have little way of knowing how the evaluation will be used and are entirely unaccountable for what they say.

Finally, students are not in any position to be objective, since students are being graded by their professors, and students have a tendency to take grading personally.

Indeed, one place that the research tends to agree is that the simplest way for instructors to improve their evaluations is to give higher grades. No wonder grades have gone up since student evaluations became common.

This is not to say that some students don’t give reasonable or meaningful evaluations. Still, there is no way for an outside observer to look at an average student evaluation score and have any idea what it means. Some research has suggested that much of a professor’s evaluation relates to superficial performance questions rather than important content. In any case, how much of the low score is due to the professor holding high standards? How much is due to a class that was particularly prone to laziness? And how much is due to the fact that a single survey for all courses (which this report recommends) cannot possibly be adequate for the wide variety of courses offered at even small universities?

Still, isn’t it a problem that teachers are being given tenure regardless of their teaching ability? It is, but here again, the Auditor General hasn’t been asking the right questions. The reason professors at some universities get tenure or promotion without a good teaching record is that they have an excellent research record. That in itself is not entirely unreasonable since it is the rare professor who is both a top notch instructor and a top notch researcher.

More to the point, one of the reasons some universities are so focused on research is that research brings in money. And universities need that money because government funding has been consistently dropping.

Why cut funding? Well, one argument that seems to be wining the day is that university education is an investment that pays off in higher income later. Therefore, the investor/student shouldn’t mind paying high tuitions since she will reap the financial payoff later. It’s all a matter of efficiency and profit in this view. But when governments cut funding for this and similar reasons, they starve universities of the funds they need and encourage them to seek out big grants. And that means more emphasis on research and less on teaching.

It is, therefore, hypocritical to insist that the university be an economic engine, to insist as the report says, that “a university’s most important mandate is that it does a good job of teaching its students and preparing them for the future workforce” and then chastise universities for being short-sighted and self-interested.

In other words, you can’t have it both ways. You can’t insist that university education is all about the money and then get upset when universities act like they are all about the money.

At least, that’s my evaluation.