{"id":234,"date":"2006-05-08T07:30:15","date_gmt":"2006-05-08T07:30:15","guid":{"rendered":"http:\/\/scienceblogs.com\/principles\/2006\/05\/08\/evaluate-this\/"},"modified":"2006-05-08T07:30:15","modified_gmt":"2006-05-08T07:30:15","slug":"evaluate-this","status":"publish","type":"post","link":"http:\/\/chadorzel.com\/principles\/2006\/05\/08\/evaluate-this\/","title":{"rendered":"Evaluate This!"},"content":{"rendered":"

Timothy Burke, my go-to-guy for deep thoughts about academia, had a nice post about student evaluations<\/a> last week. Not ecvaluations of<\/strong> students, evaluations by<\/strong> students– those little forms that students fill out at many schools (not Swarthmore, though) giving their opinion of the class in a variety of areas.<\/p>\n

(Probably not entirely coincidentally, as this is the time of year when semester-school faculty fret about evaluation scores, Inside Higher Ed offers yet another RateMy Professor.com article<\/a>, showing a positive correlation between “hotness” and positive evaluations there…)<\/p>\n

Those evaluations are still a few weeks off for me, but they remain a constant back-of-the-mind concern, especially for the untenured. Particularly with intro classes, it’s really hard to get a read on exactly what the students think of you, and more or less impossible to predict what they’ll put on the evaluations. I’ve gotten bad evaluations from classes I thought liked me, and good ones from students I was sure hated me. It’s distressingly random given the amount of weight those evaluations carry.<\/p>\n

I do, however, have a secret trick for improving the usefulness of the evaluations, which also seems to improve the numerical scores somewhat, and which I will present below the fold:<\/p>\n

<\/p>\n

The trick to getting useful information out of student evaluations is to ask for it. We have a form that we use that is half numerical (rate things on a scale of 1-5), and half written (respond to these general questions), and the questions on the written part are incredibly vague, because they need to be used for all the classes. The responses to these questions also tend to be awfully generic (“Fine” or “Yes” are common answers, and none of them are yes-no questions…)<\/p>\n

So what I do is I ask more specific questions. Before handing out the forms, I explain that the written comments are transcribed by the secretary and given to me, and ask them to use the written forms to help me improve my teaching. I make a list on the blackboard of things that I did differently that term (or that I do differently than other faculty teaching the course), and ask them to comment on those specific items somewhere within the school’s generic questions.<\/p>\n

Weirdly, this not only gets me better responses to the written questions, it also seems to improve the scores on the numerical side, I think because they have to think a little more carefully about the course and its larger context, and don’t just write down random numbers. (Though, to be fair, I didn’t really hit on this until after I’d been teaching a couple of years, so the improvement might just be a general improvement in my teaching, not a result of the questions.) And it does get me some actual useful information on the written side– I’ve had students pick up on things that I didn’t think they’d really notice, and some of them have spontaneously said some really nice things.<\/p>\n

On the larger question of whether these evaluations actually measure anything useful, I don’t really have much to say. I think the numerical scores are probably useful as a sort of threshhold indicator– if your scores are mostly above the historical course average, you’re fine, if they’re mostly below, there were some problems– but beyond that, I wouldn’t put much weight on them, if I were designing a college. Of course, I’m stuck with them, so I’m just happy to have an ethical way to improve the quality of the evaluations.<\/p>\n

Credit where due: I stumbled on this idea more or less on my own, but after the first term asking additional questions, a senior faculty member in another department mentioned that she does the same thing, and confirmed my observations. So it’s not just me.<\/p>\n","protected":false},"excerpt":{"rendered":"

Timothy Burke, my go-to-guy for deep thoughts about academia, had a nice post about student evaluations last week. Not ecvaluations of students, evaluations by students– those little forms that students fill out at many schools (not Swarthmore, though) giving their opinion of the class in a variety of areas. (Probably not entirely coincidentally, as this… Continue reading Evaluate This!<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"1","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"class_list":["post-234","post","type-post","status-publish","format-standard","hentry","category-academia","entry"],"_links":{"self":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/234","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/comments?post=234"}],"version-history":[{"count":0,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/234\/revisions"}],"wp:attachment":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/media?parent=234"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/categories?post=234"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/tags?post=234"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}