{"id":5583,"date":"2011-05-16T10:31:18","date_gmt":"2011-05-16T10:31:18","guid":{"rendered":"http:\/\/scienceblogs.com\/principles\/2011\/05\/16\/active-engagement-works-improv\/"},"modified":"2011-05-16T10:31:18","modified_gmt":"2011-05-16T10:31:18","slug":"active-engagement-works-improv","status":"publish","type":"post","link":"http:\/\/chadorzel.com\/principles\/2011\/05\/16\/active-engagement-works-improv\/","title":{"rendered":"Active Engagement Works: “Improved Learning in a Large-Enrollment Physics Class”"},"content":{"rendered":"

\"ResearchBlogging.org\"<\/a>Physics is a notoriously difficult and unpopular subject, which is probably why there is a large and active Physics Education Research<\/a> community within physics departments in the US. This normally generates a lot of material in the Physical Review Special Topics journal<\/a>, but last week, a PER paper appeared in Science<\/cite><\/a>, which is unusual enough to deserve the ResearchBlogging treatment.<\/p>\n

OK, what’s this paper about?<\/strong> Well, with the exceptional originality that physicists bring to all things, the title pretty much says it all. They demonstrated that a different style of teaching applied to a large lecture class produced better attendance, more student engagement, and better learning as compared to a control section of the same course taught at the same time.<\/p>\n

So, they showed that there are better methods than the traditional lecture. Haven’t we known that for decades? How does that get into Science<\/cite>?<\/strong> Well, this is an exceptionally clean test, with all the sorts of controls you would want for good science. They took two sections of a huge introductory class, about 270 students each, and for a one-week period, they had one section taught by the regular professor (a highly regarded lecturer) and one section taught by a post-doc trained in a new teaching method. They covered the same material, using many of the same in-class examples and “clicker” questions, and at the end of the week gave both sections a short exam on the material just covered.<\/p>\n

And the results were impressive?<\/strong> Very. The students from the experimental section got an average score of 74% on the test, compared to 41% for the control section. The two distributions were really dramatically different:<\/p>\n

\"i-d4b63bd69def4910e05d2344d85ade2a-Physics_Education_Scores.png\"<\/p>\n

Yeah, that’s a pretty dramatic difference. Are you sure they got the same test?<\/strong> It says that they did– I’m not in British Columbia, so I can’t confirm it. Interestingly, it notes that the experimental section only covered 11 of the 12 topics on the test due to time constraints, so they were starting with a slight handicap.<\/p>\n

So what’s the brilliant new method?<\/strong> Basically, making the class more participatory.<\/p>\n

<\/p>\n

The experimental method asks the students to do most of the fact-based learning outside of class– reading some explanatory material whose nature is unclear– and spends the in-class time answering questions in small groups. They’re posed a question, given a couple of minutes to discuss it with their partners and submit answers via the clicker system, then given some feedback on the answers. At intervals, they’re given more involved “group tasks,” asking them to figure out something more complicated. There are also some demonstrations, the nature of which was unclear.<\/p>\n

You used the word “unclear” twice. Unclear, how?<\/strong> Well, in annoying glamour-journal fashion, they push most of the good stuff off into the “supplementary online material.” this includes the twelve test questions and all the clicker questions and group tasks. It doesn’t explain what demonstrations were done (though a demonstration is mentioned in the text), nor does it identify what they were supposed to read. It says “students were assigned a three- or four-page reading, and they completed a short true-false online quiz on the reading,” but doesn’t say whether that was reading from a textbook or something written especially for the class. I suspect it was a textbook section or so for each class, but there’s no way to tell.<\/p>\n

So, all of these gains come from doing problems in class?<\/strong> From having the students do problems in class, apparently. Though they’re not all that involved, as problems go– the questions were all multiple-choice, as you would expect for a 270-student class.<\/p>\n

Can you give an example?<\/strong> Sure. The material covered was on Maxwell’s equations and electromagnetic waves, and one of the clicker questions was:<\/p>\n

\n

Which of the following is true?
\na) For EM waves to exist, they must propagate in a medium with atoms. With no
\natoms present, the field cannot have any effect on the system and therefore can’t
\nexist.
\nb) An EM wave can propagate through a vacuum.
\nc) An EM wave is like a wave travelling along a rope in that it needs atoms to move
\nup and down.
\nd) An EM wave can only propagate in a vacuum since any medium would get in the
\nway of its propagation.
\ne) More than one of the above is true. <\/p>\n<\/blockquote>\n

They would get this, discuss with a partner for a few minutes, enter their answers, and then the instructor would give feedback to the class.<\/p>\n

OK, so that’s a clicker question. What’s a group task look like?<\/strong> From the same lecture, we have:<\/p>\n

\n

A friend of yours reminds you that en EM wave consists of both an E and B field.
\nShe asks you if the following electric field
\nE(x,t)=100x2<\/sup>t Volts\/m
\n could be that of an EM wave.Can you help? Be quantitative in your answer. <\/p>\n<\/blockquote>\n

That’s a little subtle.<\/strong> That’s the idea. They would get a longer time to work these out, with some feedback along the way.<\/p>\n

So, the whole class is just this stuff?<\/strong> That’s the idea, yes. The control group was a more traditional sort of lecture, though it used “clicker questions” as well.<\/p>\n

It seems kind of surprising that this would lead to such a big improvement. Are you sure that this isn’t some other effect? A more enthusiastic instructor, or some such?<\/strong> While the instructors for the experimental section were authors on the paper, and thus presumably deeply committed to the project, they’re also post-docs with minimal prior teaching experience. While it’s hard to rule out an instructor effect, it’s unlikely to be just that.<\/p>\n

Could this just be an effect of novelty? You know, anything you do to liven the class up improves performance?<\/strong> The technical name for the “any change you make improves things” is the “Hawthorne Effect,” after an experiment around 1930. They vehemently deny that this is what’s going on, citing a number of sources claiming the effect doesn’t really exist (including this old paper from Science<\/cite><\/a>, which is several kinds of appalling. Interestingly, though, they sort of implicitly claim a Hawthorne-ish effect to explain one of their results, namely the vastly improved attendance in the experimental section, which rose from 57% the weeks before the trial to 75% during the experiment. They suggest that the novelty of the trial got students to come see what was happening, and the new methods got them to stay. Make of that what you will.<\/p>\n

So, what are the limitations of this?<\/strong> Well, basically, it was a one-week test of a new and different method of teaching, followed immediately by a short test that was basically identical to some of the in-class material used during the experiment. It doesn’t tell you whether the effects would hold up for a full semester (though previous studies have compared entire courses taught with new methods, and suggest substantial gains), or how well the material would be retained. It would’ve been interesting, for example, to see if the experimental section scored substantially better on final exam questions covering the material from this part of the course. <\/p>\n

That’s a good point. I wonder why they didn’t do that?<\/strong> Probably because it would’ve complicated what was otherwise a very clean test. Also, I suspect either logistical (sorting out the relevant questions from the final) or ethical (there might be trouble getting permission to use student test scores as part of a research paper, unlike using a voluntary separate assessment that did not affect the final grade) issues may have come into play. They don’t mention it at all, though.<\/p>\n

Anyway, it looks pretty impressive. Are you going to implement this?<\/strong> It’s a fairly compelling argument in favor of their methods, but I’m not sure. The “get them to read the book ahead of time” thing is problematic at best, and our more compressed schedule makes it harder to do more time-intensive methods of instruction (their experiment ran in week 12 of a semester; our entire course has to fit a 10-week trimester). Also, there’s the problem of being the one person trying a new technique (I got killed<\/em> on my student evaluations last term because I did a couple of things differently than the colleagues teaching the other sections), and the fact that the supplementary material includes the sentence: “We estimate that under normal circumstances a moderately experienced instructor would require about 5hrs of preparation time per one hour class in this format.” That’s a little daunting.<\/p>\n

It’s definitely something I’ll think about for the fall, though, as I’ll be the only person teaching intro mechanics that term, giving me a little more flexibility in terms of how I run the course.<\/p>\n

Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved Learning in a Large-Enrollment Physics Class Science, 332<\/span> (6031), 862-864 DOI: 10.1126\/science.1201783<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Physics is a notoriously difficult and unpopular subject, which is probably why there is a large and active Physics Education Research community within physics departments in the US. This normally generates a lot of material in the Physical Review Special Topics journal, but last week, a PER paper appeared in Science, which is unusual enough… Continue reading Active Engagement Works: “Improved Learning in a Large-Enrollment Physics Class”<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"1","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,13,7,225,11],"tags":[610,611,123,612,92,613,116,88],"class_list":["post-5583","post","type-post","status-publish","format-standard","hentry","category-academia","category-education","category-physics","category-researchblogging","category-science","tag-active-engagement","tag-active-learning","tag-education-2","tag-lecture","tag-physics-2","tag-physics-education-research","tag-research-blogging","tag-science-2","entry"],"_links":{"self":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/5583","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/comments?post=5583"}],"version-history":[{"count":0,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/5583\/revisions"}],"wp:attachment":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/media?parent=5583"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/categories?post=5583"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/tags?post=5583"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}