{"id":7935,"date":"2013-05-14T11:57:11","date_gmt":"2013-05-14T15:57:11","guid":{"rendered":"http:\/\/scienceblogs.com\/principles\/?p=7935"},"modified":"2013-05-14T11:57:11","modified_gmt":"2013-05-14T15:57:11","slug":"homework-is-evil-the-benefits-of-completing-homework-for-students-with-different-aptitudes-in-an-introductory-physics-course","status":"publish","type":"post","link":"http:\/\/chadorzel.com\/principles\/2013\/05\/14\/homework-is-evil-the-benefits-of-completing-homework-for-students-with-different-aptitudes-in-an-introductory-physics-course\/","title":{"rendered":"Homework Is Evil?: “The benefits of completing homework for students with different aptitudes in an introductory physics course”"},"content":{"rendered":"

\"ResearchBlogging.org\"<\/a><\/span>One of the perennial problems of teaching intro physics is getting students to do their homework, so I was very interested to see Andy Rundquist on Twitter<\/a> post a link to a paper on the arxiv titled “How different incentives affect homework completion in introductory physics courses<\/a>.” When I shared this with the rest of my department, though, I got a link to an even more interesting paper from the same group, on the effect that doing homework has on student performance<\/a>. This has an extremely surprising conclusion: for the weakest students in introductory physics, doing more homework actually decreases<\/em> their grade in the course.<\/p>\n

This is surprising enough to be worth a little discussion on the blog, so we’ll give this the Q&A treatment.<\/p>\n

See! I told you, homework is evil! And these people have proved it with SCIENCE!<\/strong> OK, let’s not get ahead of ourselves, here. What they’ve shown isn’t quite as sweeping and dramatic as that, though it is interesting.<\/p>\n

Killjoy. OK, what did they actually show?<\/strong> Well, they looked at a very large sample of students in the second term of introductory physics– a couple of years worth of classes with around 1000 students\/year– and looked at the correlation between the amount of homework those students did and their scores on the exams for the course. They first separated the students into four groups based on “physics aptitude,” though, and looked at each group independently. For the highest-aptitude group, they found what you would expect: students who did more homework got higher scores on the exams. In the lowest-aptitude group, though, they found the opposite: students who did more homework got lower exam scores.<\/p>\n

That’s… weird. But how do you sort students by aptitude in the first place? I mean, how do you know which students are good at physics before they take the exams?<\/strong> The aptitude sorting is based on grades in prerequisite classes. The specific course they looked at was the second term of intro physics, covering electricity and magnetism. Students taking that course needed to take two calculus courses and the first term of intro physics first, and the average grade in those three courses was the basis for the “aptitude” sorting. This average was reasonably well correlated with the grade in the second term of physics, and also with other measures of aptitude (SAT scores, conceptual test scores, etc.).<\/p>\n

Couldn’t that just be a measure of general student skills, though? That is, students who get lousy grades in calculus also get lousy grades in physics because they have poor study skills in both?<\/strong> That’s one of the weaknesses, yes. If it were just a measure of poor attitude, though, you might expect the “low-aptitude” students to all blow off the homework, and that’s not what they see– the homework completion rates for all the groups are pretty similar. These appear to be students who are putting a reasonable amount of effort into the class, and just not doing well.<\/p>\n

Speaking of which, shouldn’t you show us these results?<\/strong> You could look in the paper, you know.<\/p>\n

Yeah, but then I’d have to click the mouse button two whole times. Just put a graph in the post, please?<\/strong> Fine, here you go:<\/p>\n

\"Figure<\/a>
Figure 1 from the arxiv preprint discussed in the text<\/figcaption><\/figure>\n

This shows the cumulative exam score (three midterms and a final) on the vertical axis, with the percentage of the homework completed on the horizontal axis. The high-aptitude group is at the top, and the low-aptitude group is at the bottom.<\/p>\n

Those points are kind of scattered all over the place, dude.<\/strong> Yeah, but that’s pretty good for social-science type data. And you can see a pretty clear trend indicated by the straight lines on each of those graphs. For the high-aptitude group, the line slopes up and to the right, indicating the more homework is correlated with a higher grade. In the low-aptitude group, the line slopes down and to the right, indicating that more homework is correlated with a lower grade.<\/p>\n

How big an effect are we talking?<\/strong> Not all that big– very roughly, doing an additional 10% of the homework raised or lowered the exam total by about 2%. But it’s statistically significant, and the difference between groups is surprising.<\/p>\n

Yeah, that’s definitely weird. What would cause that, or is this one of those “correlation is not causation” situations?<\/strong> Well, they don’t have an incontrovertible demonstration of a causal link between these, but they have some ideas. The education-jargon term for it is that doing more homework “placed an excessive cognitive load on low aptitude students.”<\/p>\n

So,…. basically, “Mr. Osborne, my brain is full”<\/a>?<\/strong> Not quite as pejorative as that, but yeah. The idea is that students who don’t have a solid framework for doing physics end up inventing ad hoc<\/i> “schemas” for each homework problem they do, idiosyncratic ways of explaining how they got the answer. The more homework they do, the more of these they accumulate, and it becomes hard to sort out what, exactly, they’re supposed to be doing.<\/p>\n

The higher-aptitude students, on the other hand, are in a better position to interpret the problems in terms of a smaller number of universal rules, and thus have an easier time keeping everything straight. When they confront a problem, they have a smaller set of more flexible tools to draw from, so finding the right approach is easier.<\/p>\n

It’s a nice story, but is there any other suport for it?<\/strong> Sort of. They looked at each of the mid-term exams separately, and the correlation between doing the homework for the relevant section and the score on that exam (so, graphs like the above only plotting the grade on the first quarter of the class versus the percentage of the first quarter of the homework done, etc.). On the first couple of exams, the low-aptitude groups didn’t show much correlation between grades and homework at all, but on the third exam and the final, the negative correlation was very clear.<\/p>\n

And how does that help anything?<\/strong> Well, the idea is that for the first few exams in intro E&M, they’re only dealing with electric fields, and the number of techniques that can come into play is very small. In the third quarter, they introduce magnetic fields, which follow different rules. In the “excessive cognitive load” picture, this introduces a problem because the low-aptitude students can get confused between electric and magnetic field approaches. There’s a significant increase in the number of techniques they need to know and use, and that triggers the “negative benefit” of doing more homework.<\/p>\n

Couldn’t it just be that magnetic fields are harder to deal with? I mean, there’s all those cross products and stuff…<\/strong> That’s also a possibility. That probably wouldn’t produce a negative correlation with homework, though– if anything, you’d expect students to get better with practice.<\/p>\n

Which brings up another thing: What does this mean for teaching? Should you stop making weak students do homework?<\/strong> That’s one of the frustrating things about this paper (and a lot of education research, for that matter)– they identify a problem, but don’t do much to suggest solutions. They do note that this finding seems to run counter to the usual advice– most of the time, we tell students who are struggling to do more<\/em> homework, not less– but it’s kind of hard to tell what to do instead. To be fair, though, this is a surprising enough result that it’s worth publishing even without a suggested solution.<\/p>\n

But is this going to change your approach?<\/strong> Not immediately. For one thing, the effect isn’t all that big, and I’d want to see it confirmed by somebody else. More than that, though, I’m not entirely sure that this is broadly applicable.<\/p>\n

Why not?<\/strong> Well, this is a study based at the Air Force Academy, where all entering students are required to take two terms of physics. This includes the 40-ish percent of them who go on to major in non-scientific subjects.<\/p>\n

While this is great from a sample size perspective– they’re teaching 1000 students a year in intro physics, so it’s easy to get statistical power in only a few years of testing– it might mean that their sample doesn’t generalize. Their lowest tier of physics aptitude is going to include a bunch of non-scientists who probably wouldn’t take physics at all at most other schools. The students I see in Union’s intro classes might not include any from the population showing the strongest negative effect in this study.<\/p>\n

So, in the end, I think this is probably saying something interesting about the way students learn physics, but might not have that much practical application. Then again, a different approach to homework that turned the negative correlation around for the weakest students might well produce a dramatic improvement in the stronger students, as well. Which would be worthwhile.<\/p>\n

As there isn’t any such approach on offer yet, though, I’m going to hold off on making changes until somebody suggests one. This is definitely something to keep an eye on, though.<\/p>\n

And, as a nice bonus, it’s a great excuse for not doing my physics homework! I’m making a rational strategic decision to improve my grade!<\/strong> Well, as long as you’re happy with thinking of yourself as a low-aptitude student, that is.<\/p>\n

Oh, yeah. Good point. I guess my ego does demand that I do my homework, after all.<\/strong> Vanity: a powerful force for good. In physics education, anyway…<\/p>\n

F. J. Kontur, & N. B. Terry (2013). The benefits of completing homework for students with different
\n aptitudes in an introductory physics course Physics Education<\/span> arXiv: 1305.2213v1<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

One of the perennial problems of teaching intro physics is getting students to do their homework, so I was very interested to see Andy Rundquist on Twitter post a link to a paper on the arxiv titled “How different incentives affect homework completion in introductory physics courses.” When I shared this with the rest of… Continue reading Homework Is Evil?: “The benefits of completing homework for students with different aptitudes in an introductory physics course”<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":7936,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,13,7,225,11],"tags":[],"class_list":["post-7935","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-academia","category-education","category-physics","category-researchblogging","category-science","entry"],"_links":{"self":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/7935","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/comments?post=7935"}],"version-history":[{"count":0,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/7935\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/media\/7936"}],"wp:attachment":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/media?parent=7935"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/categories?post=7935"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/tags?post=7935"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}