{"id":5654,"date":"2011-06-08T08:39:08","date_gmt":"2011-06-08T08:39:08","guid":{"rendered":"http:\/\/scienceblogs.com\/principles\/2011\/06\/08\/practice-matters-the-effect-of\/"},"modified":"2011-06-08T08:39:08","modified_gmt":"2011-06-08T08:39:08","slug":"practice-matters-the-effect-of","status":"publish","type":"post","link":"http:\/\/chadorzel.com\/principles\/2011\/06\/08\/practice-matters-the-effect-of\/","title":{"rendered":"Practice Matters: “The effect of curriculum on Force Concept Inventory performance: A five thousand student study”"},"content":{"rendered":"

A few years ago, we switched to the Matter & Interactions<\/a> curriculum for our introductory classes. This has not been without its hiccups, among them the fact that there has been a small decline in the conceptual learning gains measured by the Force Concept Inventory, the oldest and most widely used of the conceptual tests favored by the Physics Education Research community. We’ve spent some time discussing whether this is a temporary glitch, due to the transition, or something inherent in the curriculum. (Our numbers are small enough that these results remain at the level of plural anecdotes.)<\/p>\n

Yesterday, via ZapperZ<\/a>, I ran across an arxiv paper showing that we’re not the only ones to see this<\/a>:<\/p>\n

\n

The performance of over 5000 students in introductory calculus-based mechanics courses at the Georgia Institute of Technology was assessed using the Force Concept Inventory (FCI). Results from two different curricula were compared: a traditional mechanics curriculum and the Matter & Interactions (M&I) curriculum. Post-instruction FCI averages were significantly higher for the traditional curriculum than for the M&I curriculum; the differences between curricula persist after accounting for factors such as pre-instruction FCI scores, grade point averages, and SAT scores. FCI performance on categories of items organized by concepts was also compared; traditional averages were significantly higher in each concept. We examined differences in student preparation between the curricula and found that the relative fraction of homework and lecture topics devoted to FCI force and motion concepts correlated with the observed performance differences.<\/p>\n<\/blockquote>\n

So, this is a clear case where a reform curriculum is outperformed by a traditional one (though, it should be noted, this is not a case of traditional large-lecture instruction outperforming more modern teaching methods– both classes use active engagement tools like “clickers” and discussion questions). Why is this? The last sentence of that abstract tells you: Matter & Interactions spends less time on concepts covered on the FCI than a traditional curriculum does. This is summarized in the following table from the paper:<\/p>\n

<\/p>\n

\"i-db630e2691589456ccd3e7cbb985ec21-MI_FCI_table.png\"<\/p>\n

This shows the fraction of the homework assignments devoted to covering topics on the FCI, with the questions being classified by examination by one ofn the researchers. Over half of the traditional curriculum assignments cover FCI concepts, while about a quarter of the M&I assignments did. In the subject-by-subject breakdown, you can see that the traditional curriculum spends more homework time on every area except Newton’s 1st Law, which both curricula give short shrift.<\/p>\n

This matches pretty well with my experience teaching under M&I. In our previous curriculum, I used a lot of FCI-type questions in class and on exams, but it’s always felt like there’s less room for that in M&I. And even where they do fit, the language and style of the M&I presentation makes it harder to directly carry questions over.<\/p>\n

So, is this a major problem? It’s hard to say. On the one hand, this is unquestionably a matter of the design of the course, not an accident of implementation– students using M&I get lower scores on the FCI because the course as designed spends less time on those topics. If asked, I suspect that the M&I authors would say that this is a feature, not a bug. The curriculum is designed to teach other sorts of physical reasoning, so the FCI is not a good match for the course, and some other sort of conceptual assessment would be more appropriate.<\/p>\n

(There are a few other methodological quibbles that might be raised, chief among them the fact that the M&I courses had a single weekly class meeting for three hours(!), compared to three weekly meetings for the traditional class, which also probably reduces the time available for engaging with FCI concepts, in a way that might not be what the designers of the curriculum intended.)<\/p>\n

On the other hand, though, a lot of the impetus for physics education research based reform has come from the fact that students in traditional lecture classes do a miserable job on conceptual tests like the FCI. If the new curriculum doesn’t stack up in those conceptual areas, then we probably have to think more carefully about what, exactly, we want to accomplish with the introductory physics sequence.<\/p>\n

So, as noted yesterday<\/a>, physics education remains a complicated and messy business.<\/p>\n","protected":false},"excerpt":{"rendered":"

A few years ago, we switched to the Matter & Interactions curriculum for our introductory classes. This has not been without its hiccups, among them the fact that there has been a small decline in the conceptual learning gains measured by the Force Concept Inventory, the oldest and most widely used of the conceptual tests… Continue reading Practice Matters: “The effect of curriculum on Force Concept Inventory performance: A five thousand student study”<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"1","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,134,13,7],"tags":[],"class_list":["post-5654","post","type-post","status-publish","format-standard","hentry","category-academia","category-course_reports","category-education","category-physics","entry"],"_links":{"self":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/5654","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/comments?post=5654"}],"version-history":[{"count":0,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/5654\/revisions"}],"wp:attachment":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/media?parent=5654"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/categories?post=5654"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/tags?post=5654"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}