{"id":5742,"date":"2011-07-20T11:41:22","date_gmt":"2011-07-20T11:41:22","guid":{"rendered":"http:\/\/scienceblogs.com\/principles\/2011\/07\/20\/reforming-education-bonuses-ar\/"},"modified":"2011-07-20T11:41:22","modified_gmt":"2011-07-20T11:41:22","slug":"reforming-education-bonuses-ar","status":"publish","type":"post","link":"http:\/\/chadorzel.com\/principles\/2011\/07\/20\/reforming-education-bonuses-ar\/","title":{"rendered":"Reforming Education: Bonuses Aren’t Enough"},"content":{"rendered":"

One of the standard education reform proposals that gets suggested every time somebody brings up the condition of American public education is that teachers should be offered some form of performance incentive, whether in the form of “merit pay” programs on a continuing basis, or bonuses for reaching particular targets. This is one of those ideas that economists swear ought to work, but a new RAND Corporation study<\/a> found that, well, they don’t. This is based on a three-year pilot program run by the New York City school district, which offered selected districts bonuses of up to $3,000 per teacher for meeting certain school-wide performance targets.Taking the key findings from their research brief<\/a>, we have:<\/p>\n

\n

The program did not improve student achievement at any grade level.<\/strong> The researchers found that the average mathematics and English language arts test scores of students from elementary, middle, and K-8 schools invited to participate in [the Schoolwide Performance Bonus Program] were lower than those of students from control schools during all three years of the experiment. However, the differences were very small and statistically significant only for mathematics in year 3 and were not significant when the researchers controlled for testing effects from multiple years and subjects. Similarly, researchers found no overall effects on state Regents Exam scores for high school students in the first two years (year 3 data were not available for analysis). The program’s effects did not differ among schools of different sizes or according to bonus award distribution plan.<\/p>\n

The program also did not affect school Progress Report scores.<\/strong> Across all years and all categories of scores for the Progress Reports (environment, performance, progress, and additional credit), the researchers found no statistically significant differences between scores of treatment and control schools. The lack of effects held true for elementary, middle, K-8, and high schools.<\/p>\n

The program did not affect teachers’ reported attitudes, perceptions, and behaviors.<\/strong> The researchers found no differences between the reported practices and opinions of teachers in treatment schools and those of the control group. The survey responses about instructional practices, effort, participation in professional development, mobility, and attitudes from the two groups were very similar, with no statistically significant differences. Furthermore, the vast majority of teachers who received bonuses said that the bonus did not affect their performance.<\/p>\n<\/blockquote>\n

That’s pretty much the definition of total failure. So, what’s going on here? Some miscellaneous items from looking through the report (the full study<\/a> is free to download):<\/p>\n

<\/p>\n

— One of the suggested reasons why this didn’t pan out is that the bonus was not seen as sufficiently valuable. The level of $3,000 per teacher (“teacher” here meaning “teacher represented by the local union,” which cooperated with the study) is generally acknowledged to be not that impressive. The report quotes a district official saying:<\/p>\n

\n

We] looked at the literature . . . and that number [$3,000] . . . is [about] fve percent of [the salary of] a fourth-year or ffth-year teacher. And that’s all we could aford. In an ideal world, it would have been more. I think there’s a little bit of literature about this that says to really change behavior–and that’s the only reason you would do something like this–that number needs to be bigger. But [we] thought intuitively that that number was big enough to have some meaning.<\/p>\n<\/blockquote>\n

(page 37 of the report, page 75 of the PDF). A teacher’s union official described it as “the minimal amount we thought was necessary.” So, you could argue that the system would work, if only the bonuses were bigger. This quickly gets into the problem of where the money is going to come from, though.<\/p>\n

— The money was disbursed to schools for meeting school-wide targets for test scores and the like, and then distributed to teachers within the schools according to a plan developed by a Compensation Committee for each school. The committees consisted of the school principal, another person chosen by the principal, and two people chosen by the teachers of the school. The committees were free to distribute the bonus money equally or not, but were forbidden from distributing it based on seniority. The majority of the participating schools distributed the money more or less evenly among all teachers in the school. Only 60 of the 198 districts included some measure of individual performance as a factor, leading to higher awards for some teachers and lower awards for others.<\/p>\n

You could argue that the somewhat communal nature of the whole process reduced the incentive for individual teachers to improve their own performance, since they got the same amount of bonus money for being just good enough for their school to meet its target as for being really outstanding. This is pretty weak, though, as the schools that included individual performance in their distribution plans didn’t do any better or worse than the schools that spread the money around evenly.<\/p>\n

(You can also argue that giving more money to the people who were already really good is exactly the wrong thing to do if you want to improve results. The way to make schools better as a whole is to make the worst performers do a better job, because they’re the ones pulling down the average.)<\/p>\n

— I really like this bland summary of one possible reason for the program’s failure:<\/p>\n

\n

Finally, the lack of observed results could have been due to the low motivational value of the bonus relative to other accountability incentives that applied to all the schools. Many teachers and other staff acknowledged that other accountability pressures, such as receiving high Progress Report grades or achieving Adequate Yearly Progress targets, held the same if not greater motivational value as the possibility of receiving a financial bonus. While the bonus might have been another factor motivating SPBP staff members to work hard or change their practices, they would probably have been similarly motivated without it because of the high level of accountability pressure on all schools and their staffs.<\/p>\n<\/blockquote>\n

This basically translates to “The extra money is nice, but we’re already working as hard as we can.” Which shouldn’t actually come as a surprise to anyone. This isn’t an accidental quirk, it’s the central principle of American public education: the people who go into education as a career are not the ones who are strongly motivated by money. If they were, they’d go into some other field, because education doesn’t pay very well.<\/p>\n

“But just two bullet points up, you said they were making $60,000 on average,” you complain, being sharp enough to notice that if $3,000 is 5% of a fifth-year teacher’s salary, they must be making $60,000 a year. “$60,000 a year is pretty good.”<\/p>\n

Yes and no. Keep in mind, this is $60,000 a year in New York City<\/em>, so that doesn’t go as far as $60,000 a year a lot of other places. Also keep in mind that that’s the salary for a person with four years of experience, with a Master’s degree<\/em>. That’s generally two years of education beyond a four-year college degree, which is only one year less than a law degree. A lawyer with four years’ experience would usually be making considerably more than that.<\/p>\n

And there isn’t a huge amount of room for advancement. That $60,000 figure is pretty close to the median value for most NYC schools, according to these data tables<\/a>, and the 95th percentile in those districts is under $100,000. Someone who really cares about making as much money as possible would be a fool to go into education.<\/p>\n

So, in the end, the results of the study aren’t terribly surprising. Offering performance bonuses doesn’t magically produce huge improvements, because the people who choose teaching as a career aren’t in it to maximize their personal wealth. They’re motivated by other factors, and already working plenty hard.<\/p>\n

Is this inconsistent with my long-standing contention that teachers ought to be paid more? No, because the whole point is that if you paid teachers more, you might draw in a different selection of people, some of whom would be more capable than many of the people now teaching in public schools. There are people out there who could be great teachers, but who don’t consider education as a career because of the money.<\/p>\n

Anyway, there’s a whole bunch of data in this study, which I confidently predict will make absolutely no difference. Those who favor performance pay as an education reform solution will find enough loopholes in the test program (not enough bonus money, not enough individual incentive, not enough union-bashing) to write it off as meaningless. Those who think the problem is more complicated than anything that can be fixed with a little bonus money will nod along happily, but we didn’t need to be convinced of anything in the first place.<\/p>\n","protected":false},"excerpt":{"rendered":"

One of the standard education reform proposals that gets suggested every time somebody brings up the condition of American public education is that teachers should be offered some form of performance incentive, whether in the form of “merit pay” programs on a continuing basis, or bonuses for reaching particular targets. This is one of those… Continue reading Reforming Education: Bonuses Aren’t Enough<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"1","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,13,28,82,75],"tags":[],"class_list":["post-5742","post","type-post","status-publish","format-standard","hentry","category-academia","category-education","category-politics","category-socialscience","category-society","entry"],"_links":{"self":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/5742","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/comments?post=5742"}],"version-history":[{"count":0,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/5742\/revisions"}],"wp:attachment":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/media?parent=5742"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/categories?post=5742"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/tags?post=5742"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}