The London School of Economics has a report on a study of academic refereeing (PDF) that looked at the effect of incentives on referee behavior. They found that both a “social incentive” (posting the time a given referee took to turn around the papers they reviewed on a web site) and a cash incentive ($100 Amazon gift card for meeting a 4-week deadline) worked to increase the chance of a referee accepting a review request, and improved the chances that they would meet the deadline. The effect of cash was a little smaller for tenured faculty, but they were slightly more susceptible to the social incentive.
The effect of cash is substantial in both cases, though. When offered a cash incentive, 58% of tenured faculty met the deadline, compared to about 30% without the money; for untenured faculty, it was 78% up from around 40%. It’s easy to explain this in purely economic terms– junior faculty are paid less, and thus the money means more– but for both groups it roughly doubled the fraction of faculty who got reviews in on time. Which is kind of impressive for not all that much cash.
Of course, there’s another factor at work in both of the incentives, namely respect. As with all volunteer or poorly compensated work, it’s a lot easier to get people to do it if they feel that their efforts are really appreciated. To some extent, the cash incentive serves as a tangible demonstration of that respect.
And that’s something that’s easily underrated in all sorts of interactions. One of the things I really hate about going to medical appointments is the all too common sense of disrespect– I take time off work to go there, I arrive on time for my appointment, and then I’m kept sitting around for a long time for no obvious reason. This can be further compounded by attitude during the appointment– back in 2007 or so when I had a serious acid reflux problem, the PA I was sent to at the local gastroenterology practice made it absolutely clear that his primary concern was to get me out of his office as quickly as possible. He didn’t want to listen to anything I had to say (to the point where when I said “The medication seemed like it worked for a day or two, but it’s just as bad now as it was before the pills” and he wrote down “symptoms controlled by medication”). That was almost as dispiriting as the continued symptoms themselves.
I got more useful feedback from the allergist I was seeing for a different problem (who had had reflux issues of his own) than from the expensive specialist I was supposedly being treated by for the stomach problem. Mostly because he was sympathetic, and listened to what I said, so I felt I was being taken seriously. In fact, it’s a particularly interesting comparison, because while the acid reflux issues went away as mysteriously as they started, I continue to have allergy symptoms. And yet, I have a much more positive association with the unsuccessful allergist than the nominally successful gastroenterologist, because I feel like the former treats me with respect, while the latter did not.
(There’s also a connection here to the studies that show that a significant fraction of medical malpractice lawsuits could be avoided by an apology from the doctor. People get driven into court not so much because of the financial impact, but because the feeling of being jerked around and disrespected makes them want some tangible form of revenge.)
Similarly, you have civic obligations like jury duty, which I was discussing with a colleague not that long ago. This is, as I blogged about a long time ago, another area that’s annoying not just because of the time it takes away from other things, but because of the palpable lack of respect for the time of the people being asked to serve. Jury service would be a lot more palatable if the process were re-thought so that the potential jurors felt their time was actually being valued, beyond the rote statements of thanks in the mandatory videos and comments from the judge. This doesn’t need to involve any real expense, just treating the people who are giving up a day of work with the same respect accorded to the judges and lawyers running the process.
Getting back to academic journal refereeing, in theory, simply being asked to review a paper ought to be some indication of respect within the profession. In practice, though, most people are well aware that journals are desperate for referees, and will accept reports from just about any warm body who will agree to submit a report. And while you can, in theory, list “Referee for Journal of Important Stuff in My Field” on your CV, that carries next to no weight, in part because there’s no way to confirm it. Which is probably why the social incentive works, as well– even a list of names on a web site is a more tangible token of respect than the form-letter thank you (followed closely by another paper to review) that is the usual reward.
Of course, cash is an even more substantial means of conveying respect. Which is why it shows up in lots of places. And I suspect that the conflation of cash and respect is a big part of why academics are so willing to invest disproportionate amounts of time and energy in wrangling over relatively small amounts of money.
We have a “merit pay” system at Union that provides a small extra payment to faculty whose activities for the year are judged particularly worthy (in addition to an across-the-board pay raise given to all faculty in the general direction of a cost-of-living adjustment). If you tally up the faculty time spent reporting activities and reviewing reports to make merit decisions, though, the cost in terms of salaries is almost certainly comparable to the amount of money being distributed in merit pay, but people are very attached to the system, and will argue bitterly about what should and shouldn’t count, and how the distribution should be handled. I consistently find myself sucked into these discussions, even though I worked out some time back that in purely financial terms, it’s not worth the time.
But purely financial terms aren’t the whole story– it’s also about respect as a professional academic. And for people who have passed their tenure review, the merit system is just about the only mechanism we have for indicating respect. The arguments about what should and shouldn’t count toward merit decisions are as intense as they are not because the financial benefits are life-altering, but because the criteria have a symbolic value that greatly exceeds the financial value– as an indicator of what the college as a whole finds worthy of respect.
There’s a negative effect to the conflation of cash and respect in this sort of system, though. In particular, there are a number of activities that are not generally recognized through the merit system because they are compensated through other means that are directly or indirectly financial in nature– outside work that’s paid directly, say, or activities that come with release time reducing the teaching load for that person. In financial terms, this makes sense, but when you view the money as an indicator of respect, it ends up having a negative psychological effect. While it’s a logical solution to the problem of limited resources with which to reward faculty, it’s difficult to keep people from feeling that their hard work has been disrespected on a professional level, even if they’re been appropriately compensated in purely economic terms.
Of course, the conflation of money and respect can get way more toxic than it does in academia– the service industry has it way worse. I suspect there’s a strong element of confusion between cash and respect going on in the problematic culture of tipping in the US, for example. All too many customers evidently feel that the financial transaction associated with buying a meal (or whatever) takes the place of needing to treat their servers with non-financial respect. And while the obvious solution would be to raise salaries for service industry workers to the point where tips aren’t financially necessary, changing the social norms so that not getting a tip doesn’t feel like an act of disrespect is a tricky issue.
So, anyway, kind of a meandering post, I know. Anyway: respect, it’s a Thing. Which may or may not involve the exchange of cash, but needs to be considered when thinking about how to motivate people to do important tasks.
And here, have a music video as a reward for reaching the bottom of the post:
All too many customers evidently feel that the financial transaction associated with buying a meal (or whatever) takes the place of needing to treat their servers with non-financial respect.
There’s at least anecdotal evidence that it’s worse than that, that many tippers think that they’re entitled to a creepy degree of control over servers because they’re tipping. See this shorter Slate overview article and this more detailed bit from a series written by a restaurant owner who abolish tips in favor of a service charge.
abolish tips in favor of a service charge
Or better yet, include taxes and gratuities in the price on the menu, which is standard practice in Europe. (Rounding up the bill is tolerated.) Over there, if the menu says that dish costs EUR20, it costs EUR20. Here, depending on where the restaurant is, your $20 course will cost you $24-26, because taxes and gratuity are not included in the quoted price.
As for refereeing, the current system has a perverse incentive. Not only is there no explicit reward for doing it well, but scientists who demonstrate proficiency are often “rewarded” by getting more assignments, while an editor who finds that a reviewer consistently gives superficial and/or late reviews tends not to give that reviewer future assignments. This is part of why I have seen turnaround times of 12 months or more on some of my proposals.
I recall a few times when I felt compensation was in order. Have you ever been the third referee on a paper, because the first two wildly disagree? Or an appeal referee when the author insisted on one? Those are nasty jobs. Or be the second or third to get the paper because the first ones couldn’t stand to read it or wanted to delay a competitor but didn’t want to actually reject it?
I am lucky to have a doctor who listens to me, but they also do a good job of scheduling (at least in the morning when I make my appointments).
I am also lucky to live in a community that has contiuously improved its jury selection bureaucracy. For example, they now have a secondary pool that is called for noonish (you phone in after 10:00) rather than bringing in 30 extra people at 8 am to be sure they have a big enough pool for all of the juries being picked that day. (They pick all juries on the same day even if the trial doesn’t start until mid week.) Not so easy if you need to have someone cover a class, but much easier on many employers.
We also have the 21st century option of deferring jury service to a particular date within a 2 month window. A web calendar shows the dates when they will be seating juries but haven’t yet sent out notices. That must make their job 100% easier.
Thanks for the Queen of Soul video, but I would have felt more respected if it wasn’t lip-synched!
Having reviewed my share of physiology papers over the years, I agree wholeheartedly with your thesis that reviewers deserve a little respect from the publishing community. It is especially galling when you see how much they charge our libraries to subscribe to the journals that depend on our uncompensated work.