Change Estimation and the Wisdom of Crowds

The results of the estimation contest are in. There were 164 serious entries (I excluded the $12,000 and $1,000,000 “guesses” from the final data). The mean value guessed by commenters was $83.30, and the median was not far off, at $77.12. The standard deviation was high– $43.10– but as you would expect with a large sample, the standard error (or standard deviation of the mean) was small, $3.37.

Or, in convenient graphical form:

i-c1692278b52f26c7e57424c221c5ca75-change_estimation.jpg

That’s a histogram with $20 wide bins showing the number of guesses in a given range. A pretty nice distribution, on the whole.

The red line indicates the actual total value of the change in the box, $165.26 (not counting $1.10 in Canadian coins that the bank wouldn’t take). Congratulations to Michael Day at comment #48, who wins the contest with his guess of $166.32, just $1.06 off from the actual total. Send me a mailing address, and I’ll send you a galley proof of How to Teach Physics to Your Dog.

Now, why were the rest of the guesses so far off? Isn’t the “wisdom of crowds” effect supposed to make an average of a large number of estimates better than any individual guess?

Well, for one thing, it’s a little difficult to make an accurate estimate of the size of the box and the depth of the coins from the image I provided. The angle of the shot makes it particularly tricky to gauge the depth.

Another factor is that my behavior with regard to coins is a little different than most people. Whenever possible, I discard pennies in those “take-a-penny, leave-a-penny” cups or boxes. As a result, quarters, nickels, and dimes are overrepresented in my change box relative to those of people who keep pennies. People trying to estimate based on past experience were almost inevitably going to come in low.

Another likely factor would be the “winner’s curse” effect discussed by the Albany Math Circle blog. People seem to systematically underestimate the value of large collections of coins. Though, admittedly, a factor of more than 2 seems a bit much.

Finally, I think there was probably a collective effect, in that the first batch of guesses were way low, and later guessers took those as some sort of consensus value and looked in the same range. The median separation between guesses was $0.67, so they’re really very tightly bunched in the middle, which I think says something about the psychology of people entering the contest.

Other guesses? Comments, questions, complaints about the contest?

(For the record, I would’ve guessed $130, based on assuming the box was about half full, and the sample was entirely quarters with a volume filling factor of 50%. So I would’ve lost my own contest…)