Kate and I went down to New York City (sans kids, as my parents were good enough to take SteelyKid and The Pip for the weekend) this weekend, because Kate had a case to argue this morning, and I needed a getaway before the start of classes today. We hit the Rubin Museum of Art, which is just about the right size for the few hours we had, got some excellent Caribbean food at Negril Village, then saw The Old Man and the Old Moon in a church basement at NYU (the show was charming, the space was stiflingly hot by the end). All in all, a good weekend.
I drove back Sunday afternoon, and was reminded how, psychologically, the way I relate to music is totally shaped by past technology. I took so many long trips in the cassette tape era, I automatically assume that I’ll either get specific songs in a specific order (I always expect “Beast of Burden” right after “Ziggy Stardust,” and “Ob-La-Di Ob-La-Da” after “American Pie,” because I was so fond of the mix tapes that had those combos on them), or completely unpredictable songs via the radio. So, as I neared Sloatsburg on the NY Thruway, I caught myself thinking, “Wow. This is a really good run of a bunch of good driving songs.” Of course, as I realized a second later, that was because I had the iPod running shuffle play on the “really good driving songs” playlist.
In an effort to get a little low-impact science content out of this “I’m a dumbass” anecdote, another thing happened with the playlist that’s worth a little math. As I said, I have a “really good driving songs” playlist on my iPod, which runs to 335 songs (I own a lot of music…). 18 of those are by The Hold Steady, one of my favorite bands working today. My iPod seems weirdly averse to playing them, though, so whenever I go on a trip using that playlist, I play a game of “How long will we go before hearing a Hold Steady song?”
Yesterday’s trip home, about three hours door-to-door, went through 51 songs, not one of them a Hold Steady song. This is pretty typical, at least in my memory, but how unlikely is it, really?
Well, the Hold Steady constitute 18/335 = 5.37% of the “really good driving songs” playlist, which means that in a random draw there’s a 5.37% chance of a Hold Steady song coming up first. And thus, a 94.63% chance of the first song not being a Hold Steady song.
A really simplistic way of estimating the likelihood of making it through yesterday’s drive without hearing a single Hold Steady tune would be to raise that fraction to the power of the number of songs:
$latex P(51)=0.9463^{51} = 0.0599 $
So, the odds of yesterday’s Hold Steady free ride should be a bit under 6%.
But, of course, that’s a really naive, physicist-y estimate. In order for that to be accurate, you would need to put each played song back into the draw. But that’s not what happens in reality– my iPod will shuffle its way through the whole 335 songs once before replaying a song. So, in fact, the odds of hearing a Hold Steady song go up as the shuffle goes on. There’s a 18/334 = 5.39% chance of hearing them on the second song, and 18/333 = 5.41% chance on the third song, and an 18/285 = 6.32% chance at the 51st song. To properly handle this, we need to count how many ways there are to play 51 songs out of a set of 335 songs, without ever playing one of those 18 Hold Steady tunes.
So, how would you go about that? Well, the easiest way to estimate it would be to count the total number of possible combinations of 51 songs that aren’t by the Hold Steady (317 of them):
$latex 317 \times 316 \times 315 \times … \times 267 = 5.09 \times 10^{125} $
and then divide by the total number of combinations of 51 songs out of the full set of 335 songs:
$latex 335 \times 334 \times 333 \times … \times 285 = 1.08 \times 10^{127} $
Take the ratio of those two numbers, and you get… 4.698%. Which really isn’t all that different from the ultra-naive estimate, is it? That’s due to the fact that 51 isn’t all that big a fraction of 335. The agreement gets much worse as the number increases– at 51 songs, the correct value is 78% of the naive value, close enough for back-of-the-envelope. At 61 songs, it’s only 70%, at 71 songs it’s 60%, and by 101 songs, the correct value is a third of the naive value (which is down to 0.4%).
(You might argue about whether the order of the songs matters– the above counting is for the case where order matters, so abcdef and abcdfe would be counted as two different strings even though they contain the same six characters. If you think you only care about which songs get played, not the order, the relevant numbers are $latex 3.278 \times 10^{59} $ and $latex 6.978 \times 10^{60}$ for a ratio of… 4.698%. So it doesn’t actually make a difference. But if you’re in the car, the order matters, thus the above….)
So, is this incontrovertible mathematical evidence that my iPod hates my favorite band? Not necessarily, because Sunday’s drive was a single event. And while the probability of things coming out that way is low (a Hold Steady free drive should happen roughly one out of every 20 trips to Manhattan), you can’t really say anything about the probability of a single event. Anecdotally, though, this seems to be pretty common– on the way down, we made it almost to the New Jersey border before getting a Hold Steady song, and on at least one previous occasion I made it into The City without one. So I’m suspicious… maybe Craig Finn stole Steve Jobs’s girlfriend, or something.
Or, maybe I was just unlucky. Tough to say, really, without more data. I guess I’ll need to make more regular trips to New York and back, keeping track of what gets played. And as long as I’m doing this– for SCIENCE!, you know– I might as well get some good food and arts while I’m there. Now I just need to convince the NSF to reimburse my mileage…
For extra credit, seize on the fact that the Many-Worlds Interpretation predicts that there will always be some branch of the multiverse in which I drive between New York and Schenectady without hearing a Hold Steady song, and attempt to draw sweeping conclusions from this about the validity of the MWI. (Extra credit offer void in California, Utah, and for people named Neil Bates.)
But that’s not what happens in reality– my iPod will shuffle its way through the whole 335 songs once before replaying a song.
I haven’t tried this with an iPod, but a few years ago I observed iTunes in shuffle play to pull up the same song twice in a row (I don’t mean a duplicate version, I mean the same artist and album), and it wasn’t so unusual to hear a song and see it (again, the exact same track, not a duplicate version) appear a second time in the list of the next 15 songs. However, that was several versions of iTunes ago, and I haven’t re-run the experiment recently.
I’m sure you know this, but isn’t the answer that the iPod’s shuffle feature is not perfectly random?
The question is how non-random is it, and in what way? That is, is it giving priority to some songs/artists over others? It’s not a higher rating thing, because all the songs on that playlist are either 4 or 5 stars.
Eric Lund: There’s a setting for that, actually. There’s “choose a song at random after each song” and then there’s “shuffle into a list then play the list in order”.
Ori Vandewalle: There’s actually a slider in iTunes to determine HOW RANDOM the random shuffler should be. “Truly random”, the default, feels nonrandom to many users because it will produce trends and patterns – you might get 3 AC/DC songs in a row, or something, So there’s a slider to make it *less* random by forcing it to pick songs farther from the one it just played – after it plays AC/DC, other AC/DC songs are downweighted to be less likely.