Sports Statistics and Numerical Arrogance

If you’ll forgive the foray into sports, there was a piece in the Atlantic Wire that I thought was interesting and worth pondering even if you’re not at all interested in sports and just interested in the policy. It’s entitled “Why people still don’t believe the best ideas in sports,” and as you might guess from the title, it’s devoted to berating those poor souls who haven’t gotten the value of advanced analytics in decoding sports.

Can we stipulate that evidence is good and a well-informed statistical analysis can take the biases out of our observations? They can. But this particular piece goes too far in the statistical direction, I think:

Another project designed a "similarity network" to group NBA players by their characteristics, defining 13 different categories of players, as opposed to the traditional 2 guard, 2 forward, 1 center framework. (You can see a version of the presentation here.) Yet, seconds after it ended we heard two guys loudly disagreeing with the presenter's classification of Minnesota's Kevin Love, based on ... what exactly? Muthu Alagappan has charts and data points and a Biomechanical Engineering degree from Stanford University. You have NBA TV on your cable package. Who would you believe?

People love evidence ... when it tells them what they want to hear. Once they hear something that doesn't intuitively make sense to them, they fight back. ….

This stuff is the new Moneyball, a book that had its cover image plastered around the convention center as one of the pillars of the sports analytics movement. Yet some audience members audibly scoffed when Alagappan posited (via a big spreadsheet with lots of decimal points) that Devin Ebanks might be just as valuable to the Lakers as Carmelo Anthony is to the Knicks. As if the idea that a cheap journeyman might be able to provide skills similar to that of a overpaid superstar had not been the premise of a bestselling book and Oscar-nominated movie that gave sports analytics its cultural relevance. Maybe Alagappan's numbers are wrong, but you better bring your own numbers.

I’m not so sure this is always true. Look—what are numbers? They’re basically another language; like all language, it’s nothing but a tool. Feel free to substitute “words” for “numbers” in the previous sentence—does it still make sense to you? Of course it doesn’t.

It’s very possible to defeat a statistical argument with the use of words alone if, for example, the assumptions underlying the statistical analysis are not at all well-founded. For instance, words alone can defeat David Berri’s statistical analysis (see: here) with little to no numbers needed. (I’m still not sure why David Berri is cited so often in the media, but if you ever see his name or his statistical analysis being cited in the piece, it’s a good time to stop reading the article and perhaps writing a kindly e-mail advising the journalist as to the error of his ways. Sometimes we don’t know better.)

I worry about the wrong people brandishing numbers because they can discredit all the people using numbers responsibly; if I want to present a statistically-based argument as to why you probably should rethink your prostate exam and/or mammogram guidelines, it doesn’t help to have a large number of arrogant people who have come before you arguing that that the numbers prove everything, your common sense matters little, and thereby defeats discussion. I don’t think Bennett—the author of this piece—is that kind of guy, but the tone of the piece often tends in that direction. Let’s try and do discussion a little better, maybe?