Shannon Phillips

This book was Sam’s Christmas present to me, since I like all right-thinking people was obsessed with Nate Silver’s blog during the election season.

I liked the book a lot better than the title. Well, actually, the title is fine, but the subtitle—Why So Many Predictions Fail…but Some Don’t—is terrible. It makes it sound like you’re in for some kind of dry introduction to the fundamentals of statistical analysis. The book is actually far more interesting than that. It’s about the ways in which the edifices of our society—financial, political, even physical—are built on “knowledge” that in some cases is flawed at the very root.

Just as an example, there’s one part of the book where Nate Silver pulls off one of the strongest gut-punches I can remember experiencing as a reader. It starts off with him discussing, innocuously enough, the science of weather prediction. It turns out that the science is better than you might think, but that local weather forecasters deliberately skew their predictions in order to account for some common biases in the audience. For instance, people want their weather forecasts to err on the side of caution. They would much rather be told to tote an umbrella, and have the day turn out to be sunny, than to be assured their Sunday will be nice and then get caught out on the golf course in a sudden thundershower. So local weather forecasters, understanding this preference in their audience, will routinely inflate their estimates of the likelihood of bad weather. If the weather model says there’s a fifty percent chance of rain, you’ll hear on the news that there’s a sixty or seventy percent chance.

This all seems interesting, if academic: it’s a neat bit of trivia, right? Then Silver sinks in the knife:

The weather forecasters did not make any apologies for this. “There’s not an evaluation of accuracy in hiring meteorologists. Presentation takes precedence over accuracy,” one of them told Eggleston. “Accuracy is not a big deal to viewers,” said another. The attitude seems to be that this is all in good fun—who cares if there is a little wet bias especially if it makes for better television? And since the public doesn’t think our forecasts are any good anyway, why bother with being accurate?

This logic is a little circular. TV weathermen say they aren’t bothering to make accurate forecasts because they figure the public won’t believe them anyway. But the public shouldn’t believe them, because the forecasts aren’t accurate.

This becomes a more serious problem when there is something urgent—something like Hurricane Katrina. Lots of Americans get their weather information from local sources rather than directly from the Hurricane Center, so they will still be relying on the goofball on Channel 7 to provide them with accurate information. If there is a mutual distrust between the weather forecaster and the public, the public may not listen when they need to most.

Ouch.

Silver goes on to discuss Katrina in a lot of detail, and he’s very evenhanded about the multiple factors that made the disaster so awful, but from his perspective the most relevant detail is that it was predictable. We knew what was going to happen in New Orleans, we knew the city had to be evacuated, and yet the evacuation was botched. And one of the factors working against those who struggled to convince New Orleans residents to leave their homes before the flood came was this widespread skepticism about disaster predictions, a skepticism that is in fact rational given the fact that bad weather predictions are routinely inflated and overhyped for the sake of news ratings.

Silver’s book covers a lot of topics. Sports, the stock market, politics, even national security—his comparison of the fields of earthquake prediction and terrorism prediction is really fascinating. At base it’s a book about how we make decisions, how we gain knowledge about the world, how we know what we know. And how our unexamined biases undermine us, and destabilize our society. It’s a great read.