Stats Man Nate Silver Releases His First Book, The Signal and the Noise

Speaking with the Odds God

Picture future statistician and FiveThirtyEight/New York Times blogger Nate Silver in 1984, brandishing a bat at home plate during a Little League game in East Lansing, Michigan.

His idols, the Detroit Tigers, will win the World Series that year, but six-year-old Nate isn't fantasizing about the Commissioner's Trophy—he's soberly assessing his level of talent and weighing his chances of success. He concedes he's not a good hitter but thinks he's pretty fast, so he envisions a counterintuitive strategy: "The pitchers can't pitch . . . draw a lot of walks . . . get on base and then steal." Even at this early age, in an environment where pressures from peers and parents would have demanded a simple, aspirational model for batsmanship—something like "hit a home run with the bases loaded"—Silver knows to ignore the clamor of the crowd and make a decision informed by hard truths, even if it doesn't make him look good.

"It was such a wussy style of playing," he admits now, but "it was helping my team, so it was totally logical."

Back then, he knew enough not to share such dispassionate strategies with his teammates, but since then, in the worlds of sports, business, and political analysis, Silver has made his name (first as anonymous blogger "Poblano") by applying his jaundiced eye to the world of forecasting. He began with a brief career in professional poker, followed by the development of PECOTA, a system for predicting the performance of Major League Baseball players, and crowned his achievements with the blog FiveThirtyEight.

That site proved to be one of the most accurate political meta-polls during the 2008 presidential race (he called every state except Indiana) and continues to testify to Silver's influence. So swift was his rise to King of Geekdom (Time named him to its 100 Most Influential People list in '09) that his followers lacked a bible. Now, though, he's releasing his first book, The Signal and the Noise (The Penguin Press, 352 pp., $27.95), a substantial, wide-ranging, and potentially important gauntlet of probabilistic thinking based on actual data thrown at the feet of a culture determined to sweep away silly liberal notions like "facts." For Silver, the key to successful prognostication is a clear-eyed examination of the difference between "noise," misleading or biased methods or faulty data sets, and "signal"—that which is likely to turn out to be true, and whose significance often seems obvious in hindsight.

In 13 chapters, he covers a panorama of the unpredictable and the state of mankind's ability to conquer it. Or come close, anyway. He analyzes our failure to predict the implosion of the housing bubble; he compares our talent for predicting the paths of hurricanes with our ineptitude at auguring massive earthquakes. His PECOTA radar picks up Dustin Pedroia before he becomes an MVP. He spills the secrets of TV meteorologists, debunks TV pundits (The McLaughlin Group is "explicitly intended as slapstick entertainment for political junkies"), and ponders the likelihood of a SARS epidemic. He profiles successful sports gambler Bob Voulgaris, separates the data from the hysteria surrounding global warming, and tells us how afraid of terrorism we should be (about as much as we should be scared of earthquakes).

Despite its astoundingly broad range of topics, The Signal and the Noise isn't a haphazard compilation of magazine articles; it feels as if Silver wrote it in one long toss of his Merlin's cape. He ties his baker's dozen of predictions together through the work of 18th-century English minister and statistician Thomas Bayes, a somewhat shadowy figure whose posthumous "An Essay Towards Solving a Problem in the Doctrine of Chances," writes Silver, "is a statement . . . about how we learn about the universe: that we learn about it through approximation, getting closer and closer to the truth as we gather more evidence." In the book, he cites early Bayesian Richard Price's analogy: "A person emerges into the world and sees the sun rise for the first time. At first, he does not know whether this is typical or some kind of freak occurrence. . . . Gradually . . . the probability he assigns to his prediction that the sun will rise again tomorrow approaches (though never exactly reaches) 100 percent. This contrasted with the more skeptical viewpoint of . . . David Hume, who argued that since we could not be certain that the sun would rise again, a prediction that it would was inherently no more rational than one that it wouldn't."

Bayes's theorem seems as self-evident as a Malcolm Gladwell thesis, but it has grand implications for a society so much in the thrall of spin, fantasy, overconfidence, and hope. "This is not a postmodern kind of book," says Silver with a nervous chuckle over his turkey, bacon, and cheese club sandwich at the Clinton Park Café in Brooklyn. "It's saying there is truth, but we can't know it, and it's hard for people to accept both those propositions. It requires you to accept that you'll always be a flawed, imperfect creature who's struggling to get better. Bayesianism recognizes the imperfection of our knowledge . . . so you have a starting point where we say, 'Here are our assumptions, and here are our beliefs,' and then you set up a structure to evaluate new information with those beliefs and biases."

1
 
2
 
All
 
Next Page »
 
My Voice Nation Help
5 comments
TimSimms
TimSimms

He is so cute, I can't even stand it. Now I find out he's gay, and I'm totally in love. Smart, sexy... who could want more?!

99GR81
99GR81

@fivethirtyeight genius Nate Silver's 1st book is out: 'The Signal and the Noise' http://t.co/XzQ98z6b #yam via @cthoen

artved
artved

@timkastelle did you finish it already?

timkastelle
timkastelle

@artved yes I did. It's not perfect, but it's very good - definitely worth reading.

artved
artved

@timkastelle thanks, grabbed @kindle version

 
Loading...