March 14, 2014

Outfoxing the hedgehogs

A review of Nate Silver’s The Signal and the Noise.


In the second chapter of his book The Signal and the Noise Nate Silver uses the old Russian parable of the hedgehog and the fox to explain how he is different from most other political forecasters. According to the parable, the fox knows several tricks to escape from predators, while the hedgehog knows only one – to curl up into a ball. The parable ends with the fox trying his myriad tricks but falling prey as none of them work, and the hedgehog ‘winning’.

In the field of political forecasting, however, Silver argues that one better be a fox than a hedgehog. For the uninitiated, Silver is a political analyst who runs the popular FiveThirtyEight blog for the New York Times. He shot to fame in 2008 when he correctly ‘called’ how 49 of the 50 states of the United States would vote in the presidential elections. In the following elections in 2012, he did one better, correctly calling all fifty states.

Most political commentators, Silver says, are like hedgehogs and usually have only one ‘trick’ –  one pet theory based on which they tend to call the elections. They tend to be consistent in their forecasts, and since they seldom change their minds, their message comes out as being strong and they make for good television punditry. Silver, however, argues that the skills that make for good punditry are not the same that makes one a good forecaster, and in order to be a good forecaster, one needs to be like a ‘fox’, in other words, have “several tricks”. In that profession, Silver says, one also needs to be amenable to changing one’s views – though that can give you the appearance of inconsistency, it will enable you to make better forecasts. After all, economist John Maynard Keynes once said, “When the facts change, I change my mind. What do you do, sir?”

The book, however, goes much beyond election punditry, the field Silver is best known for (that is restricted to one chapter). As the title states, the book is about separating the signal from the noise – the art of gleaning useful information from data in the face of information that is largely worthless and can sometimes serve only to confuse. Silver takes up a plethora of topics – ranging from climate change to computer chess – to help the reader understand the power and limitations of data.

In the chapter on terrorism, for example, Silver examines frequencies and impact of various terror attacks, showing that like earthquakes (covered in an earlier chapter), terror attacks too follow a power law distribution. He draws parallels between the Pearl Harbour bombing and 9/11 and argues that both were a case of “unknown unknowns”. The chapter on earthquakes seeks to explain the pitfalls of the concept of ‘overfitting’, which Silver calls “the most important scientific problem you’ve never heard of”.  When a model gets too specific to the data it is modelled on, it can fail spectacularly when it comes in contact with outside data, and is thus useless as a predictive tool.

Silver uses the chapter on baseball analytics (a field he made his name in with the ‘PECOTA’ system before he turned his eye to politics) to explain what to me is the biggest takeaway from the book. After the “moneyball revolution” at Oakland Athletic, he says, every team now uses statistics. However, contrary to what people expected, he says that the reliance on old-fashioned scouting has not gone down. Silver says that there is so much that data can tell you and there are some factors which can be understood only with human judgment (a player’s attitude or discipline, for example, which is not captured by data in inter-school games). He returns to the topic later in the book when he tells the story of computers in chess.

As a teaching assistant in business school, the concept I found hardest to teach was Bayes’ Theorem of conditional probability. Silver roughly dedicates the second half of the book to the theorem and the associated field of Bayesian statistics, and does a masterful job of explaining the theorem – both the mathematics and philosophical underpinnings – without ever getting too technical.

He introduces the concept to us as part of a chapter on sports betting (which is told through the lens of the story of NBA punter Bob Voulgaris). In the beginning of the 1999-2000 NBA season, odds on the Los Angeles Lakers winning the title lengthened from 4-1 to 6 ½ – 1 following a bad start to the season. A purely ‘frequentist’ statistician (one that only takes into account of the observed ‘samples’, not taking into account any ‘prior’) would have, after three consecutive losses, concluded that the Lakers were extremely unlikely to win the title. But Voulgaris (described by Silver as a ‘Bayesian’), based on his ‘prior’ concluded that the Lakers were simply going through a bad patch, and the odds underestimated their chances of winning. And he bet on them, eventually making a fortune as they won the title.

While telling the story, however, Silver cautions us that while Voulgaris may have been right in backing the Lakers at the beginning of the season, by choosing to not hedge his bets (by betting against the Lakers, for example, once the odds shortened), Voulgaris left himself exposed and nearly lost all his money as the Lakers encountered difficult playoff matches against the Sacramento Kings and Portland Trailblazers.

Silver dedicates an entire chapter to uncertainty in prediction, and states that humility is a key necessity to become a forecaster. Even the most sophisticated sports better, Silver says, turns out to be right only 57 percent of the time. As a political forecaster, he says that he is always prepared to be proved wrong (though that has rarely happened in reality), and he makes it a point to attach a “confidence interval” to each of the forecasts he puts out. Invoking the Bayesian spirit, he mentions that each forecast is an opportunity for the forecaster to improve his model, and accuses of the ‘hedgehog’ television pundits for failing to respond adequately to their failed forecasts.

While the topics are broadly well chosen and the messages delivered in Silver’s characteristic conversational style, the book is long (at 544 pages) and some chapters seem to drag. Since each chapter is a story, most include a lengthy background, which further adds to the bulk. For example, the first chapter is mostly a summary of the 2008 financial crisis and makes you wonder if you are reading a book on numbers or on finance. And there is more finance later on when Silver talks about the Efficient Market Hypothesis. In choosing related topics (two chapters on finance, four on sport and three on the environment), Silver runs the risk of putting off people who are not comfortable/interested in the chosen topics.

Fatal error: Uncaught Error: [] operator not supported for strings in /home/thinkpra/public_html/archives/wp-content/themes/layerswp/core/helpers/post.php:62 Stack trace: #0 /home/thinkpra/public_html/archives/wp-content/themes/layerswp/partials/content-single.php(81): layers_post_meta(6153) #1 /home/thinkpra/public_html/archives/wp-includes/template.php(732): require('/home/thinkpra/...') #2 /home/thinkpra/public_html/archives/wp-includes/template.php(676): load_template('/home/thinkpra/...', false, Array) #3 /home/thinkpra/public_html/archives/wp-includes/general-template.php(204): locate_template(Array, true, false, Array) #4 /home/thinkpra/public_html/archives/wp-content/themes/layerswp/single.php(20): get_template_part('partials/conten...', 'single') #5 /home/thinkpra/public_html/archives/wp-includes/template-loader.php(106): include('/home/thinkpra/...') #6 /home/thinkpra/public_html/archives/wp-blog-header.php(19): require_once('/home/thinkpra/...') #7 /home/thinkpra/public_html/archives/index.php(17): require('/ho in /home/thinkpra/public_html/archives/wp-content/themes/layerswp/core/helpers/post.php on line 62