The Signal and the Noise
Why So Many Predictions Fail - But Some Don't
Ratings97
Average rating4
The signal and the noise is all about prediction. It starts with the subprime mortgage financial crisis and discusses the combination of perverse incentives and overconfidence that caused the rating services to fail to accurately portray the risks of those securities (primarily the assumption that even with housing prices astronomically high, the risk of default of each individual mortgage was completely independent rather than affected by the economy). Next he looks at television pundits and the fact that more television appearances is negatively correlated to forecast accuracy. Here he gives a solid introduction to Philip Tetlock's work on forecasting, which can be found in more depth in his book Superforecasting. He touches on baseball, an information-rich environment, before moving on to irreducibly complex problems like the weather, seismic activity, and the economy where you fundamentally can't get anywhere near enough raw data or information on interactions between data points to paint a complete picture.
The second half moves towards giving you an idea how to approach problems probabilistically and how to improve and refine your process over time. He starts with simple problems like sports and poker before moving onto more complex problems like terrorism and global warming.
I wouldn't consider this book a complete guide to rational, evidence based decision making (ignoring that it doesn't give you the math), but it's a pretty accessible introduction to the topic and is largely technically sound. It's a solid place to start.
We like future predictions to be presented to us as absolutes: this stock will rise, it will rain, this team will win. Our brains have a hard time making sense of uncertainty and margins of error. We like our realities to be oversimplified, have clear incentives for taking actions. Plus, we don't like to get wet when we leave our umbrellas at home despite being told there's a minimal chance of rain (this goes so far that the weather channels deliberately tweak what their models predicted, to avoid angry rained-on viewers, see Wet Bias).
This book is from 2012, post Moneyball and post 2008 financial crisis. Predictions and the rise of big data was on everyone's mind. Who predicted the housing bubble and who made money of it? Silver goes through different domains where predictions and probabilistic thinking play a major role: the weather, baseball, poker, chess, the stock market, the climate. He starts on how we fail at predicting the occurrence of earthquakes, despite knowing the likely frequency of their occurrences. And he ends on the statistics of terrorist attacks, whose ratio of frequency and impact (their ‘power laws') are not unlike earthquakes.
In hindsight, after the occurrence of an event, it's always easy to look back and see the signal that hinted at it. Yet, while we're in it, it's incredibly hard to distinguish which part of the data is a signal (clear indicator) and which is noise (irrelevant fluctuations, irrelevant other signals). Which part of a baseball player's results are skill and which are luck? Statistical analysis and the accumulation of large amounts of data have helped with isolating signals in some domains, but not all. We're also living in a world that's increasing its volume of information exponentially, tied together in complex systems, making it increasingly important to develop probabilistic thinking. Bayesian logic is the use of knowledge of prior events to predict future events, while continuously updating your hypothesis about the future, when new data comes in. (So far, every morning the sun has risen from the east, therefore I assume it will rise from the east tomorrow morning as well).
Great book, still highly relevant of course. The climate chapter felt a bit messy, and there potentially was a bit too much content from the author's old poker-days, but else this was a highly informative and interesting read. Good to get comfortable with the ‘known unknowns' before having to deal with ‘unknown unknowns'.
What fun read about predictions on various fields from nature, politics and sport. Been fan of Nate Silver through fivethirtyeight.com and sports predictions but he has grown ever bigger in my mind. If I should review the whole book with one sence the quote from John Maynard quote “If facts change, I change my mind”. Actually, ‘m putting this in my social media bio.
Solid Application of Statistics. I'm a math geek who has casually followed Silver's work since he came on the national radar after the 2008 Presidential election. In this book, he uses his own mathematical background and many interviews to show how probabilistic statistics (vs more deterministic statistics) gives us great insight into a wide range of issues, from the mundane yet popular topics of poker and baseball - things he has personal experience with using statistics on - to the seemingly more substantial issues including weather forecasting, political polling, climate change and even terrorism. And overall, he is very careful to stick to his central point: follow the numbers, no matter where they lead - which he calls the “signal”. Very highly recommended for anyone trying to have a genuine discussion on really almost any topic.
This book has an ironically low signal to noise ratio. It's far too long for its insight density and is unbelievably Ameri-centric. If you know any math, or have read LessWrong you'll get much out of this book.
It goes without saying that “popular statistics” book is mostly an oxymoron. On the one hand, statistics is largely a very dry field. On the other hand, those of us who do understand statistics (and even freaks, like my husband, who enjoy statistics), find any attempt at popular statistics largely too elementary to be interesting. Nate Silver doesn't just walk the fine line in the middle, he eliminates it and writes a completely novel statistic book that is appealing to both the mathematician and the math hater: this book fascinates.
Nate Silver focuses on the forecasting in areas that are difficult to predict: weather, climate, earthquakes, poker, politics, chess and sports. Each of these areas is individually interesting – I had never spent much time considering online poker, for instance, and the chapter focusing on poker is not just mathematically-focused, but also an expose on the world of online poker and the life and times (or at least the two year subset thereof) of Silver's 6-figure gambling career. In addition, his overall thesis, which seems to be that we should use Bayesian analysis to think probabilistically about the world and continually evaluate our probabilities both builds naturally and has far-reaching applications.
I feel like I have spent years of my life trying to explain to medical students (and more advanced physicians who should really know better) why every time a paper is published with a p<0.05 we can't totally disregard all prior medical knowledge and dive after the new information. Silver's easy explanation of Bayes' theorem nicely summarizes why this is true - that alone should make this a must-read for anyone in an academic field.
