In recent months, thousands of young Australians have massed outside the Spanish retailer Zara’s stores, waiting to get in and experience the merchandise.

At the same time millions of Australians have been listening to, and reading, the predictions of political pundits, economic forecasters, broking firm analysts, experts and others about what might happen in politics and the world.

While admittedly in a more limited area, the Zara people have a better track record. Why do we listen to one group and not the other? Duncan J. Watts explains much of it in his new book, Everything is Obvious Once you Know the Answer (Atlantic, 2011). Watts is one of the world’s leading social network researchers. Having been a Columbia University sociology professor, with a background in physics, he is now a researcher at Yahoo!

Before now Watts’s fame relied on the work he had done on small world theory — the six degrees of separation problem. Whenever you mention the problem in Crikey, some reader jumps in and says it was discovered by the Hungarian poet Frigyes Karinthy who mentioned a version of six degrees of separation in his short story Chains. Others mention  town planning visionary Jane Jacobs, but most emphasise Stanley Milgram who seemed to have proved the hypothesis. Hooke guessed much of what Newton wrote about, and was probably right on some questions in which Newton was wrong, but Newton proved it. Watts, while he would be embarrassed by the comparison, largely proved Karinthy, Jacobs and Milgram. Not only did he prove it — but he also largely demonstrated where they were wrong in how social networks work and how the six degrees of separation operate.

Watts’s new book elucidates “the paradox of common sense … that even as it helps us make sense of the world it can actively undermine our ability to understand it”. He argues that common sense fails us for three reasons: first, when we think about why people do things we focus on things of which we are aware (“incentives, motivations … beliefs”); we extrapolate our beliefs about individuals to groups and the wider population; and, we learn less from history “than we think we do”.

On top of that we practise circular reasoning imagining that because things happen what happened was the reason they happened; confusing the macro and the micro in behaviour; and, consistently failing to discriminate between the relevant and the irrelevant. In this respect he talks a bit about philosopher Arthur Danto’s argument that what is relevant can only be known after the event.

He is particularly good on the area in which many pundits fail — making a distinction “between predicting outcomes and predicting probabilities of outcomes”. This is a fundamental distinction that “should change our view about what kinds of predictions we can make”.

Using the Zara example he argues that we spend too much time on “predict and control” rather than “measure and react”. For those not familiar with Zara, it is distinguished by two things: making clothes for very small slim people and having a supply chain that allows them to turn around stock holdings in two weeks.

I doubt that many in the media or the broking industry will read the book. Recent Stockbroking Hall of Fame entrant Robert “Coppo” Coppleson may be the exception but it has much to say about many current Australian issues.

On controlling carbon emissions, it cites Friedrich Hayek’s paper The Use of Knowledge in Society. The IPA was once an assiduous disseminator of Hayek’s work but seems to want to ignore his view that government-designed solutions (the Abbott direct-action plan) are always worse than market-based ones.

The Halo Effect is another example. As Watts says “The Halo Effect … turns conventional wisdom on its head. Rather than evaluate the outcome being determined by the quality of the process that led to it, it is the observed nature of the outcome that determines how we evaluate the process”.  Most corporate reputation indices, and their methodological failings, are prime examples of this. Companies have good reputations because respondents to reputation surveys believe they have.

It has an interesting discussion of science sociologist Robert Merton’s views on the Matthew Effect, which describes how successful people get more successful; the media’s preference for “human-centred narratives (as opposed to) abstract explanations based on social, economic and political forces”; and, why CEOs are not as important as they, and their remuneration committees, believe.

He also explodes much of recent conventional wisdom about “tipping points” and the role of significant influencers in social networks. Ironically, Malcolm Gladwell has a glowing bit of blurb on the cover of the edition I have, despite the fact that he proves that much of Gladwell’s stuff on tipping points and influencers is wrong.

Reading Watts could make it easy to despair about politics, the media, business and the public sector. But when you next hear bankers talking about how awful post-GFC regulation is — read his thought experiment on the bank bailouts. And when you hear somebody claiming to know the “cause” of something, read the case study of the 1995 Mexico City Western Airlines Flight 2605 crash to help get a grasp of what selection bias can do to our analyses. There is an alternative way of thinking.

Note: if you doubt the Zara track record, see Watts’s discussion of Phillip Tetlock’s research on how experts get it wrong 80% of the time or see Tetlock, P. Expert Political Judgement: How Good is it? How can we know? (Princeton University Press, 2005). Zara does much better.