The Pundit Predicament

Ever since the invention of moveable type we've had to co-exist with pundits and punditry (sort of like bandits and banditry I suppose). But it was the invention of television that really brought the pundits into our homes, into our faces.

An article in Newsweek explores pundits and why they're so often simply dead wrong:

...Knowing who's likely to be right comes down to something psychologists call cognitive style, and with that in mind Philip Tetlock, a research psychologist at Stanford University, would like to introduce you to foxes and hedgehogs.

At first, Tetlock's ongoing study of 82,361 predictions by 284 pundits (most but not all of them American) came up empty. He initially looked at whether accuracy was related to having a Ph.D., being an economist or political scientist rather than a blowhard journalist, having policy experience or access to classified information, or being a realist or neocon, liberal or conservative. The answers were no on all counts. The best predictor, in a backward sort of way, was fame: the more feted by the media, the worse a pundit's accuracy. And therein lay Tetlock's first clue. The media's preferred pundits are forceful, confident and decisive, not tentative and balanced. They are, in short, hedgehogs, not foxes.

That bestiary comes from the political philosopher Isaiah Berlin, who in 1953 argued that hedgehogs "know one big thing." They apply that one thing (for instance, that ethnicity and language are primal; ergo, any country that contains many ethnic groups will break up) everywhere, express supreme confidence in their forecasts, dismiss opposing views and are drawn to top-down arguments deduced from that Big Idea. Foxes, in contrast, "know many things," as Berlin put it. They consider competing views, make bottom-up inductive arguments from an array of facts and doubt the power of Big Ideas. "The hedgehog-fox dimension did what none of the other traits did," says Tetlock, who described the study in his 2005 book "Expert Political Judgment": "distinguish more accurate forecasters from less accurate ones" in both politics (will Iraq break up?) and economics (whither unemployment?).entative and balanced. They are, in short, hedgehogs, not foxes.

In short, what experts think matters far less than how they think, or their cognitive style. At one extreme, hedgehogs seek certainty and closure, dismiss information that undercuts their preconceptions and embrace evidence that reinforces them, in what is called "belief defense and bolstering." At the other extreme, foxes are cognitively flexible, modest and open to self-criticism.

Here's how to identify fauna: foxes pepper their speech and writing with "however" and "but," recognizing uncertainty in the face of competing forces. Hedgehogs suffer from no such doubts, which (combined with their adherence to a Big Idea) makes them especially prone to overpredict change: the House of Saud will fall, the European Monetary Union will collapse, Canada will disintegrate like Yugoslavia—in the last case, from the primal force of ethnicity.

The media, of course, eat this up. Bold, decisive assertions make better sound bites; bombast, swagger and certainty make for better TV. As a result, the marketplace of ideas does not punish poor punditry. Few of us even remember who got what wrong.

http://www.newsweek.com/id/184815