Are we better off with wrong forecasts, or no forecasts at all?

So I have been reading a book called Future Babble: Why Expert Predictions Fail and Why We Continue to Believe Them Anyway by Dan Gardner.

With such a title and cover, you can imagine what he says about experts and forecasting. The book was reviewed by Kathryn Schultz in the New York Times, and she did so brilliantly, so I will send you over there to read, save for this, which captures the book’s campy and self-indulgent contradictions nicely, where Gardner equates forecasting with paid psychic services:

To ignore this difference is to stray perilously close to anti-intellectualism. And Gardner, despite his better impulses, drifts that direction in other ways as well — for instance, by pitting “all the smart people” against “ordinary Americans.” Wait: Ordinary Americans aren’t smart? Smart people aren’t real Americans? Such distinctions aren’t just invidious. They also dodge the real issue, which is that expertise and intelligence are not intellectually or morally equivalent to charlatanism. Indeed, they often serve us exceptionally well.

Gardner serves up a nice helping of nose-rubbing for James Howard Kuntsler, one of the most shamelessly self-promotional profiteers in the urbanist universe, so, as Schultz points out, that bit is quite enjoyable for those of us who are petty.

I’d also second Kathryn Shultz’s recommendation to read Philip Tetlock’s Political Judgment: How Good Is It? How Can We Know? , which is an excellent book. Schultz’s own contribution, Being Wrong: Adventures in the Margin of Error, is also a nice contribution, and much more thought-provoking than Gardner’s rant.

But the greater point about how often forecasts are wrong is still worth thinking about. Are we better trying to know the future, and if we aren’t, well, how should we make decisions? Who has a better idea?

It’s not clear there are better ideas on how to think ahead. There’s part of me that thinks as much as forecasts are wrong, and as much as forecasts can serve the interests of power, that all the anti-forecasting rhetoric also reinforces “who screams louder, who has a bigger stick” politics of urban project development.

One indicator that there aren’t really any better ideas out there came to me at the HSR Symposium I was at a few weeks ago. Peter Calthorpe, who always markets himself has having the best, new, cutting edge idea, was describing his fabulous, much-improved-over-those-dumb-engineers’ travel demand forecast with a new whizbang *urban travel simulator tool*. Yes, this will be yet another deterministic model with inappropriately optimistic assumptions about how much Calthorpe’s pet design ideas reduce auto travel, but it’s not a forecast. It’s a *simulation.* Based on *empirical elasticities*. Soo much better than a forecast, you betcha. It will simulate outcomes, too. Like how many fat people we’ll take off the streets when we run the world the right way and design controls mankind the way it should.

So Calthorpe creates a forecasting tool and calls it a simulation, and suddenly he’s got the magic predictor bullet. But the temptation is so apparent: I’ll prove with my truthy numbers that I have the magic urban recipe. It’s worked for engineers for years. Why not the New Urbanists? Forecasts are a means of establishing “need”, right? (See: The Rhetoric of Economics by Dierdre McCloskey, a 100 percent required read.)

The question for forecasts/simulations is always what are your priors? Are any priors any good at all? Can the generic priors that Calthorpe and his group stuff into their new simulator tool really help us?

People I genuinely respect tell me, for example, that the CalHSR forecasts are way optimistic. While I’m inclined to believe them, it’s a criticism I haven’t echoed here simply because I really have no idea because I don’t think there are good priors. HSR systems in other places don’t strike me as good priors. Neither does intercity demand on other modes.

The world is meant to end tomorrow, May 21, according to one prediction. So we’ll see how that goes.

2 thoughts on “Are we better off with wrong forecasts, or no forecasts at all?

  1. Please read the passage in which I allegedly pitted “smart people” against “ordinary Americans.” Schultz grotesquely misrepresented it. As she did so much of my book.

    Incidentally, the same Philip Tetlock you and she recommend so highly praised my book as “superb scholarship” — a judgment seconded by Steven Pinker. Of course you are perfectly entitled to disagree but I think your accusation of anti-intellectualism is absolutely absurd.

    • Well, Schultz doesn’t actually accuse you of anti-intellectualism; she’s says you’re adjacent. I also don’t see why you are being so defensive. In general, her review of your book was in general positive with points where she thought your logic was troubling. Nobody expects a book to be utterly flawless, and I did spend 20 bucks on your book and I am taking the time to read it–though, I have to say, I’m not finding out much that I didn’t already know (but I’m from a field where the problems with forecasting have been the subject of long commentary.)

Comments are closed.