Tgk1946's Blog

From The Signal and the Noise (Nate Silver, 2012) pp406-11

Yet Another Reason Why Estimating Uncertainty Is Essential

Uncertainty is an essential and nonnegotiable part of a forecast. As we have found, sometimes an honest and accurate expression of the uncertainty is what has the potential to save property and lives. In other cases, as when trading stock options or wagering on an NBA team, you may be able to place bets on your ability to forecast the uncertainty accurately.

However, there is another reason to quantify the uncertainty carefully and explicitly. It is essential to scientific progress, especially under Bayes’s theorem.

Suppose that in 2001, you had started out with a strong prior belief in the hypothesis that industrial carbon emissions would continue to cause a temperature rise. (In my view, such a belief would have been appropriate because of our strong causal understanding of the greenhouse effect and the empirical evidence for it up to that point.) Say you had attributed the chance of the global warming hypothesis’s being true at 95 percent.

But then you observe some new evidence: over the next decade, from 2001 through 2011, global temperatures do not rise. In fact, they fall, although very slightly. Under Bayes’s theorem, you should revise your estimate of the probability of the global warming hypothesis downward; the question is by how much.

If you had come to a proper estimate of the uncertainty in near-term temperature patterns, the downward revision would not be terribly steep. As we found, there is about a 15 percent chance that there will be no net warming over a decade even if the global warming hypothesis is true because of the variability in the climate. Conversely, if temperature changes are purely random and unpredictable, the chance of a cooling decade would be 50 percent since an increase and a decrease in temperatures are equally likely. Under Bayes’s theorem (figure 12-12), a no-net-warming decade would cause you to revise downward your estimate of the global warming hypothesis’s likelihood to 85 percent from 95 percent.

FIGURE 12-12: BAYES’S THEOREM—GLOBAL WARMING EXAMPLE

On the other hand, if you had asserted that there was just a 1 percent chance that temperatures would fail to increase over the decade, your theory is now in much worse shape because you are claiming that this was a more definitive test. Under Bayes’s theorem, the probability you would attach to the global warming hypothesis has now dropped to just 28 percent.

When we advance more confident claims and they fail to come to fruition, this constitutes much more powerful evidence against our hypothesis. We can’t really blame anyone for losing faith in our forecasts when this occurs; they are making the correct inference under Bayesian logic.

So what is the incentive to make more confident claims to begin with, especially when they are not really justified by the statistical evidence? There are all sorts of reasons that people may do this in practice. In the climate debate, it may be because these more confident claims can seem more persuasive — and they may be, but only if they are right. Attributing every weather anomaly to manmade climate change — other than the higher temperatures the global warming phenomenon is named for — is a high-stakes gamble, rooted more in politics than in science. There is little consensus about the ways that climate change might manifest itself other than through temperature increases and probably rising sea levels. Obviously, charging that every snowfall is evidence against the theory is just as ridiculous.

“We’re in a Street Fight with These People”

The fundamental dilemma faced by climatologists is that global warming is a long-term problem that might require a near-term solution. Because carbon dioxide remains in the atmosphere for so long, decisions that we make about it today will affect the lives of future generations.

In a perfectly rational and benevolent world, this might not be so worrying. But our political and cultural institutions are not so well-devised to handle these problems — not when the United States Congress faces reelection every two years and when businesses are under pressure to meet earnings forecasts every quarter. Climate scientists have reacted to this challenge in a variety of ways, some involving themselves more in the political debate and others keeping it at arm’s length.

Michael Mann, who is director of the Earth System Science Center at Penn State University, was once at the center of a controversy. “Climategate” concerned the hacking of a server at the Climatic Research Unit (CRU) at the University of Fast Anglia,’? which produces the temperature record that the UK’s Met Office uses. Skeptics alleged that Mann and other scientists had conspired to manipulate the CRU’s temperature record.

The pertinent facts are that the scientists were cleared of wrongdoing by a panel of their peers,’°* and that the CRU’s temperature record is quite consistent with the others! — but Mann and other scientists in the hacked e-mails demonstrated a clear concern with the public relations elements of how the science would be perceived. Mann is happy to tell you as much. I met with him on a crisp fall afternoon at his office at Penn State, where we chatted for about two hours.

Mann is exceptionally thoughtful about the science behind global warming. Like most other climatologists, he has little doubt about the theoretical mechanisms behind climate change, but he takes a skeptical view toward the predictions rendered by climate models. “Any honest assessment of the science is going to recognize that there are things we understand pretty darn well and things that we sort of know,” he told me. “But there are things that are uncertain and there are things we just have no idea about whatsoever.” “In my mind, one of the unfortunate consequences of this bad-faith public conversation we’ve been having is that we’re wasting our time debating a proposition that is very much accepted within the scientific community, when we could be having a good-faith discussion about the uncertainties that do exist”. But Mann, who blogs along with Schmidt at RealClimate.org, sees himself as engaged in trench warfare against groups like the Heartland Institute. “We’re in a street fight with these people,” he told me, referring to a Nature editorial that employed the phrase. The long-term goal of the street fight is to persuade the public and policy makers about the urgency (or lack thereof) of action to combat climate change. In a society accustomed to overconfident forecasters who mistake the confidence they express in a forecast for its veracity, expressions of uncertainty are not seen as a winning strategy by either side. “Where you have to draw the line is to be very clear about where the uncertainties are, but to not have our statements be so laden in uncertainty that no-one even listens to what we’re saying,” Mann told me. “It would be irresponsible for us as a community to not be speaking out. There are others who are happy to fill the void. And they’re going to fill the void with disinformation.” The Difference Between Science and Politics In practice, Mann’s street fight is between “consensus” Web sites like RealClimate.org and “skeptical” ones like Watts Up With That,” and revolves around day-to-day scuffles about the latest journal article or weather pattern or political controversy. Both sides almost invariably stick up for others in their circle and refuse to yield ground. When you’re a Jet, youre a Jet all the way. I do not mean to suggest that the territory occupied by the two sides symmetrical. In the scientific argument over global warming, the truth seems to be mostly on one side: the greenhouse effect almost certainly exists and will be exacerbated by manmade CO2 emissions. This is very likely to make the planet warmer. ‘The impacts of this are uncertain, but are weighted toward unfavorable outcomes.

The street-fighter mentality, nevertheless, seems to be predicated on the notion that we are just on the verge of resolving our political problems, if only a few more people could be persuaded about the science. In fact, we are probably many years away. “There’s a point when I come to the conclusion that we’re going to have to figure out how to take the carbon out,” Richard Rood told me in Copenhagen, anticipating that there was almost no way the 193 members of the United Nations would agree to mutually acceptable terms.

Meanwhile, the American public’s confidence that global warming is occurring has decreased somewhat over the past several years. And even if there were 100 percent agreement on the effects of climate change, some states and some countries would make out better than others in any plan to mitigate carbon emissions. “We have some very progressive Democratic governors in coal states,” I was told by the governor of Washington, Christine Gregoire. “Boy, are they nervous about all this.”

I don’t know how to resolve these problems, which are not unique to the climate debate. What I do know is that there is a fundamental difference be- tween science and politics. In fact, I’ve come to view them more and more as opposites.

In science, progress is possible. In fact, if one believes in Bayes’s theorem, scientific progress is inevitable as predictions are made and as beliefs are tested and refined. The march toward scientific progress is not always straight- forward, and some well-regarded (even “consensus”) theories are later proved wrong — but either way science tends to move toward the truth.

In politics, by contrast, we seem to be growing ever further away from consensus. The amount of polarization between the two parties in the United States House, which had narrowed from the New Deal through the 1970s, had grown by 2011 to be the worst that it had been in at least a century. Republicans have moved especially far away from the center, although Democrats have to some extent too.

In science, one rarely sees all the data point toward one precise conclusion. Real data is noisy — even if the theory is perfect, the strength of the signal will vary. And under Bayes’s theorem, no theory is perfect. Rather, it is a work in progress, always subject to further refinement and testing. This is what scientific skepticism is all about.

In politics, one is expected to give no quarter to his opponents. It is seen as a gaffe when one says something inconvenient — and true Partisans are expected to show equal conviction about a set of beliefs on a range of economic, social, and foreign policy issues that have little intrinsic relation to one another. As far as approximations of the world go, the platforms of the Democratic and Republican parties are about as crude as it gets.

It is precisely because the debate may continue for decades that climate scientists might do better to withdraw from the street fight and avoid crossing the Rubicon from science into politics. In science, dubious forecasts are more likely to be exposed — and the truth is more likely to prevail. In politics, a do- main in which the truth enjoys no privileged status, it’s anybody’s guess.

The dysfunctional state of the American political system is the best reason to be pessimistic about our country’s future. Our scientific and technological prowess is the best reason to be optimistic. We are an inventive people. The United States produces ridiculous numbers of patents, has many of the world’s best universities and research institutions, and our companies lead the market in fields ranging from pharmaceuticals to information technology. If I had a choice between a tournament of ideas and a political cage match, I know which fight I’d rather be engaging in—especially if I thought I had the right forecast.