Worst & Worser

Complexity theory has mixed implications for worst case analysis. On the one hand, a power law precludes the possibility of a genuinely worst case: there are always even worse (though even less likely) cases to worry about. So critics of worst case analysis were right to worry about the arbitrariness of selecting a single worst case.

On the other hand, one characteristic of power laws is that the unlikely events on the right tail of the curve have a strong cumulative effect. If we focus only on what seem to be reasonably likely outcomes, we overlook the statistical possibilities for nasty surprises.

Worst case analysis can be a useful reminder that through a string of unlikely coincidences, things may go very wrong indeed. Despite the tiny likelihood of any one such freakish outcome, the total spectrum of extreme outcomes may deserve serious weight in the decision. Consideration of the worst cases -- or more accurately, of a spectrum of possible worse cases -- can compensate for the tendency to focus too heavily on the likely outcomes of an action and dismiss "speculation"about possible disasters.

At one time, CEQ’s NEPA regulations mandated a formal “worst case” analysis. The general consensus is that this requirement was burdensome for agendas and potentially misleading for the public. We do, however, need to be careful to plan for reasonably plausible though unlikely outcomes.

We also need to keep in mind the existence of critical uncertainties. A good motto for long range planning is: "We know less than we think we do. Some of what we know is wrong."

For example, in planning the New Orleans flood control system, the Corps seems to have assumed that weather patterns would be static. As we now know, hurricane frequency is probably cyclic and there is probably a long-term upward trend in intensity due to climate change. Much of this could not have been foreseen when the planning decisions were made. But what was foreseeable is that our knowledge of weather patterns might be unreliable. The planning process needed to include a margin of safety to account for this uncertainty, as well as building in opportunities for reevaluation as scientific knowledge developed.

When I went looking for an image of a crystal ball to use with this post, the best one turned out to be from this site on invasive species. That's not a complete coincidence, because of the difficulty of predicting how a new species will interact with an ecosystem. However, it is a happy accident that this happens to coincide with the subject of JB's latest posting.

Most likely, any given invasion will be unsuccessful, resulting in the elimination of the invader. But every now and then . . . .