What Could Go Wrong?

Lessons from the long tail of improbable disaster

Indeed, it seems that when we conclude that the chance of something really bad happening is very small, we wind up taking actions that either increase the probability of the disaster or the damage that it will cause.
Once the rocket scientists on Wall Street, for example, concluded that it was virtually impossible for investors in so-called “mezzanine” tranches of mortgage-backed securities to lose money, it set off a chain of events that made the prediction untrue. The heavy demand for the securities led to dramatically lower lending standards and a sharp increase in housing prices, creating a bubble so large that when it burst, it caused heavy losses for those same mezzanine investors. The declaration that a particular investment was riskless became a self-negating prophecy.
Similarly, when the government builds a levee, it may reduce the frequency of damaging floods but may also encourage even more people to build homes and businesses behind the barrier. When the Big One finally arrives, the total damage will be even greater than if no levee had been built.