The notion of robustness became my recent obsession. How to be a robust? It's not clear; it's the hard problem — and that's my problem.

THE HARD PROBLEM [5.17.10]
By Nassim Nicholas Taleb

Introduction

On April 10th, at Harvard, I attended the "Hardest Problems" Symposium at Harvard, inspired by "Hilbert's Problems" — 23 problems presented by the German mathematician David Hilbert in 1900 which were unsolved at the time. The day-long event was organized and hosted by Stephen Kosslyn, a psychologist who is Dean of Social Science in the Faculty of Arts and Sciences at Harvard University — and a frequent Edge contributor (see "What Shape Are a German Shepard's Ears?")

Speakers included: Nick Bostrom, Susan Carey, Nicholas Christakis, James Fowler , Roland Fryer, Claudia Goldin. Gary King, Emily Oster, Ann Swidler, Nassim Taleb, and Richard Zeckhauser.

Nassim Taleb's "specialty and fear and obsession is with small probabilities. We don't quite understand small probabilities". Taleb, author of The Black Swan, notes that "the fundamental problem of small probabilities is that rare events don't show in samples, because they are rare. So when someone makes a statement that this in the financial markets should happen every ten thousand years, visibly they are not making a statement based on empirical evidence, or computation of the odds, but based on what? On some model, some theory.

"...our random variables became more and more complex. We cannot escape it. We can become more robust. There are techniques to become more robust. The notion of robustness became my recent obsession. How to be a robust? It's not clear; it's the hard problem — and that's my problem."

— John Brockman

NASSIM NICHOLAS TALEB, essayist and former mathematical trader, is Distinguished Professor of Risk Engineering at New York University’s Polytechnic Institute. He is the author of Fooled by Randomness and the international bestseller The Black Swan.

Nassim Taleb's Edge Bio Page

Further Reading on Edge: "Learning to Expect the Unexpected"; "The Fourth Quadrant: A Map of the Limits of Statistics".

THE HARD PROBLEM

Slide file

[NASSIM TALEB:] My specialty and fear and obsession is with small probabilities. We don't quite understand small probabilities. Let's discuss. [slide]

You often see in the papers things saying events we just saw should happen every ten thousand years, hundred thousand years, ten billion years. Some faculty here in this university had an event and said that a 10-sigma event should happen every, I don't know how many billion years. Do you ever regard how worrisome it is, when someone makes a statement like that, "it should happen every ten thousand years," particularly when the person is not even two thousand years old?

So the fundamental problem of small probabilities is that rare events don't show in samples, because they are rare. So when someone makes a statement that this in the financial markets should happen every ten thousand years, visibly they are not making a statement based on empirical evidence, or computation of the odds, but based on what? On some model, some theory.

So, the lower the probability, the more theory you need to be able to compute it. Typically it's something called extrapolation, based on regular events and you extend something to what you call the tails.

So my telescope problem is as follows. [slide]

Consider the following. The smaller the probability, the less you observe it in a sample, therefore your error rate in computing it is going to be very high. Actually, your relative error rate can be infinite, because you're computing a very, very small probability, so your error rate is monstrous and probably very, very small. The problem we have now is that we don't care about probabilities. Probabilities are something you do in the classroom. What you care about is what you are going to use the probability for, what I have here called lambda of the impact of the event, the effect of the event, the consequence of the event.

So you're interested in the pair, probability times impact. So notice something, that the vicious aspect is that the pair, probability times effect by lambda, as the probability becomes smaller, the impact is bigger. A thousand year flood is going to be definitely vastly more consequential than the five year flood, or a hundred year flood.

So you care about that, and that pi times lambda becomes a lot more stochastic when pi gets small. In other words, very consequential events, we know absolutely nothing about and that matter a lot, particularly in some domains in which the probabilities don't fall very quickly.

And you see here the variation of a derivatives portfolio over twenty years. One day represents 97 percent of the variations. And that day is not forecastable, OK, and that's the problem of small probabilities. There are domains in which small probabilities aren't everything – very, very small probabilities, and we know very little about them.

What are these domains? [slide] There's a regular classification. I won't get into the details of the way these things are made, but there is type-1, I call something that reaches central limit in real time, and type-2 that doesn't reach it in real time. The classical distinction is between the thing they call Gaussian or Gaussian family or power law families — it doesn't work very well. It's much more effective to have what we call in real time, a real time central limit. In other words, the rare event in type-2 will dominate the properties regardless of what theory you're using.

Now what worries me. I'm here at the epicenter of the education, of Economics at Harvard, and there is something at the foundation of Economics that is completely bogus. Completely. And let me tell you what it is. Its not the problem that people are rational or irrational or people keep bickering about more problems. It's the fact that, at the center of the techniques used, in econometrics and statistics used, at the center there is this assumption that doesn't seem to hold theoretically — I mean philosophically a priori, defaulting to the type-1 is completely arbitrary and unjustified, the second one empirically. And people tell me they have techniques to adjust for fat tails. (Fat tails mean the contribution of the rare events). Not only can we not predict rare events, we cannot even figure out what role they play in the data.

This graph shows you the following. [slide] I took every economic variable I could find data for that covers 40 plus years, from stock, exchange rates, stock markets — everything that had a lot of data — everything I could find — 20 some million pieces of data — and for most of these variables, one day in 40 years represent 90 percent of a measure called Kurtosis . Kurtosis tells you how much something is not Gaussian (conditional on the distribution being type-1). In other words, you don't even know how non-Gaussian we are. That is a severe problem.

For example, take the stock market crash in 1987:  one day in 40 years represents 80-some percent of the Kurtosis, of that measure.

Measurability. [slide] Number one. That what we call the Norm L2, nobody should use when you have open ended payoff in economics, nobody should use at all something called variance, Least-Square methods, standard deviations, Sharpe ratio , portfolio theory, even the word correlation. These don't work in the domains of type-2. They don't work. That explains why the system collapsed because banks were using…. I spent 15 years fighting risk management methods in banks, they were all based on standard deviation portfolio theory because they say Nobels were proposing these things.

Second point. Social science has made some distinctions, which I don't think are very rigorous, between what they call Knightian measurable risk and what they call not measurable uncertainty. Several problems. [slide]

Number one, when you talk about uncertainty, you don't say measuring risk, you don't measure a risk, like I'm measuring this table. I'm estimating something that happened would happen in the future on one hand, and on the other one, I am measuring a certain physical entity. I'm measuring with an error of type-1 and type-2. The other problem with the Knightian stuff is that all risks in the tails, all small probabilities, are not measurable, regardless of whether you know what's going on or you don't know what's going on.

This brings me to the consequences. [slide] Visibly forecasting. I don't know if you guys are aware, but you can check all forecasts in economics and they don't work. Why don't they work? Because of the rare events. Unless of course you don't have exposure to rare events, which, with these random variables, is not the case.

I have to specify some sub problems and then go to my main problems [slide].

The hard problem is the classical problem of induction, inverse problems, you don't observe the generator; you don't know what processes you're dealing with. [slide] There are a series of smaller problems that we can probably solve for, something called the Ludic Fallacy, the fact that probabilities that are not as observable in real life as they are in games. There are a lot of problems associated with my general thing, but my main point is that small probabilities are not measurable.

There are two kinds of decisions you make in life, decisions that depend on small probabilities, and decisions that do not depend on small probabilities. For example, if I'm doing an experiment that is true-false, I don't really care about that pi-lambda effect, in other words, if its very false or false, it doesn't make a difference. So for example if I'm doing tests on individuals in medicine, I'm not really affected by tale events, but if I'm studying epidemics, then the random variable how many people are effected becomes open-ended with consequences so therefore it depends on fat tales. So I have kinds of decisions. One simple, true-false, and one more complicated, like the ones we have in economics, financial decision-making, a lot of things, I call them M1, M1+.