Our very brains revolt at the idea of randomness. We have evolved as a species to become exquisite pattern-finders — long before the advent of science, we figured out that a salmon-colored sky heralds a dangerous storm, or that a baby's flushed face likely means a difficult night ahead. Our minds automatically try to place data in a framework that allows us to make sense of our observations and use them to understand events and predict them.
Randomness is so difficult to grasp because it works against our pattern-finding instincts. It tells us that sometimes there is no pattern to be found. As a result, randomness is fundamental limit to our intuition; it says that there are processes that we can't predict fully. It's a concept that we have a hard time accepting even though it is an essential part of the way the cosmos works. Without an understanding of randomness, we are stuck in a perfectly predictable universe that simply doesn't exist outside of our own heads.
I would argue that only once we understand three dicta — three laws of randomness — can we break out of our primitive insistence on predictability and appreciate the universe for what it is rather than what we want it to be.
The First Law of Randomness: There is such a thing as randomness.
We use all kinds of mechanisms to avoid confronting randomness. We talk about karma, in a cosmic equalization that ties seemingly unconnected events together. We believe in runs of luck, both good and ill, and that bad things happen in threes. We argue that we are influenced by the stars, by the phases of the moon, and by the motion of the planets in the heavens. When we get cancer, we automatically assume that something — or someone — is to blame.
But many events are not fully predictable or explicable. Disasters happen randomly, to good people as well as to bad ones, to star-crossed individuals as well as those who have a favorable planetary alignment. Sometimes you can make a good guess about the future, but randomness can confound even the most solid predictions — don't be surprised when you're outlived by the overweight, cigar-smoking, speed-fiend motorcyclist down the block.
What's more, random events can mimic non-random ones. Even the most sophisticated scientists can have difficulty telling the difference between a real effect and a random fluke. Randomness can make placebos seem like miracle cures, harmless compounds appear to be deadly poisons, and can even create subatomic particles out of nothing.
The Second Law of Randomness: Some events are impossible to predict.
If you walk into a Las Vegas casino and observe the crowd gathered around the craps table, you'll probably see someone who thinks he's on a lucky streak. Because he's won several rolls in a row, his brain tells him that he's going to keep winning, so he keeps gambling. You'll probably also see someone who's been losing. The loser's brain, like the winner's, tells him to keep gambling. Since he's been losing for so long, he thinks he's due for a stroke of luck; he won't walk away from the table for fear of missing out.
Contrary to what our brains are telling us, there's no mystical force that imbues a winner with a streak of luck, nor is there a cosmic sense of justice that ensures that a loser's luck will turn around. The universe doesn't care one whit whether you've been winning or losing; each roll of the dice is just like every other.
No matter how much effort you put into observing how the dice have been behaving or how meticulously you have been watching for people who seem to have luck on their side, you get absolutely no information about what the next roll of a fair die will be. The outcome of a die roll is entirely independent of its history. And, as a result, any scheme to gain some sort of advantage by observing the table will be doomed to fail. Events like these — independent, purely random events — defy any attempts to find a pattern because there is none to be found.
Randomness provides an absolute block against human ingenuity; it means that our logic, our science, our capacity for reason can only penetrate so far in predicting the behavior of cosmos. Whatever methods you try, whatever theory you create, whatever logic you use to predict the next roll of a fair die, there's always a 5/6 chance you are wrong. Always.
The Third Law of Randomness: Random events behave predictably in aggregate even if they're not predictable individually
Randomness is daunting; it sets limits where even the most sophisticated theories can not go, shielding elements of nature from even our most determined inquiries. Nevertheless, to say that something is random is not equivalent to saying that we can't understand it. Far from it.
Randomness follows its own set of rules — rules that make the behavior of a random process understandable and predictable.
These rules state that even though a single random event might be completely unpredictable, a collection of independent random events is extremely predictable — and the larger the number of events, the more predictable they become. The law of large numbers is a mathematical theorem that dictates that repeated, independent random events converge with pinpoint accuracy upon a predictable average behavior. Another powerful mathematical tool, the central limit theorem, tells you exactly how far off that average a given collection of events is likely to be. With these tools, no matter how chaotic, how strange a random behavior might be in the short run, we can turn that behavior into stable, accurate predictions in the long run.
The rules of randomness are so powerful that they have given physics some of its most sacrosanct and immutable laws. Though the atoms in a box full of gas are moving at random, their collective behavior is described by a simple set of deterministic equations. Even the laws of thermodynamics derive their power from the predictability of large numbers of random events; they are indisputable only because the rules of randomness are so absolute.
Paradoxically, the unpredictable behavior of random events has given us the predictions that we are most confident in.