2003 : WHAT ARE THE PRESSING SCIENTIFIC ISSUES FOR THE NATION AND THE WORLD, AND WHAT IS YOUR ADVICE ON HOW I CAN BEGIN TO DEAL WITH THEM? - GWB

Dear President Bush:

The country suffers from a crisis in scientific literacy. Indeed you yourself have often used the phrase "fuzzy math" as an insult even though your own home state of Texas funds fuzzy mathematics at research centers at Texas A&M and at the University of Texas at El Paso.

But there is a more focused and more urgent crisis of scientific literacy: There is widespread statistical illiteracy among scientists themselves. The signature of this illiteracy is not being able to tell a number from a curve.

Please allow me to explain. Almost all scientists and engineers work with and interpret statistical data. The very phrase "scientific study" tends to mean a study conducted in accord with standard principles of modern statistics. But few scientists or engineers can distinguish the key condition that gives rise to the beloved bell curve (remember those IQ and SAT tests?) from the condition that lets a pollster accept the averaged answers of a thousand or so subjects as a reliable estimate of the population at large (remember exit polls?). The first case gives a curve and the other case gives a number and it is crucial that at least the scientists who advise policy makers be able to distinguish the two. This goes to the heart of whether in a given case scientists should even apply the statistical framework and whether they should accept the results if they do apply it.

I know you are not a detail person. But this is one detail worth knowing: The whole distinction here turns on something as simple as the square root of the number of samples. That's right: everything turns on whether you use a number or its square root.

Here is how it turns out. The square-root case gives you something called the Central Limit Theorem or CLT for short. The CLT gives you the (thin tailed) bell curve that remains the most popular probability model in science and engineering—even though more accurate bell curves need thicker tails to account for the observed frequency of "rare" events such as stockmarket crashes or big flashes of lightning. Mathematicians even named this bell curve the Gaussian after the German mathematician Gauss although more and more scientists simply call it the "normal" bell curve because they find it so normal to apply to random phenomena. (Behind this is a deeper illiteracy that confuses data dispersion with an artificial and nonrobust contrivance called the "variance" but that is too much detail for this memo.) The other case works with the number of samples rather than the square root of that number. It gives you one of many so-called Laws of Large Numbers or LLNs for short. Those LLN theorems give you a single number or "poll result" that lesser politicians might use to measure public sentiment on a given yes-or-no question. This common confusion (that CLT = LLN) over a mere square root ranges from science and engineering to medicine and the war room.

See the problem? Social policy rests on empirical science or at least it should. And empirical science rests in turn on statistics and this is a subject far trickier than all too many scientists seem to think. So a little statistical incompetence can have dramatic social effects—think junk science in the courtroom.

What to do?

There isn't time to train or retrain our scientists and engineers and physicians (and lawyers) in probability and statistics. Nor would it be either cost effective or polite to require that at least once each grant applicant submit her answers from a proctored multiple-choice exam on basic statistics when she submits her grant proposal to a federal funding agency—even though state governments periodically do require just such test results to renew a driver's license.

Instead there is a simple rule of thumb you and your staff can use to quickly weed out the least competent: Fire or at least ignore any advisor or applicant who in good faith uses the phrase "law of averages." There is no such law.

Bart Kosko
Professor of Electrical Engineering
University of Southern California
Author of Fuzzy Thinking; Heaven in a Chip; and the novel Nanotime.