| But
there is a more focused and more urgent crisis of scientific literacy:
There is widespread statistical illiteracy among scientists themselves.
The signature of this illiteracy is not being able to tell a number
from a curve.
Bart
Kosko
Dear President
Bush:
The country
suffers from a crisis in scientific literacy. Indeed you yourself have
often used the phrase "fuzzy math" as an insult even though your own
home state of Texas funds fuzzy mathematics at research centers at Texas
A&M and at the University of Texas at El Paso.
But there
is a more focused and more urgent crisis of scientific literacy: There
is widespread statistical illiteracy among scientists themselves. The
signature of this illiteracy is not being able to tell a number from
a curve.
Please
allow me to explain. Almost all scientists and engineers work with and
interpret statistical data. The very phrase "scientific study" tends
to mean a study conducted in accord with standard principles of modern
statistics. But few scientists or engineers can distinguish the key
condition that gives rise to the beloved bell curve (remember those
IQ and SAT tests?) from the condition that lets a pollster accept the
averaged answers of a thousand or so subjects as a reliable estimate
of the population at large (remember exit polls?). The first case gives
a curve and the other case gives a number and it is crucial that at
least the scientists who advise policy makers be able to distinguish
the two. This goes to the heart of whether in a given case scientists
should even apply the statistical framework and whether they should
accept the results if they do apply it.
I know
you are not a detail person. But this is one detail worth knowing: The
whole distinction here turns on something as simple as the square root
of the number of samples. That's right: everything turns on whether
you use a number or its square root.
Here is
how it turns out. The square-root case gives you something called the
Central Limit Theorem or CLT for short. The CLT gives you the (thin
tailed) bell curve that remains the most popular probability model in
science and engineering—even though more accurate bell curves
need thicker tails to account for the observed frequency of "rare" events
such as stockmarket crashes or big flashes of lightning. Mathematicians
even named this bell curve the Gaussian after the German mathematician
Gauss although more and more scientists simply call it the "normal"
bell curve because they find it so normal to apply to random phenomena.
(Behind this is a deeper illiteracy that confuses data dispersion with
an artificial and nonrobust contrivance called the "variance" but that
is too much detail for this memo.) The other case works with the number
of samples rather than the square root of that number. It gives you
one of many so-called Laws of Large Numbers or LLNs for short. Those
LLN theorems give you a single number or "poll result" that lesser politicians
might use to measure public sentiment on a given yes-or-no question.
This common confusion (that CLT = LLN) over a mere square root ranges
from science and engineering to medicine and the war room.
See the
problem? Social policy rests on empirical science or at least it should.
And empirical science rests in turn on statistics and this is a subject
far trickier than all too many scientists seem to think. So a little
statistical incompetence can have dramatic social effects—think
junk science in the courtroom.
What to
do?
There
isn't time to train or retrain our scientists and engineers and physicians
(and lawyers) in probability and statistics. Nor would it be either
cost effective or polite to require that at least once each grant applicant
submit her answers from a proctored multiple-choice exam on basic statistics
when she submits her grant proposal to a federal funding agency—even
though state governments periodically do require just such test results
to renew a driver's license.
Instead
there is a simple rule of thumb you and your staff can use to quickly
weed out the least competent: Fire or at least ignore any advisor or
applicant who in good faith uses the phrase "law of averages." There
is no such law.
Bart Kosko
Professor of Electrical Engineering
University of Southern California
Author of Fuzzy Thinking; Heaven in a Chip; and the novel
Nanotime.
|
