2012 : WHAT IS YOUR FAVORITE DEEP, ELEGANT, OR BEAUTIFUL EXPLANATION?

Redundancy Reduction and Pattern Recognition

Deep, elegant, beautiful? Part of what makes a theory elegant is its power to explain much while assuming little. Here, Darwin's natural selection wins hands down. The ratio of the huge amount that it explains (everything about life: its complexity, diversity and illusion of crafted design) divided by the little that it needs to postulate (non-random survival of randomly varying genes through geological time) is gigantic. Never in the field of human comprehension were so many facts explained by assuming so few. Elegant then, and deep—its depths hidden from everybody until as late as the nineteenth century. On the other hand, for some tastes natural selection is too destructive, too wasteful, too cruel to count as beautiful. In any case, coming late to the party as ever, I can count on somebody else choosing Darwin. I'll take his great grandson instead, and come back to Darwin at the end.

Horace Barlow FRS is the youngest grandchild of Sir Horace Darwin, Charles Darwin's youngest child. Now a very active ninety, Barlow is a member of a distinguished lineage of Cambridge neurobiologists. I want to talk about an idea that he published in two papers in 1961, on redundancy reduction and pattern recognition. It is an idea whose ramifications and significance have inspired me throughout my career.

The folklore of neurobiology includes a mythical 'grandmother neurone', which fires only when a very particular image, the face of Jerry Lettvin's grandmother, falls on the retina (Lettvin was a distinguished American neurobiologist who, like Barlow, worked on the frog retina). The point is that Lettvin's grandmother is only one of countless images that a brain is capable of recognising. If there were a specific neurone for everything we can recognise—not just Lettvin's grandmother but lots of other faces, objects, letters of the alphabet, flowers, each one seen from many angles and distances, we would have a combinatorial explosion. If sensory recognition worked on the 'grandmother principle', the number of specific recognition neurones for all possible combinations of nerve impulses would exceed the number of atoms in the universe. Independently, the American psychologist Fred Attneave had calculated that the volume of the brain would have to be measured in cubic light years. Barlow and Attneave independently proposed redundancy reduction as the answer.

Claude Shannon, inventor of Information Theory, coined 'redundancy' as a kind of inverse of information. In English, 'q' is always followed by 'u', so the 'u' can be omitted without loss of information. It is redundant. Wherever redundancy occurs in a message (which is wherever there is nonrandomness), the message can be more economically recoded without loss of information (although with some loss in capacity to correct errors). Barlow suggested that, at every stage in sensory pathways, there are mechanisms tuned to eliminate massive redundancy.

The world at time t is not greatly different from the world at time t-1. Therefore it is not necessary for sensory systems continuously to report the state of the world. They need only signal changes, leaving the brain to assume that everything not reported remains the same. Sensory adaptation is a well-known feature of sensory systems, which does precisely as Barlow prescribed. If a neurone is signalling temperature, for example, the rate of firing is not, as one might naively suppose, proportional to the temperature. Instead, firing rate increases only when there is a change in temperature. It then dies away to a low resting frequency. The same is true of neurones signalling brightness, loudness, pressure and so on. Sensory adaptation achieves huge economies by exploiting the non-randomness in temporal sequence of states of the world.

What sensory adaptation achieves in the temporal domain, the well-established phenomenon of lateral inhibition does in the spatial domain. If a scene in the world falls on a pixellated screen, such as the back of a digital camera or the retina of an eye, most pixels see the same as their immediate neighbours. The exceptions are those pixels which lie on edges, boundaries. If every retinal cell faithfully reported its light value to the brain, the brain would be bombarded with a massively redundant message. Huge economies can be achieved if most of the impulses reaching the brain come from pixel cells lying along edges in the scene. The brain then assumes uniformity in the spaces between edges.

As Barlow pointed out, this is exactly what lateral inhibition achieves. In the frog retina, for example, every ganglion cell sends signals to the brain, reporting on the light intensity in its particular location on the surface of the retina. But it simultaneously sends inhibitory signals to its immediate neighbours. This means that the only ganglion cells to send strong signals to the brain are those that lie on an edge. Ganglion cells lying in uniform fields of colour (the majority) send few if any impulses to the brain because they, unlike cells on edges, are inhibited by all their neighbours. The spatial redundancy in the signal is eliminated.

The Barlow analysis can be extended to most of what is now known about sensory neurobiology, including Hubel and Wiesel's famous horizontal and vertical line detector neurones in cats (straight lines are redundant, reconstructable from their ends), and in the movement ('bug') detectors in the frog retina, discovered by the same Jerry Lettvin and his colleagues. Movement represents a non-redundant change in the frog's world. But even movement is redundant if it persists in the same direction at the same speed. Sure enough, Lettvin and colleagues discovered a 'strangeness' neurone in their frogs, which fires only when a moving object does something unexpected, such as speeding up, slowing down, or changing direction. The strangeness neurone is tuned to filter out redundancy of a very high order.

Barlow pointed out that a survey of the sensory filters of a given animal could, in theory, give us a read-out of the redundancies present in the animal's world. They would constitute a kind of description of the statistical properties of that world. Which reminds me, I said I'd return to Darwin. In Unweaving the Rainbow, I suggested that the gene pool of a species is a 'Genetic Book of the Dead', a coded description of the ancestral worlds in which the genes of the species have survived through geological time. Natural selection is an averaging computer, detecting redundancies—repeat patterns—in successive worlds (successive through millions of generations) in which the species has survived (averaged over all members of the sexually reproducing species). Could we take what Barlow did for neurones in sensory systems, and do a parallel analysis for genes in naturally selected gene pools? Now that would be deep, elegant and beautiful.