MARTI HEARST
Computer Scientist, UC Berkeley, School of Information

Computational Analysis of Language Requires Understanding Language

To me, having my worldview entirely altered is among the most fun parts of science. One mind-altering event occurred during graduate school. I was studying the field of Artificial Intelligence with a focus on Natural Language Processing. At that time there were intense arguments amongst computer scientists, psychologists, and philosophers about how to represent concepts and knowledge in computers, and if those representations reflected in any realistic way how people represented knowledge. Most researchers thought that language and concepts should be represented in a diffuse manner, distributed across myriad brain cells in a complex network. But some researchers talked about the existence of a "grandmother cell," meaning that one neuron in the brain (or perhaps a concentrated group of neurons) was entirely responsible for representing the concept of, say, your grandmother. I thought this latter view was hogwash.

But one day in the early 90's I heard a story on National Public Radio about children who had Wernicke's aphasia, meaning that a particular region in their brains were damaged. This damage left the children with the ability to form complicated sentences with correct grammatical structure and natural sounding rhythms, but with content that was entirely meaningless. This story was a revelation to me -- it seemed like irrefutable proof that different aspects of language were located in distinct regions of the brain, and that therefore perhaps the grandmother cell could exist. (Steven Pinker subsequently wrote his masterpiece, "The Language Instinct," on this topic.)

Shortly after this, the field of Natural Language Processing became radically changed by an entirely new approach. As I mentioned above, in the early 90's most researchers were introspecting about language use and were trying to hand-code knowledge into computers. So people would enter in data like "when you go to a restaurant, someone shows you to a table. You and your dining partners sit on chairs at your selected table. A waiter or waitress walks up to you and hands you a menu. You read the menu and eventually the waiter comes back and asks for your order. The waiter takes this information back to the kitchen." And so on, in painstaking detail.

But as large volumes of text started to become available online, people started developing algorithms to solve seemingly difficult natural language processing problems using very simple techniques. For example, how hard is it to write a program that can tell which language a stretch of text is written in? Sibun and Reynar found that all you need to do is record how often pairs of characters tend to co-occur in each language, and you only need to extract about a sentence from a piece of text to classify it with 99% accuracy into one of 18 languages! Another wild example is that of author identification. Back in the early 60's, Mosteller and Wallace showed that they could identify which of the disputed Federalist Papers were written by Hamilton vs. those written by Madison, simply by looking at counts of the function words (small structural words like "by", "from", and "to") that each author used.

The field as a whole is chipping away at the hard problems of natural language processing by using statistics derived from that mother-of-all-text-corpora, the Web. For example, how do you write a program to figure out the difference between a "student protest" and a "war protest"? The former is a demonstration against something, done by students, but the latter is not a demonstration done by a war.

In the old days, we would try to code all the information we could about the words in the noun compounds and try to anticipate how they interact. But today we used statistics drawn from counts of simple patterns on the web. Recently my PhD student Preslav Nakov has shown that we can often determine what the intended relationship between two nouns is by simply counting the verbs that fall between the two nouns, if we first reverse their order. So if we search the web for patterns like:

"protests that are * by students"

we find out the important verbs are "draw, involve, galvanize, affect, carried out by" and so on, whereas for "war protests" we find verbs such as "spread by, catalyzed by, precede", and so on.

The lesson we see over and over again is that simple statistics computed over very large text collections can do better at difficult language processing tasks than more complex, elaborate algorithms.

DIANE F. HALPERN
Professor, Claremont McKenna College; Past-president, American Psychological Association; Author, Sex Differences in Cognitive Abilities

From A Simple Truth To "It All Depends"

Why are men underrepresented in teaching, child care, and related fields and women underrepresented in engineering, physics, and related fields? I used to know the answer, but that was before I spent several decades reviewing almost everything written about this question. Like most enduring questions, the responses have grown more contentious and even less is "settled" now that we have mountains of research designed to answer them. At some point, my own answer changed from what I believed to be the simple truth to a convoluted statement complete with qualifiers, hedge terms, and caveats. I guess this shift in my own thinking represents progress, but it doesn't feel or look that way.

I am a feminist, a product of the 60s, who believed that group differences in intelligence or most any other trait are mostly traceable to the lifetime of experiences that mold us into the people we are and will be. Of course, I never doubted the basic premises of evolution, but the lessons that I learned from evolution favor the idea that the brain and behavior are adaptable. Hunter-gatherers never solved calculus problems or traveled to the moon, so I find little in our ancient past to explain these modern-day achievements.

There is also the disturbing fact that evolutionary theories can easily explain almost any outcome, so I never found them to be a useful framework for understanding behavior. Even when I knew the simple truth about sex differences in cognitive abilities, I never doubted that heritability plays a role in cognitive development, but like many others, I believed that once the potential to develop an ability exceeded some threshold value, heritability was of little importance. Now I am less sure about any single answer, and nothing is simple any more.

The literature on sex differences in cognitive abilities is filled with inconsistent findings, contradictory theories, and emotional claims that are unsupported by the research. Yet, despite all of the noise in the data, clear and consistent messages can be heard. There are real, and in some cases sizable, sex differences with respect to some cognitive abilities.

Socialization practices are undoubtedly important, but there is also good evidence that biological sex differences play a role in establishing and maintaining cognitive sex differences, a conclusion that I wasn't prepared to make when I began reviewing the relevant literature. I could not ignore or explain away repeated findings about (small) variations over the menstrual cycle, the effects of exogenously administered sex hormones on cognition, a variety of anomalies that allow us to separate prenatal hormone effects on later development, failed attempts to alter the sex roles of a biological male after an accident that destroyed his penis, differences in preferred modes of thought, international data on the achievement of females and males, to name just a few types of evidence that demand the conclusion that there is some biological basis for sex-typed cognitive development.

My thinking about this controversial topic has changed. I have come to understand that nature needs nurture and the dichotomization of these two influences on development is the wrong way to conceptualize their mutual influences on each other. Our brain structures and functions reflect and direct our life experiences, which create feed back loops that alter the hormones we secrete and how we select environments. Learning is a biological and environmental phenomenon.

And so, what had been a simple truth morphed into a complicated answer for the deceptively simple question about why there are sex differences in cognitive abilities. There is nothing in my new understanding that justifies discrimination or predicts the continuation of the status quo. There is plenty of room for motivation, self-regulation, and persistence to make the question about the underrepresentation of women and men in different academic areas moot in coming years.

Like all complex questions, the question about why men and women achieve in different academic areas depends on a laundry list of influences that do not fall neatly into categories labeled biology or environment. It is time to give up this tired way of thinking about nature and nurture as two independent variables and their interaction and recognize how they exert mutual influences on each other. No single number can capture the extent to which one type of variable is important because they do not operate independently. Nature and nurture do not just interact; they fundamentally change each other. The answer that I give today is far more complicated than the simple truth that I used to believe, but we have no reason to expect that complex phenomena like cognitive development have simple answers.

STEPHEN H. SCHNEIDER
Biologist; Climatologist, Stanford University; Author, Laboratory Earth

Climate Change: Warming Up To The Evidence

In public appearances about global warming, even these days, I often hear: "I don't believe in global warming" and I then typically get asked why I do "when all the evidence is not in". "Global warming is not about beliefs", I typically retort, "but an accumulation of evidence over decades so that we can now say the vast preponderance of evidence — and its consistency with basic climate theory — supports global warming as well established, not that all aspects are fully known, an impossibility in any complex systems science".

But it hasn't always been that way, especially for me at the outset of my career in 1971, when I co-authored a controversial paper calculating that cooling effects from a shroud of atmospheric dust and smoke — aerosols — from human emissions at a global scale appeared to dominate the opposing warming effect of the growing atmospheric concentrations of the greenhouse gas carbon dioxide. Measurements at the time showed both warming and cooling emissions were on the rise, so a calculation of the net balance was essential — though controlling the aerosols made sense with or without climate side effects since they posed — and still pose — serious health effects on vulnerable populations.  In fact for the latter reason laws to clean up the air in most rich countries were just getting negotiated about that time.

When I traveled the globe in the early 1970s to explain our calculations, what I slowly learned from those out there making measurements was that two facts had only recently come to light, and together they appeared to make me consider flipping sign from cooling to warming as the most likely climatic change direction from humans using the atmosphere as a free sewer to dump some of our volatile industrial and agricultural wastes.  These facts were that human-injected aerosols, which we assumed were global in scale in our cooling calculation — were in fact concentrated primarily in industrial regions and bio-mass burning areas of the globe — about 20% of the Earth's surface, whereas we already knew that CO2 emissions are global in extent and about half of the emitted CO2 lasts for a century or more in the air.

But there were new facts that were even more convincing: not only is CO2 an important human-emitted greenhouse gas, but so too were methane, nitrous oxide and chlorofluorocarbons (many of the latter gases now banned because they also deplete stratospheric ozone) , and that together with CO2, these other greenhouse gasses were an enhanced global set of warming factors. On the other hand, aerosols were primarily regional in extent and could not thus overcome the warming effects of the combined global scale greenhouse gases.

I was very proud to have published in the mid-1970s what was wrong with my early calculations well before the so-called "contrarians" — climate change deniers still all too prevalent even today — understood the issues, let alone incorporated these new facts into updated models to make more credible projections. Of course, today the dominance of warming over cooling agents is now well established in the climatology community, but our remaining inability to be very precise over how much warming the planet can expect to have to deal with is in large part still an uncertainty over the partially counteracting cooling effects of aerosols — enough to offset a significant, even if largely unknown, amount of the warming. So although we are very confident in the existence of human-caused warming in the past several decades from greenhouse gases, we are still are working hard to pin down much more precisely how much aerosols offset this warming. Facts on that offset still lag the critical need to estimate better our impacts on climate before they become potentially irreversible.

The sad part of this story is not about science, but the misinterpretation of it in the political world. I still have to endure polemical blogs from contrarian columnists and others about how, as one put it in a grand polemic: "Schneider is an environmentalist for all temperatures" — citing my early calculations. This famous columnist somehow forgot to bring up the later-corrected (by me) faulty assumptions, nor mention that the 1971 calculation was based on not-yet-gathered facts. Simply getting the sign wrong was cited, ipso facto in this blog, as somehow damning of my current credibility.

Ironically, inside the scientific world, this switch of sign of projected effects is viewed as precisely what responsible scientists must do when the facts change. Not only did I change my mind, but published almost immediately what had changed and how that played out over time. Scientists have no crystal ball, but we do have modeling methods that are the closest approximation available. They can't give us truth, but they can tell us the logical consequences of explicit assumptions. Those who update their conclusions explicitly as facts evolve are much more likely to be a credible source than those who stick to old stories for political consistency. Two cheers for the scientific method!

XENI JARDIN
Tech Culture Journalist; Co-editor, Boing Boing; Commentator, NPR; Host, Boing Boing tv

Online Communities Rot Without Daily Tending By Human Hands

I changed my mind about online community this year.

I co-edit a blog that attracts a large number of daily visitors, many of whom have something to say back to us about whatever we write or produce in video. When our audience was small in the early days, interacting was simple: we tacked a little href tag to an open comments thread at the end of each post: Link, Discuss. No moderation, no complication, come as you are, anonymity's fine. Every once in a while, a thread accumulated more noise than signal, but the balance mostly worked.

But then, the audience grew. Fast. And with that, grew the number of antisocial actors, "drive-by trolls," people for whom dialogue wasn't the point. It doesn't take many of them to ruin the experience for much larger numbers of participants acting in good faith.

Some of the more grotesque attacks were pointed at me, and the new experience of being on the receiving end of that much personally-directed nastiness was upsetting. I dreaded hitting the "publish" button on posts, because I knew what would now follow.

The noise on the blog grew, the interaction ceased to be fun for anyone, and with much regret, we removed the comments feature entirely.

I grew to believe that the easier it is to post a drive-by comment, and the easier it is to remain faceless, reputation-less, and real-world-less while doing so, the greater the volume of antisocial behavior that follows. I decided that no online community could remain civil after it grew too large, and gave up on that aspect of internet life.

My co-editors and I debated, we brainstormed, we observed other big sites that included some kind of community forum or comments feature. Some relied on voting systems to "score" whether a comment is of value — this felt clinical, cold, like grading what a friend says to you in conversation. Dialogue shouldn't be a beauty contest. Other sites used other automated systems to rank the relevance of a speech thread. None of this felt natural to us, or an effective way to prevent the toxic sludge buildup. So we stalled for years, and our blog remained more monologue than dialogue. That felt unnatural, too.

Finally, this year, we resurrected comments on the blog, with the one thing that did feel natural. Human hands.

We hired a community manager, and equipped our comments system with a secret weapon: the "disemvoweller." If someone's misbehaving, she can remove all the vowels from their screed with one click. The dialogue stays, but the misanthrope looks ridiculous, and the emotional sting is neutralized.

Now, once again, the balance mostly works. I still believe that there is no fully automated system capable of managing the complexities of online human interaction — no software fix I know of. But I'd underestimated the power of dedicated human attention.

Plucking one early weed from a bed of germinating seeds changes everything. Small actions by focused participants change the tone of the whole. It is possible to maintain big healthy gardens online. The solution isn't cheap, or easy, or hands-free. Few things of value are.

ROGER C. SCHANK
Psychologist & Computer Scientist; Engines for Education Inc.; Author, Making Minds Less Well Educated than Our Own

AI?

When reporters interviewed me in the 70's and 80's about the possibilities for Artificial Intelligence I would always say that we would have machines that are as smart as we are within my lifetime. It seemed a safe answer since no one could ever tell me I was wrong. But I no longer believe that will happen. One   reason is that  I am a lot older and we are barely closer to creating smart machines. 

I have not soured on AI. I still believe that we can create very intelligent machines. But I no longer believe that those machines will be like us. Perhaps it was the movies that led us to believe that we would have intelligent robots as companions. (I was certainly influenced early on by 2001.)  Certainly most AI researchers believed that creating machines that were our intellectual equals or better was a real possibility. Early AI workers sought out intelligent behaviors to focus on, like chess or problem solving, and tried to build machines that could equal human beings in those same endeavors. While this was an understandable approach it was, in retrospect, wrong-headed.     Chess playing is not really a typical intelligent human activity. Only some of us are good at it, and it seems to entail a level of cognitive processing that while impressive seems quite at odds with what makes humans smart. Chess players are methodical planners. Human beings are not.

Humans are constantly learning.  We spend years learning some seemingly simple stuff. Every new experience changes what we know and how we see the world. Getting reminded of our pervious experiences helps us process new experiences better than we did the time before. Doing that depends upon an unconscious indexing method that all people learn to do without quite realizing they are learning it. We spend twenty years (or more) learning how to speak properly and learning how to make good decisions and establish good relationships. But we tend to not know what we know. We can speak properly without knowing how we do it. We don't know how we comprehend. We just do.

All this poses a problem for AI. How can we imitate what humans are doing when humans don't know what they are doing when they do it? This conundrum led to a major failure in AI, expert systems, that relied upon rules that were supposed to characterize expert knowledge. But, the major characteristic of experts is that they get faster when they know more, while more rules made systems slower. The idea that rules were not at the center of intelligent systems meant that the flaw was relying upon specific consciously stated knowledge instead of trying to figure out what people meant when they said they just knew it when they saw it, or they had a gut feeling.

People give reasons for their behaviors but they are typically figuring that stuff out after the fact. We reason non-consciously and explain rationally later. Humans dream. There obviously is some important utility in dreaming.  Even if we don't understand precisely what the consequences of dreaming are, it is safe to assume that it is an important part of our unconscious reasoning process that drives our decision making. So, an intelligent machine would have to dream because it needed to, and would have to have intuitions that proved to be good insights, and it would have to have a set of driving goals that made it see the world in a way that a different entity with different goals would not. In other words it would need a personality, and not one that was artificially installed but one that came with the territory of what is was about as an intelligent entity.

What AI can and should build are intelligent special purpose entities. (We can call them Specialized Intelligences or SI's.) Smart computers will indeed be created. But they will arrive in the form of SI's, ones that make lousy companions but know every shipping accident that ever happened and why (the shipping industry's SI) or as an expert on sales (a business world SI.)   The sales SI, because sales is all it ever thought about, would be able to recite every interesting sales story that had ever happened and the lessons to be learned from it. For some salesman about to call on a customer for example, this SI would be quite fascinating. We can expect a foreign policy SI that helps future presidents learn about the past in a timely fashion and helps them make decisions because it knows every decision the government has ever made and has cleverly indexed them so as to be able to apply what it knows to current situations. 

So AI in the traditional sense, will not happen in my lifetime nor in my grandson's lifetime. Perhaps a new kind of machine intelligence will one day evolve and be smarter than us, but we are a really long way from that.