THE IMPLICIT ASSOCIATION TEST

ANTHONY GREENWALD: In 1900, mathematician David Hilbert made a list of 23 recognized questions that the field of mathematics should regard as important unanswered questions and, lo and behold, many mathematicians decided to work on those questions. We psychologists do not have such widely agreed-upon lists of questions, but we do have a continuing good source of questions—the questions often come to us from society.

Our study of prejudice, for example, comes out of trying to deal with policy questions that leaders deal with and that we think need to be informed by science. Prejudice was a topic I discovered relatively late in my career. My dissertation advisor, Gordon Allport, may be best known for his 1954 book on prejudice, The Nature of Prejudice. However, when Allport was my advisor, in the early 1960s, he was no longer interested in the topic. I eventually came to an interest in prejudice because the work on attitudes that Mahzarin and I were doing in the mid 1990s produced a technique that turned out to shed new light on phenomena related to prejudice. The new technique, the Implicit Association Test,  was so generative of research that it produced a much more active collaboration than we had prior to that point.

MAHZARIN BANAJI:  We straddle a few different worlds within the science. We are interested in the mind as all psychologists are in one sense or another. We are interested in the social world outside the individual's mind. And we are interested in building a science of what these two do and say to each other.  We are specifically interested in these questions as they pertain to the topic of group membership:  how does our thinking change because of who the "other" is? Does it matter that the other is male or female, black or white? Does this change if I'm myself  male or female, black or white? In what ways, and why? To do so requires us to figure out new ways to peer inside the mind and these days, we have the possibility of looking at the brain directly.

We are at the same moment in our science that the older sciences were many generations ago when they first began to test the nature of the physical world. We've studied the mind only for the past 100 years but we are already at the stage of designing instruments that can look much farther than before. I have become fascinated with the history of the development of the telescope in part because I think we are at that moment where we are developing techniques that allow us to use tools , turn them inward, and look at the universe between our ears. To find that it is just as vast, just as complex, and just as tractable. This of course is immensely pleasing to people who want to be able to understand something as ephemeral and seemingly intractable as the mind.

My personal story is that I was a student in India who had studied some psychology, but veered off to questions of history and society. I ended up at a Marxist university where I learned a lot about sociology, but found its theories and methods unappealing. That is when I discovered that there was a field called experimental social psychology that its headquarters were located in the United States, in Columbus Ohio. . I packed my bags, showed up at Ohio State, and was fortunate to end up in the particular lab that I did, where I learned what I know and use today.

One of the oldest questions in social psychology is the one that interested Stanley Milgram when he asked: Why is it that reasonable ordinary Germans did what they did? I think we are asking a similar kind of question, but with a different set of methods. Ultimately the interest here is in how our behavior is out of line with our conscious intentions. By demonstrating that dissociation, between our conscious and less conscious views, we tap into fundamental aspects of the mind.

GREENWALD: At various points in my career I have worked on some very interesting questions, such as: How is voluntary action controlled?  What is the self? How does the mind function unconsciously? On that last one, I surprised myself by discovering that visual stimuli, flashed so briefly as to remain consciously unseen, can influence behavior. In starting this research I had no idea that was true. In doing the research, I found that we can learn new things by creating new methods. I had created new methods for studying subliminal perception, and shortly after than Mahzarin and I started working on developing methods to investigate things going on outside of awareness that could influence social behavior.

We were very influenced by cognitive psychologists' work on implicit cognition—which is the label for knowledge expressed implicitly in behavior even when the person performing the behavior was unaware of having the knowledge. We were looking for ways to study the implicit aspects of social behavior. An important moment came in 1995, when we performed an experiment that we had conceived a few years earlier and written into one brief paragraph of a proposal to the National Science Foundation.

In that experiment subjects gave a response on a computer keyboard with the index finger of the right hand to words that named pleasant things and to names of flowers. With left hand they were to respond to another two categories—words that named unpleasant things and insect names. This was a very easy task. Then we made one minor change: We switched hands for the flower and insect names. Now subjects had to give the same response to pleasant words and insect names and a different response to unpleasant words and flower names. Immediately the task became hugely difficult. The slowing on a response-by-response basis was on the order of 300 milliseconds, which was a magnitude of impact nobody could have expected. We certainly did not expect it.

I was the first subject in this experiment. When I experienced this slowing I found to my surprise that I could not overcome it—repeating the task did not make me faster. If I tried to go faster, I just started making errors when I was trying to give the same response to flower names and unpleasant words. This was a mind-opener.

About a month later, I modified the task by replacing the flowers and insects with the names of famous White and Black people. My thought was, if this works for flowers and insects, maybe we could use the same task to measure something that we had not yet begun to call implicit race attitudes. To my dismay, but also with some excitement, I discovered that the names of famous Black people were functioning much like the names of insects. I had a difficult time responding rapidly when I had to give the same response to famous Black people and to pleasant words. Shortly thereafter, I persuaded Mahzarin to start using this task in her lab at Yale. One of the students whom she brought along on this work was Brian Nosek, who ever since has been a very important collaborator in this research.

BANAJI: What is remarkable about this test, which is called the Implicit Association Test—the IAT—is that it allows you to be a subject in your own experiment. Most scientists do not have the remarkable experience of being the object of study in their own research.

If you are a physicist, you cannot be the material object that you are studying. If you are a psychologist, you cannot be the subject because you know the hypothesis. But when you study the unconscious aspects of the mind, you can! You can have the experience of having your hand NOT do what your mind is willing it to do. That may be part of the appeal of the test.

How does the test work? Well, I intend to associate white with good as quickly as I associate black with good—that is my conscious goal—but I fail to do that. It has to be fascinating to anybody to discover that they cannot do something simple that they will themselves to do. When we failed our own tests, we decided that this was of course for the scientific journals, but also for wider access. When in 1998 we decided that this was probably worth sharing with a wider audience than our own students, we asked, What would Galileo do, and we put it on the web.  

We were stunned that within the first month with no advertising on our part, just media coverage — we had 40,000 completed tests. We had clearly struck a chord or a nerve (depending on the participant's response!).

The test is unusual in that it provokes a reaction of surprise, even astonishment. It is both a tool to understand what goes on invisibly in our minds  but it is also a catalyst for insight. And once it has suggested to us and we may contain the views of multitudes, we can ask "Am I leading my life the way I want to?"  It is a unique test in that regard.

GREENWALD: The race IAT was critical. It is the version of the IAT that got the widest attention, and it is the one we would most often demonstrate in lectures. That is another of the IAT's virtues: you can give a group demonstration and show a room full of people all at the same time that they have this shared difficulty in giving the same response to Category A and to Category B, but if you the switch the sides of the two categories, it becomes very easy.

We, and also many of our colleagues, have been very motivated by the discovery that the IAT reveals these associations with race effect in our own heads. The effect does not show up in everyone, but in perhaps 70 to 75 percent of those who have tried the test. Some are upset by this observation. They are concerned that the test is, in effect, accusing them of being prejudiced. That reaction has led us to take pains to distinguish what this test measures from what is ordinarily meant by prejudice.

Prejudice is ordinarily understood as a state of mind that leads to intentional discriminatory behavior. People who have no intent to discriminate and no dislike of a racial group might think that their IAT result was characterizing them as prejudiced. We have repeatedly said, no, that it is not what the result means. In describing what the IAT measures, we have been careful to use phrases such as 'implicit attitude,' 'implicit preference,' and 'automatic preference,' while avoiding the unqualified word 'prejudice'. Incidentally, if we did describe the test as measuring prejudice, then we would be accusing ourselves of being prejudiced.

BANAJI: One of the great insights this test has provided for me is the ability to look at what might be evolutionarily "old ways" in which we tend to behave when left to our own devices. Our social preferences must have some roots in our early social groups and interactions. From having evolved in a world where people on the other side of the river were either people you killed or people who would kill you. We now live in a world where we have to outsource to those same people on the other side of the river! We have to be friends enough with them to understand their culture so that we can get them to do things with us and to think about our common fate.

It is one thing to say, "The law says you should do this or that."  It is quite another thing to say, Well, if we are really smart, and if we are really the adaptive creatures that we are, we are going to look at the ways in which we behave and see that they are not necessarily to our advantage. And as we learn we will change. And we will change in all of the ways in which we are going to need to. Like the work we need to do and will do to solve our environmental problems. Or our health problems. Eating too much is a problem because our bodies evolved in a world where food was far less abundant that it has come to be (for many people in the world). We cope by thinking about calorie intake and output in new ways. I think one of the tests of human intelligence will be whether we can take insights that are inconvenient truths about our minds, turn them around, and use such knowledge to create a better society – by which I mean, one that is line with our consciously chosen aspirations, rather than one we are being driven toward out of ignorance of who we are and our past.

GREENWALD: The IAT provides a useful window into some otherwise difficult-to-detect contents of our minds. In some cases, we find things we did not know were there. It may be "an inconvenient truth" that what's there is not what we thought was there or want to be there. But I think it is generally something we can come to grips with.

BANAJI: We have to believe that everyone is to blame, or rather everybody is responsible. It is not just the media. It is not just your parents. Instead, we find to be attractive those metaphors that come from air borne pollutants. What our minds acquire comes from the stuff that is hanging around in the atmosphere. It is in the water. It is in the air. When that is the case, you cannot hold individual people responsible. You can hold larger units and larger groups of people responsible in the same way that we do in order to solve the problem of environmental damage.

To come back to the test, I think an intuitive way to understand what it does is to imagine working with a deck of playing cards. If I ask you to sort the cards into two piles—red cards on one side; black cards on the other—you should be able to do so relatively easily. Let's say I measure the time it takes you to put the red cards to the right and the black cards to the left and use a stopwatch to time you. Then I say, "All right, I am going to time you again. This time put the hearts and clubs to one side, and the spades and diamonds to the other", and I start the stopwatch again. . We would all understand intuitively that the second sorting should take longer than the first. And the difference in time is indicative of the cognitive ease of the first relative to the second.

Like that, the IAT is trying to capture the difference between two sortings. When we encounter two things that have not been paired together very much in our experience, it takes a little longer to put them together because they are strangers to each other, making the task difficult. Working from this assumption, the IAT requests pairings of say Obama and good and Hillary and good and looks at the relative speed of judgment and the error rates. What is nifty is that you can replace Obama and Hillary with anything you want. If you are interested in looking at your preference for Coca-Cola versus Pepsi-Cola, you can adapt the test to do that.

GREENWALD: We have of course written quite a few papers reporting our research with the IAT, but those papers now represent only a small fraction of the total research literature on the IAT. This is because the technique has captured the interest of many of our colleagues. The question many are asking is, "Do these associations that the IAT measures have any impact on behavior?"  This has been the number one question in the research almost since the IAT's creation, and now, after ten years, there are over a hundred publications that have reported relations of IAT measures to behavior.

As many as a third of these publications have focused on the race test. What they have found—and this is not something we could have confidently predicted beforehand—is that the implicit race attitudes measured by the IAT predict behavior better than do the standard methods of measuring racial attitudes—the self-report methods of survey questionnaires.

In one type of experiment that has been done a few times, researchers video-recorded the behavior of a White person, who had previously taken the IAT, talking to a Black person in an ordinary conversation. The videotapes were then scored for indicators of discomfort, such as speech errors, maintaining distance, and turning away. The remarkable finding is that these indicators of discomfort are better predicted by the race IAT than by the same person's self-reported race attitudes. And now, with more than one hundred studies looking at "predictive validity"—the relation of what the test measures to behavior—we know that the test usefully predicts behavior.

BANAJI: Our paramount interest is the science. If it has positive social value, that is great, but in our daily lives what we spend our time on is to find ways to understand the nature of this phenomenon we've discovered – that our minds do things that we don't intend to,  can't always control, may even be embarrassed about and deny. The questions we ask as scientists are very ordinary. First of all, what is being measured? Statistically speaking, how large are the effects of the biases we are observing? Should we take them seriously? (If they are tiny, we need not bother with them; but if they are large, perhaps we should pay heed) Who and how many people are implicated? Is this a universal phenomenon? What does it predict? Does one's own group membership matter?   Can a bias be changed?

The question of prediction has been an important one. One could argue this way: Let's see, I am sitting in front of a computer and pressing two little keys. You are measuring something in milliseconds and telling me that I should take this seriously as a measure of how good a person I am? Of who I like and dislike? How I will behave as a doctor, a judge, a police officer?

To answer the question of what the test predicts, we needed studies that included the gold standard measure. The real thing. For example, if you have money or other resources, whom will you give it to? Whom will you cooperate with? Who will you hire to be on your team? If you are a doctor, to whom will you prescribe a particular medical procedure? Do you prescribe it equally? These are the real behaviors to which the test needs to speak What if if the test is simply picking up some random set of mental associations that have no value in the real world?

That piece of research on predictive validity has been done largely by people other than ourselves. There is now a robust body of work showing that in certain domains, this test outpredicts the usual questions about human behavior that are directly posed to respondents on a survey-type measure.

GREENWALD: We have tried to make the IAT method easily available to other researchers. The instructions and the stimuli are under copyright, but we have made it clear to any researcher who asks that they are entirely free to use the test in their scholarly research. For uses in commercial applications, like diversity training, market research, polling, etc. where people want to use the test to generate income, we license uses of the test.

BANAJI: I think we have taken the position that this test is useful for two purposes: as a scientific tool, and for education, at the individual level and at the level of communities discussing its meaning and its use. We have taken the quite firm position that this test is not to be used in the selection context, and we say so very explicitly at our website and whenever we have the opportunity. We do not believe that this is the kind of test you give somebody and then say, for example, "Ah-ha!  We see that you have a larger-than-average race bias, so you cannot serve on a jury." 

Is it possible, that our warning notwithstanding, that the test will be misused in this way? Yes, as it is with anything. An AIDS vaccine could be misused in some way, but we would not suggest that we not develop one for that reason. There is always potential, when you discover something new, for it to be misused. But this is why when the scientists who know most about it take the position that it cannot be used in certain contexts, we are building in a strong protection.

GREENWALD: One of the considerations prompting us to say that, ordinarily, the test should not be used for selection—such as of jurors or employees—is that we know we would not qualify for many responsible positions if a selection criterion were absence of implicit bias on an IAT measure. But we do not think we should be disqualified from such positions just because we have these associations in our heads. At the same time, we do think that it's useful for us to be aware of the associations measured by the IAT and to recognize that they can influence our behavior in ways we may not be happy with.

BANAJI:  Liberals sometimes have trouble with this test. Thy have put themselves into a separate category where they do no harm but everybody else, say those who live south of the Mason-Dixon or whatever, are the really prejudiced people.   When we show that self-professed liberals are biased too, there is some  squirming.

GREENWALD:  The test has critics, but of about 500 scientific publications on the IAT so far, perhaps two or three percent are critical. This criticism has sometimes been very useful in guiding research. In some cases, the research has shown that the criticism was not valid. In other cases, the criticism has been found to be valid and has led to improvement of the IAT's design.

BANAJI:  But there is a response from the scientists which has parallels in the older sciences. Our methods are certainly critical of older methods in the sense that we believe that for a hundred years psychology relied extensively on self report, on asking people, "What do you think about this or that?". Just as with hypertension, where I no longer ask "What do you think your blood pressure reads right now?"—I would measure it with an objective tool—here is a situation in which asking people may not always lead to the useful answer.  

Confirmation from other measures of what the IAT is tapping can be useful and we have such research, adding newly to our interpretations. One tool that I have found to be helpful in this regard is the use of fMRI to observe neural activity in predefined brain regions known to have particular capacities and to relate the IAT measure to it.

GREENWALD: To this point, there have been relatively few studies lining up fMRI imaging results with the IAT. Whereas the IAT can be administered in less than five minutes and costs very little to administer, obtaining the imaging data usually requires at least an hour, 20 or 30 minutes of which is spent inside an MRI scanner that makes a huge racket.

BANAJI: It has been ten years since the test went up on the Web—it went up in 1998—and we may have surpassed the six million mark in terms of completed tests. This makes it probably the largest such data collection on the Web. In terms of the research, it is very gratifying to see groups of people just picking it up and doing whatever they want to with it, including high school students.

One area of great interest to me personally is the question of where these biases emerge from, especially the intergroup ones. Both in terms of evolution and in terms of our development from birth to adulthood. A large part of my lab's work right now is focused on understanding how these biases operate in the minds of young children. Can we develop analogs of this test so that we can use it with infants? (Don't know yet). We'd like to figure out how early in life babies become fascinated by what is familiar, what sounds like their mother, and from which a sense of self versus other, us versus them emerges.

I am very interested in the connections to the neuroscientific evidence. Can we converge, with different types of evidence on understanding how we treat others who are not like ourselves? Members of other species, members of our own species who have different beliefs (Hindu vs Muslim). Research by my colleague Jason Mitchell shows that indeed we may not engage the same brain region when thinking about somebody who is say asocial democrat (similar to self) versus somebody who is different (a fundamentalist Christian). Judges need to know that. All of us need to know that.

GREENWALD: Within the next ten years, we expect a few uses for the IAT to become well established. One already developed use is educational, providing new ways to achieve what diversity training programs now try to achieve. There is much interest in using the IAT in this way to give people some self-insight that might help them improve how they function in their jobs.   Another developing use is for clinical psychological diagnosis. Imagine someone taking an IAT on a handheld computer in the waiting room of a hospital or clinic, rapidly providing information that can help the clinician to understand something that the patient may be unable to describe to the clinician. Clinical psychologists have recently been adapting the IAT for such uses, although we have yet created the technology needed to administer it in the waiting room.

[Click here to take the "PRESIDENTIAL CANDITATES IAT"]