IAT: Fad or fabulous?

The Implicit Association Test (IAT) is one of those rare research tools that has transcended the lab to catch the attention of not just the social psychologists, who use it in increasing numbers, but also a large swath of the general public. In fact, the IAT has been written about in newspapers, featured on radio and television and garnered more than 5 million visits to its official Web site (https://implicit.harvard.edu) by people who want to take the test.

The excitement stems from the kinds of associations researchers have used the test to measure. At its core, the IAT assesses how closely people's brains link concepts, which can be as benign as "flowers and pretty" and "insects and yucky," but can include items such as "blacks and bad" and "women and passive." Many social psychologists believe that these cognitive associations lead to "implicit bias," which may influence subtle forms of discrimination.

Although social psychologists have developed many measures for studying implicit bias, the IAT is by far the most popular. In the 10 years since University of Washington social psychologist Anthony Greenwald, PhD, first described the IAT in the Journal of Personality and Social Psychology (Vol. 74, No. 6), it's been used in about 300 published studies and cited in 800 articles.

The IAT has also attracted its share of critics. Some debate the test's psychometric validity and reliability. Others worry that the publicity is pushing it into the public sphere--particularly into the hands of legal scholars who argue it can be used to reshape antidiscrimination laws--before it's been properly vetted.

"The IAT is not yet ready for prime time," says the University of Virginia's Greg Mitchell, PhD, JD, a Berkeley-trained social psychologist turned law professor. "I think this research is important research and the people doing it are very good scientists with noble intentions. But noble intentions don't make good public policy."

Psychometric skirmish

One of the IAT's most vocal critics is Texas A&M University psychologist Hart Blanton, PhD. He worries that the IAT has reached fad status among researchers without the proper psychometric assessments to warrant its current uses in the public domain.

Blanton has published several articles detailing what he considers the IAT's many psychometric failings, but if he has to highlight one weakness, it's the way the test is scored. The IAT measures people's associations between concepts. So, the classic race IAT compares whether you're quicker to link European-Americans with words associated with the concept "bad" and African-Americans with words related to "good" or vice versa. Your score is on a scale of -2.0 to 2.0, with anything above 0.65 or below negative 0.65 indicating a "strong" link.

"There's not a single study showing that above and below that cutoff people differ in any way based on that score," says Blanton.

Guilty as charged, says the University of Virginia's Brian Nosek, PhD, an IAT developer. However, most social psychology measures use arbitrary metrics. The difference is that he and his colleagues provide the general public with feedback about what specific scores might mean.

"Where I diverge [from Blanton] is in what I think we're trying to say when we give feedback," Nosek says.

Blanton views the feedback as a diagnosis and wants it held to the same standards as any clinical diagnostic tool. Nosek, Greenwald, and their colleague Mahzarin Banaji, PhD, of Harvard, see it differently. They view the feedback as an educational device to get people thinking about implicit bias and how it may color their interactions with the world. The background material on the IAT Web site, they say, makes it clear that people should not overinterpret their results.

Dealing with the noise

Another reason IAT critics think that the Web site shouldn't provide feedback is because the measure is quite sensitive to the social context in which it's taken: In fact, people's scores often change from one test to another.

"It's not as malleable as mood and not as reliable as a personality trait," agrees Nosek. "It's in between the two--a blend of both a trait and a state characteristic."

Russell Fazio, PhD, a social psychologist at Ohio State University, describes the IAT as "noisy." There's no way to determine whether it's measuring unconscious attitudes or simply associations picked up from the environment, he says.

"The bottom line is that it has a potential to be a remarkably powerful tool," he says. "But as traditionally implemented, it really has problems."

He's developed his own version of the measure that he thinks homes in on whether associations are true unconscious attitudes or not. It's called the personalized IAT. Instead of pairing categories like race or gender with such general terms as "good" and "bad," Fazio pairs them with "I like" and "I don't like." Studies comparing the personalized IAT with the traditional IAT indicate that it provides a more valid assessment of people's attitudes, says Fazio.

While Nosek and Greenwald say they agree that the IAT measures external influences and not just personally held attitudes, they add that it's a reflection of reality, not a problem with the test.

"In my view," says Nosek, "implicit associations are the sum total of everyday associations."

Premature publicity?

In the end, the relationship between IAT scores and the real world will determine its value. In fact, to prove the test's worth, the test's developers have recently completed a meta-analysis of the IAT's predictive power, which is in press.

"We found that in the domain of intergroup discrimination--race, age, sexual orientation--the IAT does better than self-report at predicting behavior," says Greenwald.

Northwestern University social psychologist Alice Eagly, PhD, thinks the meta-analysis shows that the IAT provides modest predictions of behavior. "The IAT adds something," she says, "but it's not a direct line to the unconscious."

Given these findings, the IAT is not yet ready for use in applied settings such as courtrooms, critics say. But the hype and public promotion of the measure have garnered the attention of many legal scholars who have begun to use the research to bolster workplace and other types of discrimination cases, says Mitchell.

In fact, Greenwald and others have discussed concepts of implicit social cognition in court. In one case of workplace discrimination, Greenwald described the literature on implicit bias in rebuttal to a defense expert who, says Greenwald, had inaccurately described the research. However, he and his colleagues have made it clear that they do not support the use of specific IAT results in court either to help select a jury or screen witnesses for implicit bias.

Mitchell thinks any discussion of implicit bias in the courtroom goes too far based on the state of the science. "The idea that we have associations that may be primed by the stimuli on the IAT sounds perfectly plausible," says Mitchell. "What those associations mean and what they indicate is an open question. To equate it with automatic preferences for different social groups is much less plausible."