Science Watch

Airport security screeners may not benefit from training and assessment programs that consist of viewing a small set of pictures of the same knives or guns over and over again, according to research in the November issue of the Journal of Experimental Psychology: Learning, Memory and Cognition (Vol. 31, No. 6). The research suggests that when people practice searching for the same target objects among many distracters-like a security screener inspecting an X-ray for contraband items-they get very good at finding the specific objects with which they were trained. However, they don't improve nearly as much on finding other objects in the same category, like slightly different-looking knives or guns.

The research, led by psychologists J. David Smith, PhD, of the University at Buffalo of the State University of New York, and David Washburn, PhD, of Georgia State University, may have implications for refining and improving the program the Transportation Security Administration (TSA) uses to train and assess airport security screeners. The research was funded by the Transportation Security Laboratory, which was part of the TSA and is now under the Department of Homeland Security.

"Our main question was: When someone looks at a cluttered display inside a suitcase, how good are they at applying the general knowledge they have about knives and guns to that display to find a specific knife or gun?" Smith says.

From shapes to knives

Smith and Washburn's study combines two fundamental areas of cognitive psychology research: categorization and visual search. The researchers began by training 64 undergraduate participants to recognize three different categories of shapes, which were developed using a classic method called dot distortion. In this method, a computer randomly generates nine dots, connects those dots with straight lines, and fills in the internal areas to create a prototype shape. Then, the computer can create infinite numbers of related shapes by displacing the dots slightly from their original positions and repeating the process, making shapes that are similar to-but not exactly the same as-the prototype. All of the related shapes form a category.

"The nice thing about this method is that it's been around since the 1960s, it's been studied a lot, and it's very well understood," says psychologist Todd Maddox, PhD, who studies categorization at the University of Texas at Austin.

Smith and his colleagues trained the participants by showing them 120 images from each category-360 images total-and asking them to guess which category each image belonged to, and then correcting them if they were wrong. By the end of the training, participants could correctly and quickly categorize the shapes more than 75 percent of the time.

Next, in the screening part of the experiment, the participants viewed displays that contained seven complex shapes, one or none of which might come from one of the three target categories. The participants pressed a button to indicate whether they saw a shape from category A, category B, category C or no category. The screening task was meant to simulate an airport security screener's task, so the shapes in the display were jumbled together and overlapping, like objects in a suitcase.

The participants completed 432 screening trials, in 16-trial blocks. For the first nine blocks, the researchers used exactly the same target shapes. At block 10, though, they switched, picking a new shape from each category-and then they switched again at block 19.

The participants' performance on the task improved markedly from block one to block nine-from about 25 percent correct to more than 40 percent correct. However, as soon as the targets changed at block 10, performance dropped all the way down to about 25 percent correct. Again the performance improved to more than 40 percent by block 18, but it plummeted once more as soon as the researchers switched to novel shapes again at block 19.

"The main result this shows is that even if you give people extensive practice screening with a small set of objects, as soon as you put them into a new situation they show a strong dependence on the particular objects and minimal capacity to use category-general knowledge," Smith says.

In fact, in a follow-up study, Smith and his colleagues ran the screening test again, but this time never repeated any shapes-and in this experiment, the participants' detection accuracy never improved.

Next, the researchers wanted to see whether their findings applied to real-world objects. TSA gave them X-ray images of suitcases containing both threatening and nonthreatening objects, which the researchers cut up and rearranged to form 288 suitcase X-ray images. Then, they ran the screening trials the same way they had in the first experiment. Again, the researchers found that with practice, participants got better at finding the target objects. However, as soon as they switched from suitcase images containing one particular knife to suitcases containing a different knife, much of the improvement disappeared.

"This result, frankly, is scary," says Maddox. "When I read it for the first time, I almost didn't believe it."

Improving training

The study provides fodder for theories of cognition and for the practice of airport screening, according to experts in the field.

Anne Hillstrom, PhD, who studies visual search at the University of Portsmouth in the United Kingdom, says that most research on visual search has used simpler targets, like letters or numbers or simple shapes. "I think people who deal with security screening had a hard time applying our research because many of our displays are simpler than the displays used in security screening," she says. "What David's done by combining the visual search with the models of categorization is something quite new."

On the practical side, TSA has already refined one of its training and assessment programs based on the concerns that prompted Smith's research. The Threat Image Projection (TIP) system, which is installed in baggage screening X-ray machines at airports around the country, randomly projects images from a library of threatening objects onto the X-ray of an otherwise innocent bag. If the screener finds the TIP object, he or she is congratulated and the image disappears.

Smith's research highlights the fact that TSA avoided a potential flaw in this system-that once screeners became familiar with the TIP images, they might have been more likely to accurately identify those images than they would to identify new, and therefore real, threats.

In a prototype version of the system, the agency included only a couple hundred images in the TIP library, according to Transportation Security Lab psychologist Josh Rubinstein, PhD. But before the system was finalized, they increased that number to several thousand.

"With several thousand items, a screener won't see the same object more than once per year, if that," Rubinstein says.