Are psychologists shooting themselves in the foot?

Some researchers? ?bad habits? may be undermining the social and behavioral sciences.

By Beth Azar
Monitor staff

Mathematical psychologist Trisha Van Zandt recently had to step out of her lab and face what most researchers consider a grueling experience: A live interview and spontaneous questions from a call-in audience on CNN?s ?Washington Journal.?

?I didn?t want to do it,? says Van Zandt, who was invited on the show because she was a winner of the Presidential Early Career Award. ?Basic research is terribly hard to explain and I have stage fright.?

But National Science Foundation (NSF) officials urged her to do it, saying it was her duty to explain, her research to the people who fund it. She agreed.

?Part of the reason behavioral science has such a squishy standing among the other sciences is that we don?t talk about what?s going on,? she says. ?And when we try to explain the hard stuff, it comes across badly because we haven?t practiced enough. We shoot ourselves in the foot every day that way.?

Psychologists? reluctance to hawk their research?or, in some cases, their lack of experience in working with the media?is just one example of how they undermine their own efforts and inadvertently damage the field, say policy experts.

Researchers have also been accused of some other ?bad habits,? such as shying away from applying for competitive research grants and ?eating their young? by being overly harsh reviewers of their peers? work.

It?s unclear from any data whether psychologists as a group are truly worse than other scientists on these issues. However, they are important issues for all scientists to ponder, say science policy experts.

?Researchers certainly have to look out for themselves and their reputations,? says APA Executive Director for Science Richard McCarty. ?But we also have to realize how our actions?whether it be refusing to talk to the media or deciding not to send in that grant application?impact our field as a whole.?

A hard sell

Psychological researchers? failure to court the media may do the most damage to the field, policy-makers say. The mass media have enormous influence on how the public understands and appreciates science, and, therefore, on how much money legislators are willing to spend to support research endeavors.

?It?s not like funding is easy to come by any more,? says Van Zandt, an assistant professor at Johns Hopkins University. ?I have to convince my peers as well as my legislators that what I?m doing has broader implications for the nation?s welfare.?

But scientists aren?t particularly adept at communicating what they do, says Daryl Chubin, PhD, who worked as assistant director for Social and Behavioral Sciences at the White House Office of Science and Technology Policy for the past year. And, in some ways, social and behavioral scientists have unique problems to overcome.

For one, the public often responds to behavioral and social science findings as if the researchers are rediscovering something ?grandma always knew,? says Joe Young, PhD, NSF?s Social, Behavioral and Economic Research information director. Or, if researchers use too much jargon, people think they?re trying to make something simple seem more complicated than it is.

Psychologists also don?t often have striking breakthroughs to report, says Van Zandt. Indeed, a 1995 survey by William Evans found that the media often fail to depict the social sciences as ?science.? Evans evaluated science coverage by The New York Times, The Los Angeles Times and the evening news telecasts of the three major television networks.

He found that:

? Less than 10 percent of the stories published in the science sections of the newspapers involved social science. Instead, social science articles appeared most often in the papers? general news sections.

? Reporters more often named a scientific source for natural science stories than for social science stories: 51.4 percent versus 14.3 percent in The New York Times; 47.8 versus 6.5 percent in The Los Angeles Times; and 16.7 percent versus never on the evening news.

? Both newspapers and all three news shows used the terms ?scientists? or ?researchers? to describe those responsible for natural science research more than 80 percent of the time. In contrast, when describing social scientists they used the terms ?the authors of the study? or ?the writers of the report? more than 80 percent of the time.

Despite these impediments to good media coverage, psychological researchers must continue to try to translate their research for the public to advance public and legislative interest in the field, says Young. (See sidebar for tips on handling media interviews.)

Send in the grants

Researchers may also risk decreased funding if they fail to send in grant applications when funding gets competitive?another bad habit some say psychologists tend toward. There doesn?t seem to be an overall downward trend in the number of behavioral research grant applications being submitted to funding agencies, says Norman Anderson, PhD, director of the National Institutes of Health Office of Behavioral and Social Sciences Research (OBSSR). But agencies such as the National Institute of Child Health and Human Development (NICHD) have seen such a sharp decline in psychologists? applications over the past few years, they?re beginning to worry about the health of their behavioral research portfolio.

In particular, HUD-1, the NIH review panel that reviews applications on normal child development, receives half the number of applications per funding round than it did three years ago, says Anita Sostek, PhD, chief of the NIH Biobehavioral and Social Sciences Institute Review Group and overseer of HUD-1 and the HUD-3 review panel, which reviews applications on abnormal development.

Fewer child health applications are coming into HUD-3 as well, says Sostek. They used to make up as much as 80 percent of the grants HUD-3 reviews. But that proportion has dropped to about 50 percent. Overall, the behavioral applications are still coming in, says Sostek, but child health applications seem to be on a decline.

And such declines can hurt, says Sostek. NICHD, along with many other NIH institutes, funds a certain percentage of all the grants it receives. So a field of research that sends in fewer grant applications loses out.

?Thirteen percent of very little is little and 13 percent of more is more,? says Sostek.

Also, if a funding agency or a review panel notices a decline in the number of grants in a given area, the organization may assume there?s little research activity there and stop funding that area altogether, says Sarah Friedman, PhD, director of NICHD?s Program on Cognitive, Social and Affective Development.

Several private funding agencies have also complained of a dearth of behavioral research applications. The March of Dimes Society has seen its supply of behavioral research proposals decline by more than half over the past several years, says Vice President of Research Michael Katz, MD. And in 1996, the American Cancer Society was so concerned with declining behavioral research proposals, it launched a special program to attract more behavioral researchers.

A critical nature?

Even if psychologists send in plenty of grant applications, they may face another obstacle from within their ranks?their own harsh reviews of each others? work. Most psychological scientists and funding agency officials admit that, although they have no hard data to prove it, they have developed the impression that psychologists batter their peers during grant review.

One impression is that psychologists pick apart methodological details and statistical analyses. In contrast, researchers from other disciplines seem focused more on the overall quality of a grant proposal, says Georgia Tech psychologist Anderson Smith, PhD, who has chaired many interdisciplinary grant review panels during his 30-year research career.

?There are very good reasons to be exceptionally critical? of research methodology, says neuropsychologist Reid Lyon, PhD, acting director of the Learning Disabilities, Cognitive and Social Development Branch at NICHD. ?But there?s a balance between being critical and realizing that there are many things that the researcher can fix that aren?t fatal flaws.?

Picky critiques won?t necessarily hurt a grant application?s score, but institutes often use them when choosing between two applications with similar scores. One institute director complained that when his council debates which grants to fund, behavioral applications often lose out because the critiques are so harsh, says OBSSR?s Anderson.

He and his staff are now investigating whether behavioral science review committees at NIH are tougher than biomedical review committees. Preliminary findings suggest that there?s no difference in the average scores the review committees assign grants. Now they must examine whether the critiques that accompany grant reviews are more critical of behavioral research than biomedical research.

Some believe Anderson will find that other sciences are just as critical of each other as psychologists. Geneticists allegedly tear each other apart in review, say several officials. In the end, it could be that there are pockets of hypercritical reviewers in various branches of psychology but not others, says Jaylan Turkkan, PhD, director of the Basic Behavioral Research Branch at the National Institute on Drug Abuse.

Anderson hopes the OBSSR study will finally determine whether the myth of the hypercritical behavioral scientist is based in fact or fiction. He expects to have some answers this summer. Either way, researchers need to become more aware of how the wording of their reviews can sway the funding of their entire field.

?We shouldn?t cease to be critical reviewers, nor should we waste time writing unfundable grant proposals that have no prospect of funding,? says McCarty. ?But we need to be mindful about the implications of our actions as reviewers on the applicant and on the discipline.?