Psychology needs to develop mechanisms for data sharing

There's little common ground on how, when and whether data should be shared.

By Beth Azar
Monitor staff

Although many psy-chologists are per-fectly willing to share their research data with colleagues, few are yet prepared to deposit their work on the Internet for anyone who wishes to peruse, re-use or re-analyze it.

But that's what it would take to make data sharing truly useful to the psychology field, say researchers who pool data and draw broad conclusions from many different studies. How psychology can share data on a broad scale, with everyone participating, is anyone's guess.

Some suggest that funding agencies should require it as a provision of receiving grant money (see article page 1). Others think it's up to journal editors to make it a condition of publication--although none are ready to require archiving.

And psychologists, who have no real tradition of data sharing, are not ready to have a policy thrust upon them, says psychologist Fredric Wolinsky, PhD, editor of the Journal of Gerontology: Social Sciences, who's worked with the National Institute on Aging and the National Institutes of Health (NIH) to formulate a data-sharing policy. He says there's little common ground on how, when and whether research data should be shared.

"When I took over as editor in January 1998, I thought I would require minimal data sharing--archiving the data used in an article," he explains. "But it became very clear that I couldn't declare as policy what the field hadn't already agreed upon."

Data sharing is a concept psychologists will have to gradually accept on their own terms, adds Gary VandenBos, PhD, APA's director of publications and communications. Although APA's ethics code states that researchers should be willing to provide their data for scrutiny if someone wants to double-check their conclusions, such requests aren't common and are often found antagonizing.

"We don't have a tradition of data sharing," says VandenBos. "So if we're going to get it, we'll have to engage in a process of educating everyone about the value and benefits and make it into a cultural norm. I see that as a minimum of a 10-year process."

But even a consensus that data sharing is important to the field won't be enough to create a culture of data sharing in psychology, says David Johnson, PhD, director of the Federation of Behavioral, Psychological and Cognitive Sciences.

The discipline will also need the infrastructure for storing and sharing large amounts and multiple types of data. Disciplines such as sociology and economics have developed data archives and mechanisms for storing and accessing data at universities and on the Internet. Most of the data come from large longitudinal studies funded for the most part by the National Science Foundation (NSF). But psychology--with data ranging from surveys to PET scans to experimental statistics--has no such infrastructure.

Psychology has begun to wrestle with creating such structures, says Steven Breckler, PhD, project officer for the social psychology program in NSF's Social, Behavioral and Economic Science Directorate (SBE). In fact, SBE is running a competition for people with ideas for building up psychology's research infrastructure.

Several researchers have applied for grants to create a mechanism and methodology for psychologists to archive their data, says Breckler. One proposal came from VandenBos and Johnson, who would like to build a network that would house archived data and link it to full-text journal articles as well as to journal abstracts stored in searchable databases, such as APA's PsycINFO. The network would provide a powerful research tool for scientists who want to conduct meta-analyses or re-analyze controversial data. Such a network would also serve as an ideal teaching tool, says Johnson. Students, for example, could re-analyze data using several different statistical techniques and compare the results as a way to discover the powers and limitations of various methods.

NSF will announce the winners of their infrastructure grants this fall. Researchers who look broadly at the field of psychology hope that at least some of the proposals for creating data archives for psychological science will be selected.

But the greatest challenge to any data-archiving project, say psychologists, will be whether researchers will submit their data for archiving.

Some researchers balk at data sharing

the original researcher is unfair and inappropriate, he says.

Certainly, data sharing policies should proceed with caution and ensure that researchers have a reasonable period of exclusive use of their data, say funding agency administrators. But the question should not be whether data will be shared, but how and when. And, in some cases, agencies will force investigators to share their data, says Enoch Gordis, MD, director of the National Institute on Alcohol Abuse and Alcoholism (NIAAA).

"We'd all love it if we didn't need laws that prohibit things like robbery," says Gordis. "But people don't always act the way we'd like and there are some very distinguished researchers who have been very bad about sharing."

The virtues of sharing

David Johnson, PhD, director of the Federation of Behavioral, Psychological and Cognitive Sciences agrees.

"There needs to be an evolution in thinking by psychologists about what it takes to build a reliable body of knowledge in an area," he says. "If people are willing to only share results and not data, they will never have a great way of checking new results against previous results."

Sharing data also increases the amount of reliable data available on any scientific question, says University of Virginia psychologist and methodologist Jack McArdle, PhD, who chairs the Evaluation Advisory Committee of the NIH Center for Scientific Review.

"The inferences people and scientists alike often want to make are nationwide," he says. "So we have to be more careful about the representativeness and kinds of people we measure. For the behavioral sciences to move forward in terms of meeting our national needs, we must emphasize the use of large and representative samples that can only be collected through multiple large data bases, which can then be shared by all scientists."

It's this large-scale data that funding agencies are pushing researchers to archive and share first. For example, by next year the National Institute of Child Health and Human Development (NICHD) will provide public Internet access to its massive multisite longitudinal study of childcare.

"Scientifically, it's the right thing to do," says NICHD Director Duane Alexander, MD.

By the time the material is made public, he notes, NICHD investigators will have had several years to analyze the data as a group and another year for individual investigators to perform their own analyses.

It was a matter of balancing their right to a period of exclusive use, he says, with the rights of the broader scientific community and taxpayers who funded the study to see this unique resource.

Many NIH institutes have also begun requiring genetic researchers to deposit their data into archives accessible by other researchers. Some fear that this will allow mavericks to analyze data collected by someone else, and take credit for all the work. NIH institutes have tried to give the original researchers room to maneuver. The National Institute of Mental Health, for one, allows researchers to set their own time limits in data sharing plans submitted with their grant applications.

"Protection of the investigators is not simply a moral obligation--it's a public health question," says NIAAA's Gordis.

"If researchers have no incentive to do the hard work up front, in the future they won't do these studies. Data sharing is an important mandate--but we don't want to turn off the spigot [of good researchers]."

Rules are coming

How to protect researchers while also providing the widest possible access to federally funded data are the issues NIH and other funding agencies are struggling with as they try to formulate data-sharing policies.

Most of the policies on the table would take data out of the control of the researchers who collected them and put them into a public archive to be accessed either exclusively by researchers--as is done with genetics data--or by anyone who's interested--as will happen with the NICHD day-care study data.

Some researchers chafe at the idea. For data sharing to truly work, it's better if it's done as a collaboration between the original researchers--the data "owners"--and the researchers borrowing the data, they say.

"We don't want to get going on [data sharing] unless it's seen as win-win by both parties," says Penn State's Graham, who both borrows and shares data in his own work on drug abuse. "By building something together, everyone will be more productive. If you wrest it away--if there's a hostile takeover of the data set--it won't work."

While funding agency administrators tend to agree that researchers should be given as much control as possible over their data, they note that taxpayers actually own the research funded by the federal government. Others believe fears over data misuse or misinterpretation are misguided. For one thing, notes Ronald Abeles, PhD, special assistant to the director of the NIH Office of Behavioral and Social Sciences Research, the original investigator is not responsible for shoddy work by a secondary investigator. Poor research by secondary analysts will hurt his or her reputation and not that of the original investigator.

Dialogue over these issues is critical, says NIH's Baldwin, because data sharing policies are inevitable as the federal government pushes its agencies to open themselves up to public scrutiny. The hope is that the funding agencies will be able to formulate their own policies that keep the needs of researchers in mind.



Read our privacy statement and Terms of Use

Cover Page for this Issue

PsychNET®
© 1999 American Psychological Association

APA Home Page . Search . Site Map