Feature

Some attendees likely thought they'd stumbled into a computer-science class: With a curriculum covering programming languages such as MySQL, PERL and JAVA, a new five-day APA workshop held in July taught psychological scientists and students how to conduct Web-based experiments. As emerging technologies and complex research methodologies augment time-tested techniques such as reinforcing rat behavior with cheese, APA is offering such workshops to help researchers learn and master new skills.

"Performing Web-based Experiments" made its debut as an APA Science Directorate Advanced Training Institute (ATI) at California State University at Fullerton (CSUF). Organized by Michael Birnbaum, PhD, a psychology professor at the host campus, and taught by him and six other international experts, the workshop also covered more familiar psychological science with additional discussions of methodology, recruiting, dropouts, retention, ethics, multiple submissions and online panels. But the workshop put a Web-experimentation spin on those and related subjects.

Birnbaum, who directs the university's Decision Research Center, has literally written the book on the workshop topic. His works include "Psychological Experiments on the Internet" (Academic Press, 2000) and "Introduction to Behavioral Research on the Internet" (Prentice-Hall, 2001).

"Because of his expertise on the topic, and because he already had taught similar courses in the past, we knew he would provide the high quality training we sought," says Steven Breckler, PhD, APA's executive director for science.

A hands-on experience

In this five-day course, divided between lectures and hands-on work, each student learned how to run experiments via the Web. In Birnbaum's typical semester-long CSUF courses, students collect data on 150 to 250 participants, write an APA-style paper and give a convention-type presentation on the results they've learned to gather and analyze online. The class covered even more computer techniques, but the time did not allow for completion of intensive, long-term research projects.

"The time went by fast because we were following along each session by programming on our own computers," recalls attendee Diane Milillo, a social psychology graduate student at the University of Connecticut. "I don't think I would have learned as much if I weren't participating in the hands-on activities."

Web-based experimentation in psychology cuts costs--just as it does in industry and other spheres of society, note workshop organizers. In research, that means savings on lab space, assistants, data entry and time collecting data. The Web also enables researchers to recruit large samples quickly and to reach specialized types of participants who are rare in the general population--something that appealed to Milillo, whose dissertation examines women's perceptions of sexism. She wants to reach groups that are stigmatized or otherwise hard to reach through a normal undergraduate psychology participant pool.

The international faculty included Birnbaum and John Krantz, PhD, of Hanover College in Indiana; Anja Goeritz, PhD, of the University of Erlangen-Nuremberg in Germany; Gary McClelland, PhD, of the University of Colorado at Boulder; William Schmidt, PhD, of the State University of New York at Buffalo; Ulf Reips, PhD, of the University of Zurich in Switzerland; and John Williams, PhD, of the University of Northern Iowa.

In demonstrating the basic techniques for implementing and managing Web-based surveys and experiments, they filled a void because, according to Birnbaum, most psychologists doing Web work are self-taught. Yet Milillo says, "Designing and programming your own studies, specifically with HTML and JAVA [programming languages] can be very specific. It is critical to learn with the help of experienced others."

Conducting experiments on the Web requires researchers to know the basics about programming pages and forms for Web use, no matter whether they code from scratch or use licensed software that automates the process.

And no Web-based anything would be possible without hardware and software. Thus the workshop taught participants how to install and run their own servers (the computers that host an owner's Web information and post it to the Internet), install and use databases and make the most of the newest generation of programming languages.

Faculty demonstrated how these programs can help researchers to control and measure response time, as well as to prepare and control visual and auditory media.

"I felt like I was at a conference of the founding fathers of Web research," says attendee Peter Caprariello, a graduate student in clinical and social psychology at the University of Rochester. "I have no doubt that the knowledge that I gained from this ATI will be applicable to all aspects of my career as a researcher."

Birnbaum expects that, eventually, most graduate programs will cover this kind of research. Most of the computer techniques and methodological tactics of this area of research are quite new, dating only to 1995, when Krantz ran the first Web experiment and Reips opened a Web-based experimental psychology lab. Within the field, the subfields of social and cognitive psychology appear to lead the pack in terms of research conducted on the Web, Reips found in a recent empirical analysis in press.

Advanced Training Institutes

Since 2000, APA has offered Advanced Training Institutes (ATIs) to teach psychological scientists the newest, most sophisticated ways to learn about behavior. ATIs offer hands-on experience in a variety of burgeoning psychology research areas, such as functional magnetic-resonance imaging (fMRI).

Virginia Holt, APA's assistant executive director for science, says that the association's Board of Scientific Affairs (BSA) wants to address the need for scientists to get up to speed on new methods, techniques and technologies in psychological science.

As a result, every summer, prominent research institutions across the United States host the ATI courses. This past summer, in addition to the Web workshop, an ATI at the National Magnetic Resonance Center at Massachusetts General Hospital showed students how to use fMRI, and another at the University of North Carolina at Chapel Hill offered instruction on using large-scale databases, focusing on the National Institute of Child Health and Human Development (NICHD) Study of Early Child Care. Another ATI workshop held at the University of Virginia in Charlottesville showed psychologists how to do structural equation modeling in longitudinal research.

Including this year's attendees, more than 500 psychological scientists--faculty members, postdocs and advanced graduate students--have taken advantage of ATI training. Tuition varies for each workshop, depending on grant underwriting and student or faculty status. APA members received about a one-third tuition reduction, and non-members can apply for membership when they sign up for a workshop.

"The ATIs are specifically aimed at those whose primary work is research," says Holt. "Each ATI attracts a slightly different audience--for example, the people who want to attend the ATI on using large-scale datasets are often developmental, cognitive or educational psychologists, as the dataset we are using is the one resulting from the NICHD 10-site day care study."

The Science Directorate now offers several recurring ATIs. The first, on fMRI, was held in 2000 and funded by NIMH; it's been taught six times. The APA-funded Institute on Longitudinal Research Measurement and Methods is in its fifth year at the University of Virginia. And in Chapel Hill, the NICHD-funded workshop on large-scale databases is in its third year.

ATI topics can be hard to find elsewhere. "Some of these topics are already found in graduate coursework, but they are often so specialized that you don't find them many places," Holt explains. "Some, such as fMRI, might be more likely found in independent study situations rather than a formal course."

The Science Directorate is encouraging more attendance from minority and international learners and hopes to expand its ATI offerings, notes Breckler.

"We constantly review our course lineup and want to hear ideas for new courses to develop," he says.

Rachel Adelson is a writer in Raleigh, N.C.

Further Reading

To sample online experimentation, visit Michael Birnbaum's site for judgment and decision-making experiments. Go to the Science Directorate's Advanced Training Institutes for more information.