By John E. Williams, PhD
Because of the potential of web-based research to enhance and facilitate data collection, and the lack of instruction on these methods in most psychology training programs, many psychologists are now seeking to acquire web-based skills on their own. One way that colleagues and I are contributing to the skillful use of web-based surveys and experiments in psychology is by organizing an Advanced Training Institute in Performing Web-Based Research, sponsored by the American Psychological Association's Science Directorate. The next workshop will be held July 9-13, 2007, at the University of Northern Iowa.
Conducting psychological research using the Internet as a delivery tool opens many possibilities to researchers. Many researchers may think the Internet is only suitable for presenting surveys to unknown possible participants. However, the Internet can be used with as much control over methodology as a lab setting and used imaginatively in combination with traditional methods.
Several possibilities come to mind: using the Internet to collect data from low base rate populations; using the Internet to collect a diverse and representative sample for a specific population; using Internet based methodology to accomplish automatic data collection; using Internet based methodology to accomplish real-time data and trend analysis. There are many more possibilities limited only by the researcher's inventiveness and access to appropriate expertise.
One example of the use of the Internet for research was when I conducted a study designed to collect clinician perceptions of various automated psychological reports then provided for the MMPI-2. In this study, clinicians logged into a website to view, download, and print their respective reports. They also responded to questions online. Data was collected in an online database and summary statistics provided on a webpage. In addition, clinicians were able to view past reports, and track the status of their participation, all online.
I used this approach for several practical reasons; there were few available clinicians in my relatively rural area; I wanted a sample representative of clinicians on a national level; I wanted to avoid the expense of mass mailings via the postal service; and I wanted automatic data collection avoiding possible input errors. Using the Internet as the means of method delivery accomplished all of these.
Another example employs using the Internet to coordinate and collect data from multiple lab settings. In a multi-site study, using the Internet allows all data to be collected centrally in a database. I use the same procedure for collecting information from three to four sites where all participants use the same online application. The data is summarized and participants get immediate feedback on their performance. Depending upon the need for stringent method control, these procedures can be carried out in a lab under the supervision of a research associate or independently by the participants at a location of their choice.
Multiple options exist for creating web applications for psychologists. Survey research can often be accomplished through the use of html and one of several available server-side scripting languages (i.e. php, vbscript, perl). Using a server-side script is crucial for saving the data to a database or text file. In addition, it allows a researcher greater flexibility than can be obtained easily in a pencil-and-paper version of a survey. For example, surveys can be automatically randomly presented or presented in counter-balanced order. In fact, individual questions can be randomized or presented in ways that would not be possible otherwise. However, a researcher may be interested in more interactive procedures, such as measuring reaction times, choosing objects on screen, pattern recognitions, implicit association tests, and manipulating objects. All these can and have been done through the Internet but to do so requires more advance programming often utilizing something like Java, Flash, or Authorware. One site which Ken McGraw, Mark Tew, and I created at the University of Mississippi, (Psychexperiments) provides many examples of these types of tasks and studies (McGraw, Tew, & Williams, 2000a).
The Internet based psychology laboratory, PsychExperiments, has been using Macromedia's Authorware since its inception in 1998. Experiments in social psychology, experimental psychology, and cognitive psychology have been conducted over the last nine years.
When Authorware is combined with a web server, html, and a database the development of a full featured web application is quickly available. PsychExperiments uses Authorware programs as the front end for user interactions. The data is then transmitted to web server based scripts and placed into a database. Data can be just as easily extracted from the database and either displayed on a web page or within Authorware. Authorware is deliverable through any web server.
PsychExperiments is set up primarily for in-class lab demonstrations. This includes student participation in research concerning common psychological phenomena, often used in lab courses. The downloading and analyses of class data are also components available to instructors. Typical lab exercises include facial recognition tasks, implicit association tests, Stroop experiments, Ponzo illusion, pitch memory test, numerical memory, the Mueller-Lyer, mirror drawing, and mental rotation tasks. Although class use is the main focus of PsychExperiments, it also offers assistance and placement of research experiments programmed in Authorware. Experiments such as perception of gender, person recognition, semantic differential studies, and Stroop experiments have been programmed in Authorware and run from the PsychExperiments Internet laboratory.
Reaction Time Measurements
In order to determine the accuracy of response timing from within Authorware, the programming tool used at PsychExperiments, McGraw and Tew (2002) conducted a series of experiments. They examined the ability of Authorware to accurately detect keystrokes at 150, 200, 250, and 1000 ms intervals. Performing these measurements under several conditions, they found that for the 150, 200, and 250 ms intervals – with the exception of one extreme condition which involved running an additional CPU intensive program – Authorware's measurements were within plus or minus one millisecond of the target intervals on 36 out of 36 sets of measurements. For the 1000 ms interval, Authorware was within one millisecond on 29 out of 36 sets of measurements. (McGraw & Tew).
Authorware runs on different Windows platforms and there is some suggestion from the data mentioned by McGraw and Tew (2002) that Windows platforms based on the NT kernel (Windows 2000, Windows XP) may offer better timing.
Visual illusions such as the Stroop, Line Motion, Mueller-Lyer, Poggendorff, and Ponzo are all possible through the Internet and readily presented within the Authorware programming environment. The ability to layer graphics such as in the Poggendorff illusion where an obstructing graphic is imposed between two segments of a moveable diagonal line is a built in feature of Authorware. Layering of different graphics, ability for a user to manipulate graphics by moving or adjusting the length, the ability to animate graphics along paths or within certain areas, and finally the ability to import detailed display graphics created in external programs provide the necessary tools for many different visual effects within an experiment.
Matching to Sample
Studies involving human learning and utilizing a matching-to-sample procedure are easily implemented within Authorware. An example of an exercise of this type, titled "Learning and Memory," can be viewed at Psychexperiments.
Conducting research through the Internet requires the researcher to have an understanding of differences in ethical treatment. Online studies may require more attention to issues such as debriefing, deception, and data security. While a full discussion of these issues is beyond the scope of this article, there are steps a researcher can take. Debriefing, if absolutely required for a particular study, may have to be conducted in a lab setting. Otherwise, debriefing information can easily be provided for participants to view online, though with no guarantee that it will be viewed. Data security is not unique to Internet-based research, but a researcher should be aware of how the data collected is stored, transmitted, and retrieved. If particularly sensitive data is being collected, steps will need to be taken to ensure limited access to the information. For example, a researcher may decide to physically remove the data from the server after each study session or every day.
Conducting Internet-based research is readily achievable to most psychologists. Most academic institutions will have the resources available to faculty to host and serve survey type studies. Additional expertise may be available as well to help with more interactive programming. For those researchers not associated with an academic institution, there are plenty of low-cost web hosting companies which provide database and programming access.
McGraw, K. O., & Tew, M. D. (2002). The accuracy of response timing by Authorware programs. Unpublished manuscript, The University of Mississippi, Oxford.
McGraw, K. O., Tew, M. D., & Williams, J. E. (2000a). PsychExps: An online psychology laboratory. In M. Brinbaum (Ed.), Psychological experiments on the Internet (pp. 219-233). Orlando, FL: Academic Press.
About the Author
John Eustis Williams, Assistant Professor of Psychology, University of Northern Iowa, received his undergraduate degree in Psychology from Tulane University in 1991, his master's in clinical psychology from Western Carolina University in 1997, and his Ph.D. in clinical psychology from the University of Mississippi in 2002. He has served as the Internet Editor of The Society for the Teaching of Psychology, Division 2 of APA and on the Web Advising Committee for Society for Personality and Social Psychology. He has also been instrumental in the development of PsychExperiments and several other websites.