Feature

Internet-based tests work well when held to the same psychometric standards of reliability and validity as any other type of examination, says a recent report from APA's Task Force for Psychological Testing on the Internet. However, that's a difficult task given some of the challenges posed by the freedom of the Web, the report notes.

APA's Board of Scientific Affairs and Board of Professional Affairs established the task force in 2001 to inform psychologists about the state of Internet testing. Its report underscores that the common standard for ethical use of all psychological tests applies in the online world.

"We think that Internet testing doesn't really change those basic principles of reliability and validity," says Fritz Drasgow, PhD, of the University of Illinois at Urbana-Champaign, who co-chaired the task force with Jack Naglieri, PhD, of George Mason University in Virginia.

The principles Drasgow cites have been set forth by the Joint Committee on Psychological Tests and Assessment--made up of representatives of the American Educational Research Association, the National Council on Measurement in Education and APA--in its Standards for Education and Psychological Testing, revised in 1999 and available online.

The standards outline, for example, how to justify a particular use of a test, such as a standardized high-stakes test. They also identify what populations can validly take different kinds of tests and what safeguards make them fair and reliable. An example is ensuring age-appropriate IQ testing.

Drawbacks of Internet testing

Problems emerge online when tests don't conform with those standards, according to the task force report, which will be published in the April issue of the American Psychologist (Vol. 59, No. 4). For example, some tests on the Internet, such as popular IQ tests, offer scores based on a small set of poorly explained questions.

"The problem is there's much less regulation on the Internet than you would have with paper-and-pencil tests," says Naglieri. "Some of the programmers have terrific skills to put IQ tests on the Web but they don't have a sense of the implications of the results. They aren't equipped to deal with the ethical responsibilities they're taking on either."

Naglieri provides a case in point: A parent might have a child take an IQ test online, and, due to a fault or inconsistency with the test, the child's score might turn out disturbingly low and fail to give the parent an explanation of the results. "How devastating could that be?" he says. "It could cause the parent to lose confidence in the child, and the child to lose confidence in themself."

For psychologists, Internet testing also brings other problems--not new to the testing pursuit, but different because of the medium, Drasgow says. For one, ethical considerations require that psychologists know something about the people using their assessment tools. But if the test is on the Internet, psychologists can glean little reliable information about users. As long as people are honest and well-meaning, the information may be fine; the problem is that researchers have no way to know for sure, Drasgow notes.

Aside from confirming identity, other challenges associated with Internet testing include:

  • Guaranteeing the security of test results.

  • Preventing unauthorized use of testing materials.

  • Maintaining copyrights across international borders.

  • Providing sufficient explanation of test results.

  • Accommodating test-takers' special needs.

Another problem is lack of education about testing ethics and protocol among untrained businesspeople who post tests on the Internet for profit. The task force hopes to raise awareness of those issues among that group as well, but admit that they will be hard to reach through the usual means--professional associations and scientific journals, Naglieri says.

"Some of the people who have Internet testing sites are businessmen and women who don't have the same professional obligations as APA members," says Naglieri. "It's going to be very difficult to suggest that these people need to be aware of the ethical issue, for example, because they don't have to play by our rules."

Benefits of online tests

Problems with for-profit Internet tests notwithstanding, there are some functional, fair and reliable tests online, Naglieri says.

For example, some employers who hire many entry-level workers, such as retailers, use Internet tests effectively as part of their online application process. The tests give employers an idea of applicants' cognitive abilities and other job-related skills, which limits time wasted by both employers and applicants by eliminating in-person interviews with very unqualified applicants. Even if an applicant cheats, employers face little risk because, after establishing the dishonesty with an in-person test, they can avoid hiring that applicant.

Well-designed tests can offer benefits like these at a low cost, Naglieri says. Other potential benefits outlined in the report include:

  • Worldwide use. A new test can be made available around the world almost instantly.

  • Paperless updates. Updating tests and circulating them widely can happen without reprinting paper copies.

  • Quick results. Software that computes test scores and generates interpretive reports can be run immediately following the test, bypassing long return times common to paper-and-pencil tests.

  • A wealth of information. Low costs allow tests to be taken for free by large numbers of people, providing a mechanism for rapid information for researchers, employers or testing agencies.

  • Consistency. Test materials are easily presented in a consistent, uniform manner allowing test-takers to become familiar with the procedures and test location, which can limit environmental stress such as not understanding the computer interface.

  • Appeal. Information can be presented in a precise and interesting manner, so the test-takers' attention to the task is enhanced.

In addition to bolstering understanding of the potential benefits and drawbacks of Internet testing, Naglieri says, it's also important to keep the public aware of the difference between testing available on the Internet and the assessment services psychologists working directly with test-takers can provide.

"What's done on the Internet is just testing," he says. "You take a test, you get a score, and that's where it stops. There is no professional to tell you what the test score means and relate the score to your unique characteristics. This is the role of the psychologist. Internet testing is not the same as an assessment conducted by a qualified professional. You can't take the psychologist out of assessment."

Further Reading

Read the task force report in April on the Science Directorate Web Site.