Why experts often disagree

Whether experts agree depends in large part on the complexity of the question.

By Beth Azar
Monitor staff

People like to joke that an "expert" is someone who knows more and more about less and less. For a given question, it seems, you can find an expert to support any argument: Global warming is a problem, or maybe not. Storing nuclear waste in Yucca Mountain is safe, or maybe not. Pesticides on your food will cause brain damage, or maybe not.

Psychologists Wendy Williams, PhD, and Stephen Ceci, PhD, took advantage of experts' tendency to disagree in their recent book, "The Advice Trap" (Andrews McMeel Publishing, 1998) when they asked several relationship experts to proffer advice on a number of common relationship problems. The result? Conflicting counsel. The reason? There's no one answer to such complex questions, Williams and Ceci conclude.

In fact, we often expect too much by way of consensus from experts, says judgment and decision researcher James Shanteau, PhD, professor of psychology at Kansas State University.

Most complex problems have no single answer, he says. It's only when the problem is simple or the route to an answer is well-structured that experts in that area will agree a large percentage of the time, he says.

Shanteau places decision-makers into four categories based on the nature of the problems they try to solve:

* "Aided decision-makers," including weather forecasters, astronomers and insurance analysts, make their decisions based on data provided by complex computer models. Because computers do much of the work, people in these jobs tend to agree quite often--their answers correlate at rates that range from around .70 and .95, with 1.0 being perfect agreement.

* "Competent decision-makers," such as chess masters, grain inspectors and livestock judges, have rules and expertise that allow them to make competent decisions based on the rules. As they gain in experience, they agree more often, correlating at rates that range from around .50 and .60.

* "Restricted decision-makers," for example, clinical psychologists, parole officers and student admissions officers, make decisions in the face of incredible uncertainty and complexity. As a result, they agree at far lower rates--with decisions correlating at around .40.

* "Quasi-random decision-makers," including astrologers, polygraph readers and stock brokers, have few hard and fast rules to go by and operate at levels close to chance, with little agreement between individuals examining the same data.

Many experts--nurses, physicians, auditors and most researchers, to name a few--work at all four levels depending on their task, and their rate of consensus varies in relation to the question they're answering, says Shanteau.

Psychologists can predict when experts will disagree by analyzing the nature of the task an expert is tackling.

"When you have random, highly variable stimuli, it will be hard to give anything but random responses," says Shanteau. "While in domains where there is some structure to the decision-making process--such as swine judging or chess--consensus should get better as people gain experience."

Read our privacy statement and Terms of Use

Cover Page for this Issue

PsychNET®
© 1999 American Psychological Association

APA Home Page . Search . Site Map