Doing Research into Research Methods
Two people answer two different questionnaires assessing depression. Both say “yes” to half the questions. Are they equally depressed? That depends, says Michael C. Edwards, PhD. “Saying those two scores are the same makes a whole bunch of assumptions,” he says. “What if one questionnaire measures really severe symptoms and the other measures really mild symptoms? This raises an even bigger question: What are we saying when we just add up how many times someone says ‘yes’ and ignore what they are saying ‘yes’ to?”
That’s the kind of problem Edwards studies. He’s an associate professor in quantitative psychology at The Ohio State University (OSU), where he teaches and does research involving measurement issues in the social sciences. In other words, he studies the statistical methods psychologists use in their research, with a special emphasis on the methods used to study constructs we can’t directly observe. “I work on trying to make sure we’re measuring what we think we are and getting good results as efficiently as possible,” he says. He specializes in methods called item response theory and factor analysis. These are techniques that help researchers study things that they can’t directly observe – such as mathematical ability, intelligence or depression.
“We have a group of symptoms that we use to define something like anxiety disorder,” he explains. “There’s no blood test to determine if you have an anxiety disorder.” But there are questionnaires that can help a researcher or mental health practitioner gauge whether a person has an anxiety disorder and, if so, how severe it is. He also works on something called adaptive testing, which is a way of modifying tests based on how someone is responding to the questions. For example, he says, “Imagine you are taking a questionnaire measuring anxiety. I ask you an average severity question and you say ‘yes.’ I ask you another average severity question and you say ‘yes.’ That means I don’t have to ask low severity questions. If you’re saying yes to average severity questions, I know you would probably say yes to lower severity questions.” This kind of test adaptation is increasingly important because, he says, “our attention span for surveys is dropping.”







