Transcript
SP: Hi everyone, I’m Steph Pollock, the Associate Publisher of community initiatives for APA Journals. APA Publishing's vision is that openness and transparency promote not only a more rigorous science but also a more equitable one. This video is one of the series of conversations with APA journals editors about how they’re working to promote equity through open science.
Today I’ll be talking to Keith Yeates, editor of Neuropsychology, about some new initiatives in his journal. Great to have you, Keith. Can you tell us a bit about the type of content published in Neuropsychology?
KY: Neuropsychology publishes articles, systematic reviews, research really focused on broadly brain-behavior relationships: How does the brain relate to behavior both in healthy individuals as well as in clinical populations?
SP: Great, thanks. You’ve enacted several new initiatives in support of equity, diversity, and inclusion (EDI), and open science. Can you tell us more about what you're working on and what this means for authors?
KY: I think openness and equity and diversity go hand in hand as we try to make our science more transparent and open. I think we’ll inevitably also try to make it more diverse and equitable and vice versa—the two efforts reinforce each other. On the open science side, we’ve had open science badges for a few years. We’ve started using the CRediTopens in new window author contribution statements as a way of letting people really say what they’ve done in the science rather than the old you’re either “first, last, or in the middle” way with authorship. Along with APA of course we’ve began to adopt the TOP (Transparency and Openness Promotion) Guidelines, for transparency and openness.
Along with these efforts, we are also doing several things to try to improve the diversity of our science at multiple levels—what we study, how we study it, who we study, who does the science, and who judges the science.
We've really begun to encourage more diversity in our content. More work for example in specific historically excluded groups. More attention to things like brain health disparities. More willingness to look at nontraditional methods (like community-based sorts of methods). So really kind of changing how we do the science and what we do the science on.
We're also trying to change the reporting standards for the journal. Some of that goes along with openness and transparency but a lot of it also has to do with diversity so that we are really encouraging people to look beyond the traditional ‘WEIRD’ samples—the western, educated, industrialized, predominantly white samples that have characterized most of our science—to emphasize that papers ought to be very clear about who they’re studying and why they’re studying that population.
And one of the reasons we’re talking today is that we’re encouraging the use of Constraints on Generality (COG) statements as another way of promoting more openness but also I think ultimately promoting more equity in our science as well.
SP: That’s great. I think your approach is a great example of a holistic approach to engaging in a more equitable and inclusive science, so that’s great to hear. Let’s talk a little more about the COG statements, the constraints on generality statements. Can you tell us more about them? Specifically, what are they and how do prospective authors use them?
KY: Constraints on Generality statements were first described by Simons, Shoda, and Lindsay in an article in Perspectives on Psychological Scienceopens in new window. A COG statement, in essence, is a way for authors to say, “here are the reasons we studied who we studied, here are where our findings apply or to who they apply, and where it might not be applicable to other populations or where we need to broaden the scope of our samples and populations to say whether or not a set of findings applies”.
It also applies to methods and things like historical context. If you know that you studied a group, say, just after the hurricane in New Orleans, you’re going to have a different effect than if you studied a group who wasn’t in New Orleans, and you happened to be studying stress.
So Constraints on Generality are formal ways that authors can say here’s why we studied the population that we did, here’s the limits to the population, and that our findings may or may not generalized to other populations, here’s the methods we used and they may have limits too in their applicability.
For example, we know lots of neuropsychological tests were developed in western countries only normed in English. We may not know if our tests are valid measures of cognitive constructs or underlying brain function in populations that are not from western countries that don’t use English, just as one example.
So, it’s about the populations and the methods, and letting authors be more circumspect about making claims about their findings.
We know that psychology—but not just psychology—has suffered from a bit of a replication crisis. Part of the reason for that I think is that we’ve been in a position where we tend to assume our findings are applicable to other populations even if we’ve used other methods, even if we’ve changed our experimental paradigms, and I think that’s where we need to be more honest in effect, or maybe a little bit more humble about our science and how general we can say our findings are.
SP: That’s great. So obviously COG statements are a great tool for promoting a more reproducible final product. But how do COG statements also feed into the equity piece? I know you spoke to this a bit about sort of bringing that humility and questioning about the general approaches to COG statements but how do they feed into your vision of a more equitable science?
KY: I think they have both direct and indirect effects. So obviously, directly it’s a way of saying our findings are only applicable to this population. But what it does, I think, is it makes us have to admit that in many cases our science is limited to predominantly white, predominantly well educated, predominantly western samples. I think that will help push us to realize that we need to do better than that—we need to try to have samples that are more representative of whatever population we’re working in.
For Neuropsychology as a journal, because it gets a lot of submissions from North America, there’s sometimes I think a tendency to think that if we do science in the United States or we do science in Canada that it is applicable even to other western countries, much less to the rest of the world. We need to be more humble and recognize that what is true of individuals in North America may not be true in individuals in the southern hemisphere or Western Europe. And we need to say that because it will encourage submissions from other parts of the world where scientists have access to populations that we may not necessarily have access to, where we can begin to ask the question of “is this true of other populations, or do we need to take a different perspective? Where are the boundaries of our data and our findings?” So, I think it in the long run will help us have to admit with some humility that we are subject to whatever populations we can access and who agrees to be in our research.
We already know, for example, in many contexts that individuals with lower socioeconomic status tend to be less likely to stay in longitudinal studies, for example. So, we need to be honest about that. We need to be forthright about, we need do better, we need to find ways to broaden our samples, to check whether the findings we’re saying are universal—a word I’ve had to strike from several journal articles—because we do have a tendency to want to make big claims, it’s what gets the attention, it’s sometimes what gets the big citations that scientists want to see for their work.
I think Constraints on Generality statements are a way of checking our ego a little bit and in so doing really highlighting where we need to be more equitable in the science we are doing.
SP: Great. Thank you so much for sharing your vision and for joining us today.
KY: Thanks for the invitation, I enjoyed it.
Keith Yeates, PhD, professor and head of the psychology department at the University of Calgary as well as editor-in-chief of Neuropsychology, discusses Constraints on Generality (COG) statements and the ways in which they encourage studying samples that are representative of diverse populations.