Rival scientists are teaming up to break scientific stalemates

The year was 2018, and Cornell University psychologists Wendy Williams, PhD, and Stephen Ceci, PhD, had been volleying papers back and forth with Boston University’s economists Shulamit Kahn and Donna Ginther over bias against women in science for what felt like forever. Williams and Ceci kept publishing data suggesting that science, while once unfriendly to women, was now largely fair; Kahn and Ginther kept finding evidence of explicit sexism.

This could have continued, with Kahn and Ginther (the latter is now at the University of Kansas) pressing on with their point of view while Williams and Ceci argued theirs. But instead, the stalemate triggered something different: Williams and Ceci reached out to Kahn and Ginther, asking if they would join them in an adversarial collaboration.

It was a rare request. Adversarial research collaborations are projects in which two (or more) teams with opposing theories, hypotheses, or interpretations of evidence get together to hammer out a mutually acceptable research study that will—ideally—reconcile their differences. Though adversarial collaborations were first developed decades ago, they’re still fairly uncommon, often stymied by the extra time, effort, and interpersonal savvy needed to pull them off.

But advocates say these projects are not as scary as they seem. Dueling researchers often end up friendlier than they were when they started, and they typically resolve hot-button scientific controversies much faster than by ping-ponging combative journal articles at each other for years or decades. Williams, Ceci, and Kahn ended up needing nearly 5 years to tackle the question of sexism in science (Ginther had to leave the project because of a job change), but what they published was, Ceci said, “the most accurate, fairest description of what the scientific evidence really shows.”

The teams’ findings revealed sexism against women in some areas, such as salary disparities, but not in others, such as grant funding and hiring ( Psychological Science in the Public Interest , Vol. 24, No. 1, 2023 opens in new window). What was exciting, Ceci said, was how these results were received.

“Both [teams] had worked on this issue for decades, and every time either side published something, there was always pushback. If you looked on social media, you could get dogpiled by people on the other side,” he said. “When we did the adversarial collaboration, there was none of that. It was really refreshing.”

Given that psychologists study many issues of widespread public and political interest, these collaborations could be a tool to increase trust in psychological science across partisan lines, advocates say. They may also be key to breaking impasses over findings with broad societal implications.

Adversarial collaborations are an idea first used by psychologists Gary Latham, Miriam Erez, and Edwin Locke to resolve a 1980s dispute about goal-setting ( Journal of Applied Psychology , Vol. 73, No. 4, 1988 ) and then independently developed and popularized by Nobel Prize–winning behavioral psychologist Daniel Kahneman in the late 1990s. Kahneman conducted three adversarial collaborations between 1998 and 2003, all of which ended, he wrote, with “new facts accepted by all, narrowed differences of opinion, and considerable mutual respect” ( American Psychologist , Vol. 58, No. 9, 2003 ). He argued that the model could bring science closer to the ideal of a cumulative social project.

Today, though, adversarial collaborations are still far from the norm. One reason, said Cory Clark, PhD, a behavioral scientist at the University of Pennsylvania and director of the Adversarial Collaboration Project, is simply that they’re hard and time-­consuming. Researchers must work with a rival with differing ideas on how to set up experiments, what measures to use, and how to analyze data. Ideally, they will agree on what evidence is needed to convince them that their own position is wrong. Sometimes, an arbiter is necessary to iron out disputes.

Another problem has to do with incentives. Though any science project is ideally a search for truth, Clark said, the realities of scientific publishing often prioritize flashy new findings, and scientific careers often benefit from championing a pet theory or intervention.

“We have a culture of tolerance for scholars publishing information that contradicts other experts,” Clark said. “We’re totally fine with two, three, four Ivy League professors out there writing books and saying things that directly oppose one another, and we never ask them to explain what’s going on and to take their critics seriously.”

The Adversarial Collaboration Project aims to support these projects and help researchers tackle them successfully. This includes providing grants, helping researchers from opposite sides of issues connect, and popularizing best practices, such as preregistration and effective moderation.

There are, however, increasing glimmers of interest in adversarial collaborations, including Clark’s Adversarial Collaboration Project, which launched in 2021. The APA journal Decision recently added registered adversarial collaborations to the type of manuscripts it will accept, according to incoming editor Tim Rakow, PhD, a reader in psychology at King’s College London. Authors committing to an adversarial collaboration can receive an in-principle acceptance for publication in Decision on the basis of a strong study proposal before the data are collected, Rakow said.

Other journals that have solicited adversarial collaborations include the Journal of Experimental Social Psychology, Theory and Society, and the Journal of Applied Research in Memory and Cognition.

Psychology pioneered adversarial collaborations and remains the leader in conducting them, but other fields are showing interest, too. The journal Public Budgeting & Finance recently issued a call for adversarial collaboration–based papersopens in new window for a special issue in 2026 or 2027. The business journal Academy of Management Perspectives also announced plans to overhaul its “exchanges” format for debating hot-button issues into adversarial collaborations dubbed “Constructive Confrontations.”opens in new window Clark said she has also been speaking with researchers in nutrition science who are interested in pulling in the adversarial collaboration approach.

These collaborations may also help increase trust in psychological science at a time when politicians are targeting research in equity, diversity, and inclusion as well as other topics of bipartisan disagreement (see January/February 2024 Monitor).

“Although it’s hard to imagine assuaging some of these people who are pretty far out with their positions, for the most part, I think an adversarial collaboration assuages people who instead of feeling like, ‘My side lost,’ feel like, ‘Okay, my side agreed that this was accurate, and who am I to turn on them now?’” Ceci said.

There are advantages in using adversarial collaborations for less partisan issues, as well. Methodological issues that could stymie a paper in review tend to get worked out in the planning phases of an adversarial collaboration, Rakow said, since authors are prone to challenge one another on their decisions. Challenges can also bring clarity, said Cameron Martel, a doctoral candidate at the MIT Sloan School of Management who was lead author of an adversarial collaboration about whether accuracy prompts work as well to stop conservatives from sharing misinformation as they do for stopping liberals ( Psychological Science , Vol. 35, No. 4, 2024 opens in new window).

“It made all of us think about exactly what hypotheses we wanted to test and think through exactly what our positions were, what predictions we had, and how to best evaluate them,” Martel said.

Working with adversaries can also force authors to truly understand the opposite side’s position and to face the limitations of their own claims, Rakow said. Some researchers also say they learn more about how their methods differ from those of their adversaries, details that wouldn’t be apparent from publications alone.

This process can be difficult, and interpersonal conflict is one of the biggest fears that keeps people from launching an adversarial collaboration, Clark said. Some collaborations do fall apart. The majority succeed, however, Clark said. “In my experience, relationships tend to get better rather than worse.” Rival academics come to see each other as working in good faith, and even if they don’t ultimately agree, they often finish a project respecting each other more than they did when they began.

That was the case for Sander van der Linden, PhD, a social psychologist at the University of Cambridge who was on the opposite side of Martel in the adversarial collaboration on accuracy prompts and misinformation. A good deal of friction accompanied that collaboration, both teams agree. Without an arbiter (Clark), the project probably would have fallen apart, said Gordon Pennycook, PhD, a Cornell University psychologist and one of the authors on the study. One key role the arbiter played, the researchers said, was pushing the groups to come to a compromise rather than get stuck at an impasse when disagreements arose.

There was a lot of back-and-forth between the groups about methods and analyses, van der Linden said, and even at the end of the project, the two teams didn’t entirely agree on the meaning of their findings. Nevertheless, he said, the joint findings caused him to update his views. And though the groups didn’t entirely see eye to eye, they’ve developed a more collegial relationship since the project, sharing their papers before posting preprints online and giving one another constructive feedback so that their debates don’t sprawl into the public square prematurely.

“We joke that what we learned from this project is we all need to get together again and write a paper about something we aren’t in conflict about,” van der Linden said. “Or maybe even have a movie night or start a band or something and then attempt to try to do this kind of science again.”

An arbiter can be invaluable for getting through an adversarial collaboration when passions are high, said Barbara Mellers, PhD, a psychologist at the University of Pennsylvania who acted as an arbiter in two of Kahneman’s adversarial collaborations. In one study, Kahneman and his coauthor, German psychologist Ralph Hertwig, got so irritated with one another that they’d only speak through Mellers, she said. Nevertheless, they succeeded in publishing a paper and eventually became friendly.

“People are complex, and the process is going to get difficult,” Mellers said. She recommended that arbiters document as much as possible, keep organized, and focus on getting parties to move forward when they hit an impasse. It can often be helpful to choose an arbiter with an interest in open science principles, such as data transparency, Clark said. In one project she worked on, the arbiter blinded the data and ran the analyses herself to ensure neutrality. Arbiters often write large portions of the final paper and are listed as authors.

Adversaries working together need to stay open-minded and humble and be willing to be proven wrong, Pennycook said. For principal investigators who are used to full control in their own labs, this also entails taking a more team-based approach and leaving ego aside. “You don’t get to control it,” he said. “You have to be a member of a group, genuinely.”

Many academics fear that if their findings aren’t as solid as they’d hoped, their value will decline, but that’s not true, Pennycook said. “We have this model in academia where if you are the person who discovered an effect, your value is tied to that effect,” he said. “But actually, your value is tied to the quality of the research that detects whether an effect is there or not … If I see somebody engage in an adversarial collaboration, my respect for them goes up immediately.”

Almost any field could benefit from adversarial collaborations, Clark said, on everything from highly contentious topics like the effect of social media on teens to more basic science like theoretical views of working memory. (And psychology is not unique in benefiting from this method, she added, noting that scholars in many disciplines have approached her: “Apparently there are a lot of really heated debates in the butterfly world.”)

Journals and funders could do more to encourage adversarial collaborations, Ceci said. Journals could commission more adversarial papers or offer guaranteed acceptance to preregistered adversarial collaborations. Funders could offer bonuses on grants for principal investigators willing to team up with rivals, funding multi-lab efforts to do adversarial work.

“We should incentivize this,” Ceci said. “We should say, ‘It’s really great that you refuted your decade-long position; that’s admirable.’ We should reward people for being willing to risk their long-established conclusions.”