“Philosophical work would be better with adversaries collaborating to keep each other honest.”
Why is that? What would it look like?
In the following guest post, members of the Mind and Morality Lab at Boston University—Victor Kumar (Boston University), Derek Anderson (Boston University), Emilia Lacy (Boston University), Josh May (University of Alabama, Birmingham), Rachel McKinney (Suffolk University), Tino Themelis (Boston University), and Velana Valdez (Boston University)—explain adversarial collaboration and offer answers to those questions.
Adversarial Collaboration in Philosophy
by Victor Kumar, Derek Anderson, Emilia Lacy, Josh May, Rachel McKinney, Tino Themelis, and Velana Valdez
Scientists, being only human, often collect and analyze data in ways that unintentionally skew the results in their favor. Thankfully, this occupational disease has a known cure.
In adversarial collaboration, researchers with conflicting hypotheses lay down their arms, joining forces to work out a study design and statistical criteria—not eliminating confirmation biases, just neutralizing them in the aggregate. Then they run the study and accept the results, come what may.
As philosophers, we applaud this development. It’s about time, really. We’ve been practicing adversarial collaboration ourselves since circa 387 BC, haven’t we?
Not exactly. Adversarial collaboration isn’t just the Socratic method. It’s a commitment to abide by prespecified evidence. And how could we agree in advance about what would settle philosophical disputes? We don’t run experiments, nor do we have any data to crunch. (Mostly.) You can’t precommit to a thought experiment: by the time you pose one, the results are already in.
The idea seems like a non-starter, but in fact, adversarial collaboration can be adapted for philosophy. The first step is to isolate its true functions in science.
The Method
The late Nobel laureate Daniel Kahneman argued that scientific disagreement was too combative. Scientists rarely change their minds. So in the late 1990s, he thought: if you can’t beat ’em, join ’em. (And he did. Many times.)
Kahneman’s final adversarial collaboration was on income and happiness. Previously, he’d found that happiness rises with income but only until about $75,000. Another researcher, Matthew Killingsworth, found no plateau at all. Instead of trading criticisms, they decided to analyze the data together. What they found is that the plateau is real but only for the least happy fifth of people: unless you’re miserable and already somewhat wealthy, more money does mean more happiness.
Adversarial collaboration of this kind makes sense if, as Hugo Mercier and Dan Sperber argue, a primary function of reasoning is to justify yourself to others. We’re biased when we produce arguments but much sharper when we evaluate someone else’s. So if you’re not in close dialogue with people who disagree, you risk fooling yourself about how much the evidence supports your position.
Decades after Kahneman pioneered adversarial collaboration, the practice was renewed by Cory Clark and Philip Tetlock, who saw it as a solution to science’s methodological woes.
The replication crisis in science has been driven by so-called Questionable Research Practices like “p-hacking.” Classically: run a study, peek at the results periodically, and end data collection once you hit statistical significance. Other QRPs include generating post-hoc hypotheses to suit the data; choosing statistical criteria that best support your hypothesis; selectively reporting only significant results; and removing outliers.
Open Science can address some QRPs. For example: preregistering a hypothesis before running a study that tests it. But adversarial collaboration is another effective solution. A research team won’t be motivated to skew the results in one direction if each side has opposing priors. What’s more, Clark and Tetlock’s team argued, adversarial collaboration can also address other methodological problems that Open Science leaves untouched, such as when scientists “operationalize theoretical constructs in ways that load the dice.”
Clark and Tetlock didn’t just argue for adversarial collaboration. They institutionalized it. The Adversarial Collaboration Project at Penn funded studies, fostered multilab collaborations, created dedicated special issues in journals, and carried out a meta-study on the outcomes of completed projects. And unlike Kahneman, Clark and Tetlock saw how adversarial collaboration could improve scientific inquiry that bears on politically contentious issues.
Political Research
What are the effects of affirmative action? The risks of gender-affirming care? The benefits of neurodiversity? It’s natural and perhaps even vital for moral and political commitments to guide research on these topics. In their absence, important topics would be neglected or misunderstood.
Problems arise when these commitments lead to dogmatism. Politically motivated scientists are susceptible to confirmation bias—reinforced by like-minded intellectual communities—and more likely to fall into QRPs. Yet as standpoint theorists themselves often suggest, intersubjective coordination across standpoints can increase objectivity. This is what adversarial collaboration brings to the table.
Through such collaboration, research on politically contentious questions can provide answers that are not only more objective but also more trustworthy. Policymakers, journalists, and the general public often don’t trust political research because the scientists behind it are (or appear to be) biased, especially as science has grown more politicized. Neutralizing those biases can restore trust in science.
Gender biases in academia are a case in point. When do feminist researchers have special insights into the ways that female academics are professionally disadvantaged? And when do they see illusory patterns that confirm their preconceptions? A group of researchers with conflicting hypotheses and diverging political orientations decided to find out by running an adversarial meta-analysis.
Gender bias is difficult to study because of confounding variables. For example, male and female job candidates differ in many ways beyond their gender. One study finds that women in philosophy are 10-17 percentage points more likely than men to land a permanent job, but that might be just because they have to be more talented and determined to push through all the sexism in the field. To really test the idea, you have to match candidates on overall quality.
That’s exactly what the adversaries did. They found no biases in grant funding, journal acceptances, and recommendation letters. Women do get worse student evaluations and are paid less. But even when matched on quality, female candidates are more likely to be hired.
Women in academia experience other disadvantages, such as sexual harassment, hostile climates, and obstacles related to pregnancy and childcare. Yet adversarial collaboration enabled the researchers to uncover the biases their method could detect—and recognize what it couldn’t.
If only this method could be exported.
Why Export It to Philosophy?
If Kahneman thought scientists are too combative, imagine what he must have thought about philosophers. We’re regularly in “soldier mindset” rather than “scout mindset,” defending our positions rather than mapping the terrain. We often act like “lawyers for a view,” especially for political topics. Empirical research (by one of us and others) also suggests that philosophical views are often shaped by personality and other psychological traits. All of this helps explain why philosophical dialogue rarely changes minds.
And like science, philosophy has its own QRPs. Philosophers cherry-pick thought experiments that support their theories, leaving out the variants that don’t; bite the bullet selectively; and treat opponents uncharitably, ignoring the strongest version of their arguments. So the conditions that call for adversarial collaboration in science also exist in philosophy. Philosophical work would be better with adversaries collaborating to keep each other honest.
There’s another reason to export adversarial collaboration to philosophy. When philosophers write about politically contentious topics—gender identity, policing, AI—it’s reasonable for the public to discount their conclusions as motivated. Philosophical work that comes from both sides would be not only higher quality but more trustworthy.
Some philosophers have answered the call—but so far mostly just in subfields that involve collecting or closely analyzing empirical data. Opponents have joined forces in experimental philosophy and some areas of philosophy of mind. Christoff Koch and David Chalmers engaged in “adversarial forecasting” about the science of consciousness. Why not in more abstract areas of philosophy? Why not in ethics or political philosophy?
It’s hard to see how this could work in practice. Generally, philosophers analyze concepts, run thought experiments, explore logical implications, and synthesize evidence. We don’t know what evidence we’re seeking until we already have it, so we can’t decide how to approach the evidence in advance. Moreover, it’s routine for unanticipated evidence to arise in the course of inquiry.
The challenge runs deeper. Tetlock and Mitchell argue that adversarial collaboration is hardest under certain conditions: unclear falsification criteria, methodological disagreement, and background intellectual or political tribalism. All of these conditions hold in philosophy.
And yet, adversarial collaboration is viable in philosophy, once you see how it actually works in science.
From Science to Philosophy
In theory, adversarial collaboration can resolve scientific disagreement. In practice, though, it’s rare for one side to be vindicated; more often, the research uncovers middle ground (as in Kahneman’s final adversarial collaboration on income and happiness). In part, that’s because research can settle the results without settling the correct interpretation of those results.
So what’s the purpose of adversarial collaboration in science? To answer that question, best to go straight to the sources.
Here’s Kahneman and collaborators:
The authors did not agree when they began the collaboration, nor do they agree now. But [such] agreement is not necessary for a successful adversarial collaboration. Success should be gauged by the extent to which joint efforts yield surprising results, insightful discussions, and testable hypotheses about outstanding issues.
And here’s Isch, Tetlock, and Clark:
Rather than producing a clear “winner,” the most common outcome was a deeper understanding of the problem space through the integration of opposing perspectives.
This is precisely what good philosophy does. Decisive resolutions are rare in philosophy, but they’re rare in scientific collaborations too. Rather, adversarial dialogue can clarify disagreement, refine arguments, identify presuppositions, disambiguate terms, and map logical space. Sounds familiar.
But wait. Philosophers already engage in adversarial dialogue: back-and-forth in journals, Q&As during colloquia, author-meets-critics sessions at conferences, journal refereeing, feedback on draft papers, and informal discussion at the bar. What would distinguish adversarial collaboration from existing practices in philosophy? Now that we have a clearer picture of the practice in science as adversarial dialogue, haven’t we been doing it since circa 387 BC, after all?
How to Do It in Philosophy
Not exactly. Adversarial collaboration in philosophy isn’t just business as usual. It’s a structured method through which philosophers with genuine disagreements commit to investigating a topic together—cooperating rather than competing.
The collaborators agree to be intellectually open rather than attached to one side; calm rather than combative (avoiding “fighting words”); transparent about their assumptions; willing to revise; oriented toward truth-seeking rather than victory. These are possible in ordinary philosophical practice, of course. But it’s not enough to personally commit to these values. They have to be built in, structurally. How?
Through a philosophy lab (aka a research group).
The collaborators are then invested members of the same intellectual tribe rather than opposing tribes. They have buy-in: as members of the same lab, they’re practiced at constructive inquiry that aims to refine one another’s ideas rather than shoot them down. They’re open to being convinced and seeking compromise. As topics shift from essay to essay, collaborators’ allegiances shift naturally, and they can avoid factionalism.
This is a far cry from Reviewer 2. It would also avoid the pitfalls of Scott Alexander’s adversarial collaboration contest, where most groups bailed: lacking a prior relationship, they couldn’t successfully cooperate. Members of the same lab give more credence to conflicting opinions and countervailing arguments.
Philosophers can’t precommit to study design and statistical methods, but they can and should precommit to certain values, norms, and rules. Our own collaborative practice suggests the following possibilities for orienting a lab’s approach:
Values: trust; respect; good faith; fallibility.
Norms: Disagree without being disagreeable; attempt to give no offense and take no offense either; be constructive rather than destructive (“yes-and” rather than “no-but”); seek objections that aim to strengthen a view rather than defeat it; avoid seeing views only in terms of your commitments or identity; aim for nuance rather than a decisive verdict; avoid treating answers to questions as objects of status or esteem competition.
Rules: Roughly equal representation on all sides; commit to authorship, wherever the argument leads; land on ideas that some lab members disagree with; no edits without track changes; assign final cut to a managing editor (who may change from project to project so that no one person monopolizes authorial control).
Such values, norms, and rules support collaboration, but the work has to be adversarial too. For example, a lab might attempt to capitalize on political diversity among its members. Academia is politically much more homogeneous than the general public, yet diversity does exist. In our lab, there is real disagreement between doomers and anti-doomers; between democratic socialists and free market liberals; between those more or less sympathetic to social justice politics of the last 10-15 years; between those who are pessimistic about AI and those who are optimistic (or, er, only mildly pessimistic).
That’s the idea, but what would a philosophy lab produce?
Two Types of Essays
One type of essay is written in a single voice. A lab meeting begins with an idea or just a topic. As a thesis forms through discussion, it attracts supporters and dissenters. If there are students in the lab, they must be encouraged to voice their disagreements rather than defer to faculty. Ideally, you want a roughly even split, though what’s necessary is just a critical mass on each side, with faculty spread between them. Then open-minded discussion between adversaries yields arguments, objections, and replies. They go off and do some research and come back to discuss. Rinse and repeat.
In the end, the essay has only one voice, though it may express nuance that reflects internal difference of opinion. And it’s shaped by not just those who endorse it but those who dissent too. It’s even possible for the essay to become an expression of the lab’s “collective mind,” such that no individual endorses it entirely. No one, then, would be fully accountable. (Is that troubling? Or freeing?) Perhaps it leads to spin-off essays from dissenters—as happened after our lab’s first single-voice essay. (We’ve been using this method to write public-facing essays, but it could also be used for journal articles.)
The main strength of the single-voice model is that the essay takes a familiar form, making it straightforwardly readable. The main weakness is that the essay must reflect some sort of consensus view.
A second type of essay is written in many voices, divided into multiple sections. Debate books have been doing something like this for decades, at least when authors provide feedback on one another’s draft chapters. Examples include Walter Sinnott-Armstrong vs. William Lane Craig on atheism and Gilbert Harman vs. Judith Thomson on moral objectivity and moral relativism. What about collaborating within a single essay?
It could have a thesis section endorsed by one side, an antithesis section endorsed by the other side, and then a synthesis endorsed by both—as in our first many-voices essay. Or it begins with the main argument from one side, a dissenting opinion, and then a reply. Or it’s a philosophical dialogue, a classic form and perhaps the best way to let readers decide.
Another version of an essay with many voices mimics a format in science. A lab member who serves as a neutral arbiter writes the introduction; each side gives its argument; then the arbiter writes a final, evaluative section. (Some of us think it would be interesting to enlist an LLM as arbiter; others are skeptical.)
The many-voices model has the advantage of clearly reflecting both sides, though the essays might tend to look more like a traditional debate rather than a collaboration with synthesis. Then again, perhaps the real value of adversarial collaboration is the process rather than a legible product: fewer essays of higher quality.
Ultimately, we don’t know which model to prefer, and probably neither is uniquely best. Our goal is to continue pursuing both and see what results.
Another goal is for the practice to spread. Adversarial collaboration could even be institutionalized, with philosophy journals hosting special issues, funders incentivizing it, and skeptical philosophers publishing criticisms. But before all that, other labs can try running their own experiments. If you find something, let us know.
This is great! But why think that, in order to achieve the kinds of collaborative dynamics described here, the underlying values “have to be built in, structurally” through a formal institution such as a philosophy lab? You say that the collaborative dynamics “are possible in ordinary philosophical practice”, but then in the next line you seem to retract that idea.
I’m sure this will sound mawkish (and probably it is), but aren’t these intellectual dynamics also the exact ones you see in circles of philosophers bound by long-term friendship and love? Friendship prompts us to care about one another’s ideas for their own sake, to invest in one another’s intellectual perspectives over the long run, to put aside personal prejudices for the sake of warm engagement, and to pursue consensus across difference for the sake of connection. I would think a more primary, and more fundamental, piece of advice would be to make genuine friends with your colleagues (and stop taking yourself qua thinker so damn seriously). This is valuable for its own sake, but also, through the collaborative environment friendship creates, it will have instrumental benefits of the kind mentioned here! In my experience, at least.
I guess I’m saying that a philosophy lab, done well, might be sufficient for these collaborative dynamics, but it doesn’t seem necessary. Are you committed to the necessity claim?
Relatedly, in my anecdotal experience, relationships like this are the main conduit through which philosophical views shift. Individual tokens of philosophical dialogue rarely change minds. Certainly, some random peacocking caviler at the APA is not going to change my views on any big philosophical questions. But repeated instances of philosophical dialogue with people I care about and feel natural sympathy for often do, in the long term.
Love this, thanks.
Great points. It’s certainly too strong to say a lab structure is the ONLY way to produce the fruits of adversarial collaboration. There are other ways, and fostering the right relationships can yield similar results. But it does help to have a structure designed to foster such relationships. That’s my take at least. Maybe my collaborators disagree!
I really like this bold and original new proposal. I totally agree that it’s great to find people who disagree with you, talk to them seriously, and try to write something better together. I can’t wait until someone tries this—maybe we could even found entire departments around it. We could call them “philosophy departments.”
We made that joke already
Jokes aside, if renaming the subset of the department that works on related subjects a “lab” encourages more collaborative research, then that’s already a win (on the assumption of the post, which I endorse, that overtly collaborative research is good philosophical method).
Pablo,
I never understand the pointless snark in the comment threads here. Seriously, why bother?
Philosophy departments rarely do this, for mundane practical reasons.
1) Probably no-one in your department works in exactly the area you do. The interpretation of quantum mechanics is probably my #1 area. No-one else at Pitt does it: why? because why would we hire an expert in the interpretation of quantum mechanics when we already have one? Hirings are serving our teaching and graduate-advising needs, and it’s wasteful to double up in one area and leave another underserved.
2) Even if you do hire someone whose area overlaps with yours, probably you agree with the broad thrust of their approach. My #2 area is probably quantum field theory. My colleague Porter Williams also works in quantum field theory, but we agree in the broad strokes of how to think about it. Why? A bunch of reasons, but fundamentally you hire people you think are good, and it’s not easy to separate “good” from “agrees with me”.
3) Even if by happenstance you have a colleague who works in the same area as you and has sharply contrasting views, it’s much more likely to lead to duelling papers than to an adversarial co-authorship.
Reason 1 is, in my experience, the biggest issue. Few departments have the money to hire overlapping specialists.
#3 is encouraged, especially especially pre tenure folks, by norms that favor single authorship. I think those are fading in philosophy, but they still exist and are an obstacle. If one can get a “stronger” publication, and do it more quickly because they are not expending effort on coordination, then it’s hard to say no.
I think this is an interesting idea, and the lab idea will work probably for some topics.
But: I guess my concerns are about whether the lab idea is sufficient, rather than necessary. Here is the worry: is there any evidence that you’re more likely to just “go along” with the views of others if they’re your friends (I guess I would’ve thought so)… or… your (long time) lab mates? In the case of Kaheman’s adversarial collaborations, he’s not doing them with folks already in his lab (presumably too much agreement within the lab?)
So if two folks already disagree (because they’ve been in different labs), it might make sense to bring them together for a collaboration.
But if they’re already in the same lab, aren’t they less likely to disagree? More worrisome: Might not being in the same lab lead people to agreement… but for non-epistemic reasons?
Maybe I have the wrong idea of what the “lab” involves. I think it is good for philosophers to have long term groups that meet regularly. I’m just not sure that is the best for this kind of thing.
True, a philosophy lab isn’t always sufficient for ad collab. Perhaps we’ve been lucky having some differences of opinion within ours! Other structures and organizations can work, like the Adversarial Collaboration Project at Penn, which brings researchers and labs together. And it’s not impossible to cold call an adversary to see if they’re up for it. But it sure helps to have some social or institutional scaffolding that supports collaboration.
Otherwise you have to just wait and hope for a collaboration to germinate at the bar one night after a day of conference talks. It would be great if our post just spurred philosophers to sometimes go beyond disagreement at the conference to writing something together.
I like this idea very much. Constructive disagreement is the lifeblood of philosophy, and conversely, the echo chamber is a real danger. This isn’t the only way to encourage constructive disagreement, but it looks like one good way.
Thanks for this guest post. I’ve also been thinking a lot about how to utilize adversarial collaboration in the more theoretical context, especially philosophy:
https://arxiv.org/abs/2607.16374
I agree that collaborators need sufficient openness for the collaboration to bear fruit, and that this needs to be built into the structure of the collaboration somehow. A lab is an interesting way to go although as some have mentioned, it might be institutionally difficult.
We consider a few strategies. One is steel-manning the opponent e.g. one takes on the role of defending their opponent’s theory in seminars or discussion groups. The opponent may supervise this to ensure that one actually understands the theory being defended, and how to best defend it. I did this myself and found it helped a lot in an adversarial collaboration I was involved in (https://arxiv.org/abs/2607.16379
Another emphasizes a feasible condition for success, which does not require conversion. On our proposal, the collaboration succeeds when it produces a substantive advance beyond the existing literature, specifically: either the critic concedes that an objection to the opponent’s theory can be answered but only through a novel, non-obvious argument using the theory’s existing resources, or the advocate concedes that addressing it requires revisions or new conceptual resources. Both parties can therefore recognize publishable progress while retaining their opposing views on the theory’s ultimate truth.
So glad to learn of these ad collabs, Kelvin! What timing: looks like these just came out a couple of months ago. Great minds (sometimes) think alike! The template for ad collabs that you’ve put together is great. We’ve toyed with the idea of a “Neutral Mediator” too—perhaps even using AI for it (as mentioned in the post). Consider reaching out to Cory Clark and others at the Penn project. They’re interested in others trying this out in other disciplines.
This is a cool idea. That said, I wonder whether philosophical disagreements tend to have the features that make adversarial collaboration between disagreeing scientists fruitful. At the risk of sounding cynical, I would be somewhat surprised if this is the case. The examples of successful adversarial collaboration in the sciences that I know of featured scientists with shared data, at least an approximately shared sense of what counts as evidence for what, and similar methodological principles, whereas philosophical disagreements routinely concern what even counts as a reason for something else, lack shared data (or any obvious points of contact with empirical reality) to serve as a neutral court of appeals, and quite quickly run all the way down, so to speak, to methodology and metaphilosophy. Not all philosophical disagreements are so deep, but I wouldn’t be surprised if, on average (assuming talk of the “average” philosophical disagreement makes sense), philosophical disagreements are more intractable than average disagreements between scientists within particular areas of expertise.
Following up on myself, here: I suppose we could, as the authors and some commenters suggest, reconceive the purpose of adversarial collaboration in philosophy, such that the goal is to make dialectical progress or better map logical space rather than arrive at agreement. The question then becomes whether the benefits of adversarial collaboration on that front would outweigh the costs, relative to our current “dueling-papers” system, which has the advantage of aligning with current incentive structures (prioritizing single-authored papers advancing theses rather than co-authored papers and explorations of logical space) while also avoiding some costs of the adversarial collaboration model.
Good points. Perhaps the major impediment to adversarial collaboration in philosophy or elsewhere is the incentives to publish single-authored papers. Co-authorship is so much more common in the sciences. It could be that adversarial collaboration will be easier for later career philosophers, but I hope earlier career folks don’t rule it out!
Could the authors say a little more concretely what the benefits of adversarial collaboration in philosophy are supposed to be? I’ve found that often disagreements are substantial enough that the argument between the two sides isn’t very helpful or revealing. If someone else likes nonnaturalist realism as a metaethical view, I’m having a hard time imagining what the payoffs of our working together, either for them or me, might be.
I realize in the sciences this potentially corrects for confirmation bias, but I don’t think that the debate, for example, between Harman and Thomson over moral relativism is explained by bias.
I doubt that in these cases the philosophy would be improved by coauthoring in which only premises both sides of the debate would agree to are used. And if we’re just talking about books in which authors debate back and forth, that’s old.