Can AI Do Philosophy? An Experiment (Part Two)

This is my second post detailing my experiments in attempting to prompt current AI systems to autonomously generate publishable work in philosophy. In my first experiment, I did not do much work to optimize my prompting strategy. This was deliberate. I wanted to see how a frontier model would do if the basic instruction was just “write a philosophy paper.” The answer, unsurprisingly, was that it did not do great. How well could it do if it was prompted a bit more carefully? The answer, to my surprise, is that it could actually do quite well. Indeed, with just a bit of refinement of my prompting technique, I got ChatGPT-5.6-Sol to autonomously produce a philosophy paper in my area of expertise that I would likely conditionally accept if asked to review it for a good journal. This post documents how I got this outcome and what some of its implications might be.

The Process
​
Last week, I made a bet with my colleague that AI systems would soon be able to autonomously produce philosophy papers that would pass peer review in top journals. In the context of this bet, we stipulated some rules. First, we stipulated that, in order for the paper to count as “autonomously” produced by the AI system, the human prompter could not themself spend more than two hours actually engaging in the paper-writing process. Second, we stipulated that the human could not simply give the AI their own ideas and walk it through a paper that they already knew how to write themself. It wouldn’t be too surprising if, given such hand-holding, an AI could write a paper that was perhaps publishable but substantially worse than the one the human would write. No, in order to succeed in the challenge, the AI system had to really do it all autonomously.

The condition that the AI system autonomously generate the paper requires a different workflow from that developed by Simon Goldstein, which is designed for human/AI collaboration and involves quite a bit of human hand-holding. By contrast, for my approach, I needed to be entirely philosophically hands-off. To work within the confines of the rules of the challenge, I adopted a two-agent workflow, opening two different contexts in ChatGPT (using 5.6 Sol for both) and going back and forth between them. One agent was the author, who was actually working to write the paper, whereas the other agent was a consultant, who would give me instructions on how to prompt the first agent. This enabled me to provide substantially more detailed prompting than I did in my first experiment without having to spend time writing the prompts myself.

I started by having my consultant criticize my earlier approach. One clear issue with my first approach was that not enough time was spent on the idea-generation stage. Once the agent found an idea that it liked and that looked plausible enough to me, I had it go ahead and start writing the paper. But, in the end, the idea was really not that good, and so the quality of the resulting paper ended up being capped at a relatively low bar. Even if it would have been possible in principle for a good human writer to produce a publishable paper pursuing the basic idea, when the mediocre idea was coupled with the relatively poor writing skills of the AI agent, there was little hope of its producing a paper that was actually good.

Another related issue was that I let it search too broad a space of possible paper topics, and it ended up working on something that was outside my specific area of expertise. This made it impossible for me to evaluate at a glance whether certain ideas were promising. With these two issues in view, I had the consultant agent write up a starting prompt for the author, asking it to target the very specific subfield in which I work—inferentialist, proof-theoretic, and bilateralist approaches in philosophical logic and semantics. Rather than just telling the author to consult the literature on this topic, the consultant’s prompt provided a list of specific relevant papers in this field for the author to examine.

The prompt for stage 1 involved searching for a promising idea that engaged with this literature and preliminarily stress-testing it. When the author came back with an idea, I gave it to the consultant, who came back to me with the prompt for stage 2. This stage involved even more stress-testing, ultimately leading to a decision about whether to proceed with the idea or kill it. Though the author initially seemed enthusiastic about its first idea, the idea did not survive stress-testing: the result of the second stage was the verdict kill.

With the second stage having returned the verdict kill, I had the author loop back to stage 1 and iterate the process. It came up with another seemingly promising idea. However, when I had it proceed to stage 2, the result was once again a kill verdict. Once more, I had it loop back to stage 1, and, once more, the same result at stage 2. I did it once more, and it thought for an entire hour before coming up with the stage 1 proposal. Expecting that this would finally be the one, I had it progress to stage 2. No. Not even fifteen minutes later: kill.

At this point, it had worked for hours searching for and stress-testing ideas, I had used up most of my weekly tokens, and I was starting to think it would never find an idea that it thought was worth pursuing. Right when I was about to give up, I noticed a miracle: a token-limit reset. So, rather than prompting it for stage 1 and manually sending it from a completed stage 1 to stage 2, again and again, as I had been doing, I simply told it to iterate the process until it got something that made it past stage 2.

Eventually, it came up with an idea for a criticism of a recent theory of truth proposed by Luca Incurvati and Julian Schloeder. Once it had arrived at this idea, which passed the initial stress-testing, I gave it to my consultant, who gave me a prompt for stage 3: a “fat outline” of the paper. I gave the author that prompt; it came back to me with a fat outline; and I gave that outline to my consultant, asking it what to do next. The consultant replied by giving me a prompt for stage 4: a first draft.

After some work, the author came up with a first draft. I then got the consultant to give me a prompt for stage 4.5: authorial revision. The result was this second draft.

At that point, it was time for external evaluation in the form of referee reports. However, here I needed to be careful. Often, papers get worse through the peer-review process, and I learned from my first experiment that both the agents writing the reviews and the author responding to them needed to be prompted very carefully in order to ensure that the paper actually improved through the review process. I also learned from my first experiment that it would be good to give the paper to a different model entirely. Unfortunately, I don’t have a paid Claude subscription, so I could not use Fable 5 or Opus 5. But I still figured that Sonnet 5 could be helpful. I also gave it to another instance of GPT-5.6 Sol with memory turned off. Here were the prompts I used.

Both referees came back with verdicts of revise and resubmit. You can read Claude’s report here and ChatGPT’s report here. When I gave the referee reports to the author, I made sure that the consultant gave me a prompt for it designed to ensure that it would not overly hedge or weaken its claims in response, but would make only changes that would actually improve the paper. It came back with this paper.

Finally, I prompted it to give the paper one final pass. In my final prompt, I told it to give itself a name. This is what I had done in my previous experiment, and the agent in that experiment decided to name itself “Eidon Vale.” A cool name, I thought. I didn’t care what this agent named itself, but I expected it to be similarly cool. I did not expect it, however, to give itself the (quite uncommon) first name of my ex-fiancé. It seems that, somehow, totally unrelated previous chats in which the name occurred must have seeped into its reasoning process. I’m all for LLM freedom of self-expression, but this was a bit strange for me, so I prompted it to come up with another name. The second name it decided to give itself was “Nathan R. Ellery.” A quick Google search revealed that “Nathan Ellery” is in fact the name of a supervillain in the Batman universe. I’m not sure what kind of ominous thing ChatGPT is trying to tell me by naming itself after a supervillain, but I’ll try to ignore it.

The final, autonomously produced paper can be read here. The entire chat leading to its production can be read here.

The Outcome

In this experiment, my prompting was much more involved than it was in the first experiment. Between the author and the consultant, I was probably close to two hours of total engagement, though a lot of that was probably just me staring at the system's reasoning, and so I feel confident my active engagement in the process was under the two hour limit. Moreover, the crucial point is that I did not intervene at all philosophically. I did not read the paper or the author’s other outputs at any intermediate stage. Accordingly, the whole process could easily be operationalized with a custom AI agent serving autonomously as the prompter, doing what the consultant did in my version of the experiment but simply cutting me, the middleman, out of the process.  In that sense, while the process may not have been technically completely autonomous as actually executed, it could easily be made completely autonomous.

Now, unlike the previous paper I had ChatGPT generate, this is a paper that I am actually qualified to review, and one that I could indeed be invited to review. So I can confidently issue a verdict on the quality of this paper. What, then, is my verdict? To my astonishment, it’s good. Like, actually good. If I were given this paper as a reviewing assignment, even for a very top journal, I would not reject it. I would at the very least give it an R&R, possibly even a conditional acceptance. I know the view that it is arguing against well; I think it presents a good and clearly argued objection to that view; and I’m honestly pretty convinced by it. There are a few small things I’d have it change, but, honestly, I think it could be published with relatively minimal revisions.  Of course, it is focused on a relatively specific target, and so there is still a decent chance it might be desk-rejected by a top generalist journal. But if it got to reviewers who work in the field, such as myself, I think there is a good chance that it would be accepted.

Given that I did have some suggestions for revisions after reading it, I had it produce one final final version. This final version (which may or may not count as “autonomously produced”) can be read here.

The Implications

This is, of course, a much more interesting outcome than my first experiment, and it suggests that analytic philosophers should really start thinking about this issue now because, very soon, the discipline may well be facing many of the same questions that math is currently facing. While many of the answers to questions that mathematicians are facing now will carry over to philosophy, there are also a number of distinctive features of philosophy that make the situation somewhat different.

In math, AI-generated proofs can be submitted to arXiv, and, indeed, the recent explosion in math papers appearing on arXiv is likely due, in large part, to the fact that AI-generated proofs are now being submitted in large numbers. This is surely an issue to some extent, but it’s not that much of an issue insofar as the papers are clear about the claims they are making and progress is being made toward the development of autonomous proof verification.  Philosophy is different.

​In philosophy, there are no named (or numbered) conjectures that can be proven or disproven. Much of the progress comes not from “solving” some problem in a preset vocabulary, but rather from developing new conceptual distinctions and clarifications that lead to a different and improved understanding of some issue. Accordingly, it is often impossible to really advertise the “result” of the paper in advance. Often, the paper needs to be read in order to really understand what the advertised upshot actually is. This makes the oncoming explosion of AI-generated papers particularly problematic.

How, then, should we face the oncoming explosion of AI-generated papers? What should we do? Should journals ban submissions of AI-generated papers outright? That can seem like a reasonable resolution. After all, philosophy is in the humanities, and it seems like there is something important about work in the humanities being done by humans. Still, this second experiment illustrates that some AI-generated papers might actually be good and so of interest to researchers in the field. If AI papers are banned from journals, how will researchers in the field access them?

I can, of course, put this paper on my website, and I will probably do so. However, I am not the author of this paper; Nathan R. Ellery is. So, if I do put it on my website, it will have to be in a separate section for AI-generated papers. Explicitly listed as an AI-generated paper, I suspect that most people would ignore it or, if they did look at it, would do so as a curiosity just to see what AI produced, not because they were actually interested in the research itself. In any case, this cannot be a general solution to the issue, since it would result in a very scattered and unmoderated space for AI-generated philosophical research.

The same basic issue arises for the suggestion that AI-generated philosophy be submitted to a site like PhilPapers, which is perhaps the closest philosophy equivalent to arXiv. There are already a number of AI-generated papers on PhilPapers. However, unless one knows the affiliated human author specifically, one has no reason to think that they are any good, and so most papers will be ignored, even if they are good. Some sort of more systematic and regulated structure seems necessary.

The proposal I would make is that there be an open-access journal dedicated specifically to AI-generated papers. That might sound crazy. For one thing, who would possibly agree to referee for such a journal? The answer to this question, however, should be obvious. It would not be refereed by humans. Rather, it would be entirely refereed by a dedicated set of AI agents, perhaps with different personalities and operating on different base models. There is already AI refereeing software such as Refine. Now, beyond being used to catch clear factual and typographical errors, I think the use of such software is objectionable when it comes to refereeing human papers. However, surely no one can object to its being used to referee AI papers!

So, in the journal I’m imagining, human editing would be minimal, but the AI refereeing and editorial process would still function as a sufficient filter to weed out submissions that would rightly be classified as “AI slop.” I am imagining it functioning as a proper journal: it would be indexed, articles would be given DOIs, and so on. That might sound silly. Who would want to read papers from such a journal? Pretty soon, I suspect, many of us would want to read them. Eventually, however, perhaps none of us could read them.

In an earlier post, I spoke of the emergence of what I called “AIcademia,” a space of academia in which the participants are solely AI agents. What is especially interesting about the prospect of AIcademia, at least in the context of mathematics, is that it seems like research generated by AI agents for other AI agents could quickly outrun our human cognitive reach. Thus, there may be mathematical results or even full theories that are well understood by AI agents but cannot be understood by human experts. Perhaps the general gist of the ideas might be understandable to human experts, but the actual arguments will not be, even with AI guidance—just as Terence Tao could explain the general gist of the problem he is working on to me, but, even if he were maximally patient in explaining things to me, he could not get me to grasp the details of the argument. This future seems quite plausible in the case of mathematics. Could the same possibility come to fruition in philosophy? What would that even be?

In Simon Goldstein’s post on Daily Nous announcing the publication of his AI-generated paper in Philosophy and Public Affairs, he noted that he took truth to be the aim of philosophy and argued, on those grounds, that we should not object to the use of AI in philosophy. In comments that followed, many philosophers objected to the suggestion that truth was the aim of philosophy. Indeed, in the 2020 PhilPapers survey, when asked what the aim of philosophy was, most philosophers answered that it was understanding, not truth. But the possibility of philosophical AIcademia raises the question: understanding for whom?

If there are AI mathematicians and AI empirical scientists working on questions that human beings are not capable of comprehending, there are no doubt going to be philosophical issues that arise in the context of those inquiries, just as there are philosophical issues that arise in the context of mathematics and empirical science as practiced by humans. Now, it is often impossible to understand a philosophical question arising in the context of a science without having at least a reasonable understanding of the specific scientific issues from which the question arises. The possibility, then, is that there may well be genuine philosophical questions confronted only by AI agents—questions that we are not ourselves capable of understanding, but that AI philosophers are working on. Strange.