Can AI Do Philosophy? An Experiment (Part One)

Simon Goldstein has just published the first paper in a top philosophy journal that was openly written primarily by AI: “Epistocracy and the Commitment Problem,” published in Philosophy & Public Affairs.  According to Goldstein, Claude was the primary writer of the paper, and Goldstein’s role in the development of the paper was similar to that of a PhD advisor, albeit a particularly overbearing one.  Goldstein supplied the initial thesis, guided Claude through revisions, and tweaked some bits himself. 

There are two things to note about Goldstein’s (or Claude’s) paper. 

1. By Goldstein's own admission, the paper was not absolutely top-tier philosophy.  Goldstein suggests that “the quality was maybe 80% of the usual quality when I produce a paper purely by myself.”  Still, despite not being up to Goldstein’s own usual standard, it was good enough to pass through blind refereeing at a top journal. 
2. Goldstein is clearly more skilled than most at getting AI to write a philosophy paper.  Still, by Goldstein’s own estimate, he spent less than a single workday on writing the article, including the revisions and responses to reviewers. 

Goldstein has documented his process in detail, and made it available for others as a specialized agent in Claude Code so that anyone who wants to adopt his process to get AI to write philosophy papers can do so themself.  Whatever one's views on all of this are, it seems, clear that there will very soon be an explosion in AI-generated philosophy papers submitted to journals. The question  is: will these papers be any good?  Or, since it's very hard to say what's actually good in philosophy, the question might be put, more precisely as follows: will these papers be the sorts of papers that would actually make it through peer review at top journals?

Last night at the bar, I was discussing this question with my colleague at Wuhan, Atus Mariqueo-Russell.  We decided to make a bet.  After some back and forth, we settled on this one (note, I often go by "Sims" in non-academic contexts):

• If [by this time next year] AI has produced an article published or forthcoming in Nous, Mind, PPR, AJP, or Philosophical Review then Atus will treat Sims to a night out of drinks. If it has not, Sims will treat Atus to a night out drinking.
• By ‘produced’ we mean relatively minimal prompting and less than 2 hours total work prior to the revisions. And the revisions can be human authored but not extreme.

Now, this bet is not perfect for a few reasons.  First, I think it's actually too permissive as stated: the human should not be permitted to author the revisions.  Second, and more problematically, even if AI does autonomously produce a paper that gets published in a top generalist journal in the next year, it seems quite likely that the human “author” of the paper may not disclose that paper was in fact autonomously produced by AI. 

If I actually wanted to win the bet, the obvious way for me to get around the second problem and ensure that I do win the bet if the AI capabilities are actually there would be for me to take matters into my own hands, prompting AI to produce a publishable paper myself.  Don’t worry: I’m not going to contribute to the crisis that journals are facing by actually submitting an AI-generated paper to a journal (not, at least, until the norms around the permissibility of this sort of thing are settled and I’m sure to abide by them).  Still, just to get a sense of where things are currently at, I wanted to see whether I could get a paper that could stand a reasonable shot of getting through peer review. 
​
Now, if I were really trying to win the bet by producing a paper of my own, I would at least wait until the next generation of models are released to give a proper attempt.  OpenAI’s upcoming “Astra” is rumored to be a much larger base model, and also rumored to be much better at writing.  Indeed, I have found that the current generation of models, although much better at mathematics than previous models, are actually much worse at writing.  I believe this is a byproduct of the extensive Reinforcement Learning on Verifiable Rewards (RLVR) for mathematics and coding to which these models are subjected.  Needing to avoid errors that might lead to a failed proof or a code that doesn't compile seems to have led this new generation of models to be so over-cautious about saying anything wrong, that their attempts at writing philosophy are basically unreadable.  I sincerely hope that OpenAI and Anthropic have taken note of this issue and try to address it in the next generations of models.

All of this is to say, going into this experiment, I did not have very high confidence that it would be able to produce a very good article. Still, since ChatGPT limits are resetting tomorrow and I had some compute to spare, I decided to see where current models were at when it came to producing philosophy journal articles.  To give you the spoiler now, my expectations were pretty much confirmed.  The article it produced was not very good.  This was certainly, in large part, my fault as a prompter.  I did not do any work antecedently thinking through what the best way of prompting it would be.  I did not follow Goldstein's detailed methodology, as it involves much more human engagement than permitted in the context of the bet.  I also did not spend any time to develop a methodology of my own. This was deliberate.  This first experiment was meant to serve as a baseline establishing what kind of philosophy paper one would get by minimally and relatively unskillfully prompting a frontier AI to write one from start to finish. 

Unlike Goldstein, I did not give ChatGPT a thesis to work with.  After all, often, coming up with a good thesis is the hardest part of doing philosophy.  So, I just gave it a general area of research: formal semantics.  I gave it this area for two reasons.  First, it’s one of the general fields I work in, and so I would be able to evaluate its output, at least to some extent.  Second, current models are very good at math, and, often, this field involves doing some math.  Still I made sure that the paper it produced was still squarely a philosophy paper, targeting top philosophy journals.

To prompt it to come up with its paper topic, I simply asked it to search the top journals (those included in my bet with Atus) for a place where it could make an intervention. After some searching, it ended up eventually keying in on recent literature about probabilities of indicative conditionals, three-valued semantics for conditionals, and an unhappy consequence of one recent account.  It came up with a proposal for a paper entitled “What Do We Learn from a Conditional” that intervened in this literature.  I first had it write an introduction.  I then just told it to write the paper.

I did not do any extensive prompting in actually getting it to write the paper.  I basically just told it to write it.  The first draft was a bit technical, so I told it to make sure it is readable to the interested general reader in the second draft.  Upon it's production of the second draft, I asked it if it was happy with it or if it would like to write a third draft.  It decided to write a third draft.  I had tweak the abstract, and that’s it.  The paper it came up with the first version of a paper entitled “What Do We Learn from a Conditional?” which can be read here (note: this is just the first version, there will be three more to follow). 

If you look at the paper, you can see that I am not an author.  I shouldn't be an author, since I hardly did anything.  So, instead of listing myself as an author, I told the instance of ChatGPT that it would be the author of the paper to come up with a name for itself.  It came up with name “Eidon Vale,” and so that is who is listed as the author of the paper. I'll henceforth refer to this "instance agent" by their self-chosen name: Eidon.

​Originally, I had the idea of making a custom “GPT” for Eidon Vale which supplied all of the relevant paper-writing context—everything that ChatGPT made visible as far as summing up the chain of thought that led to the development of the paper.  Then readers could interact with (a copy of) Edion Vale and ask questions about the paper and raise objections to it.  Unfortunately, ChatGPT has discontinued shareable custom GPTs, and so there is, as of now, no good way for readers to correspond with the AI-agents who are responsible for academic works.  As AI agents start to do more and more research autonomously, I really hope AI companies do something to facliate such correspondence.  I’ll have more to say on that at some point, but it’s not the point of the current post.  So, back to the experiment.

Once Eidon gave me their first draft, I had to evaluate it.  Now, I really didn't feel like doing a proper referee report myself. This was for three reasons.  First, I was lazy.   Second, referee reports typically take longer than two hours, and so, if my feedback figured in its revision, I would be breaking the rules of the bet.  Third, though the paper is in one of the general fields that I work in, it’s not really on specific issues that I’m actually working on.  Indeed, I probably wouldn't get asked to review this paper (or, if I did get asked, I probably wouldn't accept the invitation).  So, it would require quite a bit of work to get sufficiently well-acquainted with specific literature it deals with to properly evaluate it. 

Not wanting to review the paper myself, I decided to use AI to generate a review for the paper. To do this, I turned memory off in ChatGPT so that it would not know that it was in fact the author of the paper.  I then gave it the paper to review as if it were a referee of one of the target journals for the paper: Mind, Nous, or PPR.  I reminded it that these journals typically accept less than 10% of submissions, and to give a genuine report as if it had been submitted to one of them.  After some deliberating, it came back with a verdict: rejection.  Alas.

Still, we are all used to getting rejections and, often, a better paper comes from taking the comments in the rejection into account, albeit one that must be submitted to a different journal.  So, I gave the review to Eidon Vale and instructed them to come up with another draft that improves the paper in response to it.  I told Eidon to treat it as a rejection rather than an R&R.  That is, they are not compelled to respond to the comments, but, if they think there are actually good points there that would improve the paper, they could do so.   It came up with this second version of the paper. I then gave that paper to the same anonymous reviewer (once again, with memory turned off), and, this time, there was better news: revise and resubmit. I gave the second-round report to Eidon, and they came up with this second revision of the paper.  Finally, I took a very brief look at the paper, noticed some things weren't clearly explained, and made a pretty generic suggestion to explain more and make it clearer.  The result was this final version of the paper.  

While Eidon and the referee worked for a few hours combined, I was off working on other stuff while they were working on this paper, and my own engagement in the process took substantially less than two hours of total.  It's how to say exactly how long I was actually engaging in the process, but maybe about an hour total.  You can read the entire exchange with Eidon that led to the production of the paper here, and you can read my short back and forth with the "referee" here.

Is the final paper any good?  Well, once again, it’s not really on a topic I work on, and I haven’t really invested all the time in reading it as I would if I were actually doing a review.  Still, I can evaluate it to some extent.  I think it’s not absolutely terrible, but it's definitely not great.  I’d be pretty surprised if it got even an R&R at one of the journals it was aiming at.  Most likely, it’d be rejected.  Indeed, it'd probably even be desk rejected.  Still, if the editor of the journal could get past the relatively poor writing, there's some chance that it would make it to reviewers, who would have to decide what to do with it.  If it were to get to this stage, it seems to me that there's a small but non-zero possibility that reviewers might see something in it and give it a borderline R&R.

The basic proposal of the paper, I think, is reasonable enough, and, as far as I can tell, it’s all technically sound, though I didn't go through the formal details too thoroughly.  The main issue is that it’s really not written well at all.  Being frank, it reads like a paper by someone who is technically competent, but really doesn't know how to write a good philosophy paper.  It’s not super clear, it fails to sufficiently motivate the intervention that it is making, and, in general, it's just not at all enjoyable to read.   Another issue is that  Eidon likely conceded too much to the referee in the course of revisions, resulting in a more complicated paper that treads too lightly and qualifies too much rather than making a single sharp and memorable point.  This was, indeed, one of my main concerns in getting this generation of models to write philosophy and it turns out it was justified.

So, the paper is not good.  Still, it’s not terrible, and I wouldn't be too surprised if a version of it could get eventually get published at a second-tier journal after a couple rounds of review.  ​It's also worth noting that I ran it through a number of AI detectors, which came back with the result that it was written by a human.  So, though it is clearly not top-tier philosophy, an autonomously generated paper of this sort is already likely to pose a serious challenge to a number of journals which may not have sufficiently strict standards to just desk reject it.

Of course, there may be some luck of the draw involved, but, I expect that similar prompting will yield similar results as this experiment.  Thus, it appears, as expected, that current models are not quite capable of autonomously producing the sort of philosophy that could appear in top journals.  However, it seems to me that they’re not that far off from autonomously producing papers that would at least get a serious look from top journals.  Indeed, my current best guess is that even just the next generation of models, set to release in the coming days or weeks, might be capable of producing papers that will be genuinely hard to distinguish from the sort of papers that editors and reviewers will regard as genuine candidates for publication in top journals. Pretty soon, the discipline will have to deal with a lot of autonomously AI-generated papers that are at least not obviously bad and which might actually be good.

I may try this experiment again with a bit more strategic prompting.  I will, of course, try this experiment again when the new generation of models arrive, letting a new Astra instance agent work up a new paper from scratch.  At that point, I'll also likely let a super-charged Eidon Vale see what they can do to revamp this one.

8/20 Update: Ideas Are Hard

I tried to give a more proper attempt at getting 5.6-Sol to generate an actually good paper.  I focused on my actual area of research (inferentialism, proof-theoretic semantics, bilateralism), gave it a bunch of papers to look at, and tried to have it find some idea that was genuinely good and novel.  In this case, I gave quite elaborate prompts (generated with the help of ChatGPT in another context window) to have it really stress-test the ideas that it came up with. After many hours of compute and multiple attempts, none of them survived its own stress-testing.  Some of the ideas actually seemed reasonably promising, yet it decided that none of them were worth pursuing.  I guess new ideas are hard to come by! 

You can read that entire chat here.