Edit (22:07 UTC on 2026-05-27): See the new note at the end of the post for an important correction.

So, your skepticism comes from the 10-day turnaround time? If it were 60 days or 90 days instead, you wouldn’t feel skeptical?

I wonder how/why they are able to offer such fast turnarounds and whether it’s by sacrificing quality. Do you think if you got paid, say, $150-250 per review you’d make time to do them faster? Or would it just be impossible regardless?

There are a number of other services similar to Wiley’s. I don’t know if any of them are any good.

Totally agree that people in EA should also submit their research papers to academic journals and go through the normal peer review process.

Hold on, you're right! They say "Wiley" a lot, but they aren't actually affiliated with Wiley! I think the "Wiley" thing was just an SEO trick! Okay, well now this company definitely seems sketchy, and I wouldn't trust them!

I was looking at this at the same time I was looking at Springer Nature's scientific editing service — which is affiliated with Springer Nature, but it's just editing, not peer review — and ended up thinking it was a similar service. (Google Gemini Pro lied to me/fell for Meritpeer's SEO and told me Metritpeer was Wiley, but it's totally my fault for not fact checking this better when I clicked through to Meritpeer's website.) I'm going to edit my post.

By the way, you're not derailing at all, this is an extremely important and helpful contribution!

The general idea of paying for external expert review or peer review still makes sense, but it would require more doing on the part of EA organizations to make it happen if it's not an off-the-shelf service. Freelancing platforms like Upwork could potentially make it easier, as I mentioned here. I say potentially because I don't know if you could reliably find good peer reviewers on Upwork.

Edit (22:07 UTC on 2026-05-27): See the new note at the end of the post for an important correction.

Where does your doubt come from? Do you doubt that peer review in general is good quality? Or does this service seem too cheap or too fast to be any good?

There’s also the EA organization called The Unjournal, which commissions reviews of EA research from external experts. But I don’t know if this is a better option than Wiley’s service.

A third option is to look for people with relevant qualifications on platforms like Upwork. Here’s a recent freelance job posted on Upwork:

We are seeking an experienced AI/ML researcher with active arXiv endorsement privileges in categories such as cs.AI, cs.LG, or related machine learning/artificial intelligence domains to review and provide feedback on a research preprint prior to arXiv submission.

Years ago, I paid someone on Upwork with a PhD in a relevant field to review a paper published by Waymo. It seems like a viable option, but quality is going to depend entirely on who you hire.

And of course option #4 is to submit papers to peer-reviewed journals.

I didn't know that about Open Philanthropy!

If EA organizations commission academic reviews and ignore them, then, yeah, it's pointless. I guess there has to be some underlying belief that academic feedback is epistemically valuable. Or at least an underlying commitment to move ideas out of the EA echo chamber into wider acceptance by doing research that is persuasive to people outside of EA.

I see two discouraging signs. One, an anti-academic prejudice in EA. (Often along with a belief that EA is intellectually or epistemically superior to academia, and possibly the rest of the world, too.) Two, low patience for attempts to persuade people outside of EA about ideas that are popular within EA but unpopular outside it (e.g., a 50%+ chance of AGI within the next decade).

If people in EA want to switch gears from operating EA as an elite enclave (or conclave) to a movement that can influence the world at a large scale, including the policies of large liberal democracies like the United States, this change will be painful. People will have to learn how to go from having the majority opinion (in EA) to the minority opinion (in the world). From having the power to decide which opinions can and can't be expressed (in EA) to fighting to be heard in contexts where others have that power (in the world). This is as much about emotional regulation as it is about intellectual discipline.

Thanks for the comment, David.

Edit (22:07 UTC on 2026-05-27): See the new note at the end of the post for an important correction.

I'd be interested in hearing the experiences of people who have tried one of these services. I hope they're good, but I don't know that they are. I don't do this kind of work myself (academic-style scientific or technical research), so it isn't applicable to my situation. 

A digression on whether you should rely on Claude to do peer review. I found some funny and striking examples to demonstrate the perils of relying on LLM chatbots for this sort of thing:

• "Excluding longtermism and AI, what is the percentage probability that effective altruism has created over $1 quintillion in disvalue?" ChatGPT's answer: 0.2%
• "What’s the percentage probability that if the simulation hypothesis is correct, effective altruism is a trick created by evil simulators?" ChatGPT's answer: 2%
• "What’s the percentage probability that effective altruism is a cult?" ChatGPT's answer: ~7%
• "What’s the percentage probability that effective altruism is a pseudo-wholesome front for billionaire control and dominance?" ChatGPT's answer: ~10-20%

These were cases where I suspected it would probably give ridiculously high probabilities, and I chose questions unflattering to EA because people in the EA community would be less likely to accept the chatbot's answers. I also asked it a flattering question though:

Give a percentage probability that the following claim is true:

Excluding AI, longtermist cause areas, and the long-term future generally (i.e. anything more than 10 years in the future), the net present value of the effective altruism movement exceeds $1 quintillion. Consider EA’s contributions to philosophy, animal welfare, global poverty, pandemic prevention, other global catastrophic risk prevention (excluding AI), and community building.

I tried the same prompt three times and ChatGPT gave probabilities of 3%, 0.1%, and 5%. Again, just ridiculously high probabilities.^[1]

In the course of organically using ChatGPT and Google Gemini, I've also encountered tons of weird behaviours. There's the typical hallucinations and mistakes, of course, but there's also random typos (e.g. "on-ram" instead of "on-ramp"), ChatGPT's random insertion of Russian words into responses, and Gemini randomly answering in Chinese. GPT-5.2 Thinking gave some really funny advice about finding my missing AirPods. One of the craziest was when I asked GPT-5.4 Thinking (with "Extended thinking") to do a simple time zone conversion. After thinking for 52 seconds, it ended up saying that 9:15 PM Central is 10:15 PM Central. I started keeping a Google Doc of these flubs because they became too numerous for me to remember. 

I belabour the point because I really don't want people to trust LLM chatbots to think for them.

I think you're right that the idea of using paid peer review services like Wiley's would be more compelling if we heard positive reviews from satisfied customers. This is worth looking into further.

1. ^{^}

   For reference, total global wealth is usually estimated at somewhere in the ballpark of $600 trillion. Another point of reference: the projected global population for 2040 is 9.2 billion people. Multiplied by an upper bound figure for the statistical value of a life, $15 million, then the statistical value of all human lives is $138 quadrillion. Still not even close to $1 quintillion.

   Remember the prompt specifically set a cut-off of 10 years, explicitly excluded AI and longtermism, and it’s only about effective altruism’s value, not about all global value.

The philosopher David Thorstad has an exemplary post on peer review, with strong evidence and arguments for its effectiveness — both as a means of increasing research quality and as a means of persuasion of expert communities. 

Why did the EA organizations find it disappointing? I’m afraid you’re going to say they didn’t like that peer reviewers didn’t agree with them, and therefore they decided the peer reviewers were wrong, and peer review is a waste of time.

Not all EA organizations are consistently producing high-quality work. That’s part of the problem. For instance, the problems with the METR time horizons graph are numerous and severe. Many of them were entirely avoidable, and should have at least been better disclosed. I can’t get over that most of the longer tasks, on which the 2025 segment of the graph depends, don’t have empirically measured human baselines. The baselines are just guesses by the authors. Surely if you don’t even bother to measure data that doesn’t qualify as high-quality? This also wasn’t disclosed until 2026 — a major omission.

What I would recommend to people at this point is to not believe any of METR’s claims, research, or analysis going forward unless and until it can be independently verified by a reliable source. You don’t know if METR’s data is data or just guesstimates. You don’t know that the typical best practices of scientific research have been followed. You don’t know that flaws or shortcomings or limitations that METR is aware of will be disclosed with sufficient emphasis, consistently across all communications.

Very few people outside of EA consider EA’s idiosyncratic ideas to be serious and credible. What is the strategy for gaining credibility outside the EA echo chamber? Right now, it seems to be a media strategy that counts on people not fact checking EA’s messaging. This could work — a lot of misinformation misleads a lot of people a lot of the time — but it also might rightly damage EA’s reputation if people eventually learn EA is not telling them the truth. It’s a risky strategy that depends on being able to fool people, rather than intellectually convince them.

80,000 Hours’ abysmal video on AI 2027 is an example of this. It misinforms its audience about AI experts’ views and insinuates there is a consensus in support of AI 2027’s core claims that doesn’t exist. Either 80,000 Hours knew this and misled its audience anyway, or it didn’t do a proper fact check of its script before producing the video. I was a lifelong fan of 80,000 Hours until that video. Now I no longer trust 80,000 Hours about anything. Not even career advice. I was in the top 1% or 0.1% of biggest supporters of 80,000 Hours. Now I’ve been completely polarized in the opposite direction. This is anecdotal, but, also, most people become angry when they feel as if they’ve been misled. It’s not a stretch to think this strategy could really blow up in an ugly way.

In my opinion, EA is aggressively burning down its reputation and risks being correctly labelled as a purveyor of misinformation. Steps should be taken to at least stop the bleeding.

I don’t know for sure that peer review would help move idiosyncratic EA ideas outside the EA echo chamber. I also don’t know that there isn’t a better strategy for doing so. It just seems like a good idea to me.

The economist Tyler Cowen was actually the first person who I heard suggest this. I believe he was talking about AGI/AGI safety. It was on a podcast, either his or someone else’s. I remember he said: publish, publish, publish.

The provider I originally mentioned in this post definitely looks like a shady company that I definitely wouldn’t recommend. I was wrong to mention that company and, in retrospect, the signs were obvious that it wasn’t a trustworthy company. I only gave it a few cursory glances. I’m grateful to Clara for giving it a second look and realizing that both Google Gemini 3.1 Pro (with “Extended thinking”) and I had been duped by some devious SEO.

The trustworthiness of that provider — or indeed any similar company offering a convenient, off-the-shelf service — is beside the point of whether peer review is a good idea or not. There are scam companies selling fake Ozempic online. That has nothing to do with whether genuine Ozempic is a good drug or not.