At GiveWell, we've been experimenting with using AI to red team our global health intervention research—searching for weaknesses, blind spots, or alternative interpretations that might significantly affect our conclusions. We've just published a write-up on what we’ve learned, both about the programs we fund through donor support and about how to use AI in our research.
We're sharing this to invite critiques of our approach and to see if others have found methods for critiquing research with AI that work better. Specifically, we'd love to see people try their own AI red teaming approaches on our published intervention reports or grant pages. If you generate critiques we haven't considered or find prompting strategies that work better than ours, please share them in the comments—we'd be interested to see both your methodology and the specific critiques you uncover.
Our process
Our research team spends more than 70,000 hours each year reviewing academic evidence and investigating programs to determine how much good they accomplish per dollar spent. This in-depth analysis informs our grantmaking, directing hundreds of millions in funding annually to highly cost-effective, evidence-backed programs.
Our current approach for supplementing that research with AI red teaming:
Literature review stage: An AI using "Deep Research" mode synthesizes recent academic literature on the intervention
Critique stage: A second AI reviews both our internal analysis and the literature summary to identify gaps in our analysis
We applied this approach to six grantmaking areas, and it generated several critiques worth investigating per intervention, including:
- Whether low partner treatment rates in syphilis programs lead to reinfection (with pointers to relevant studies)
- Whether children recovering naturally from severe malnutrition could make treatment programs appear more effective than they actually are
- Whether circulating parasite strains differ from malaria vaccine targets, potentially reducing effectiveness
For more on our current approach and the critiques it identified, see our public write-up.
Our prompting approach
Our red teaming prompt (example here) has a few key features:
- Explicit thinking process: We instruct the model to generate 20-30 potential critiques first, then filter for novelty and impact before selecting its top 15. This is meant to increase creativity before filtering down.
- Verifying novelty: The prompt explicitly asks the model to check "Is this already addressed in the report?" for each critique before including it. In practice, this helps but doesn't eliminate redundant critiques.
- Structured categories: We ask for critiques across specific categories (evidence quality, methodology limitations, alternative interpretations, implementation challenges, external validity, overlooked factors) to encourage broader coverage.
- Concrete prompts for novel thinking: We include questions like "What would someone from the target community say about why this won't work?" and "What existing local solutions might this disrupt?" to push toward less obvious concerns.
We arrived at this through trial and error rather than systematic testing. We're uncertain which elements are actually driving the useful output or are counterproductive.
What we learned about using AI for research critiques
A few initial lessons:
- Best for closing literature gaps: AI is most useful when substantial academic research exists that we haven't yet incorporated into our models. It found several studies on syphilis that we were unfamiliar with, but added little for interventions we’ve thoroughly-reviewed like insecticide-treated nets.
- Quantitative estimates are unreliable: AI often suggested specific impacts ("could reduce cost-effectiveness by 15-25%") without a solid basis, most likely because it can't effectively work with our multi-spreadsheet models.
- Relevance filtering required: ~85% of critiques were either unlikely to affect our bottom line or represented misunderstandings of our work. AI was not helpful for filtering its own results, and our researchers needed to filter the critiques for relevance and decide which ones were worth digging into.
A note on timing: This evaluation was conducted 4-5 months ago. While we haven't done systematic retesting with the same prompts and context, our impression is that critique relevance has improved, primarily through better alignment with the types of critiques we're looking for. Our rough guess is that the rate of relevant critiques may now be closer to ~30%, a meaningful improvement but not enough to change our research workflows.
Improvements we've considered but not pursued
We've deliberately kept our approach simple—running prompts through standard chat interfaces (Claude, ChatGPT, Gemini) that our researchers are already comfortable with. We've considered but chosen not to pursue:
- More complex prompt architectures: Breaking the task into more specialized sub-prompts, or using multi-agent workflows where different AI instances debate or build on each other's critiques.
- Custom tooling: Building dedicated applications using AI automation platforms or command-line tools.
- Specialized research platforms: We briefly evaluated AI-powered research tools but found them too narrowly focused on specific tasks (e.g., cliterature landscaping) to perform well at generating research critiques.
We suspect the gains from adding complexity to this workflow would be marginal and unlikely to outweigh the friction of adopting less familiar tools. But we hold this view loosely—if someone has achieved meaningfully better results with more sophisticated approaches, we'd consider spending more time on these other approaches.
Why we're sharing this
We think we're still in the early stages of learning how to use AI well, but we've developed preliminary views about what works and what doesn't, and we'd appreciate input from others thinking about similar problems.
Specifically, we’d welcome hearing about:
- Blind spots in our methodology or prompting strategy
- Improvements we're missing (including on the approaches above that we've deprioritized)
- Alternative workflows or prompting strategies that have worked well for similar tasks
- Pointers to research on AI-assisted critical analysis we might have missed
If you're interested in trying your own approach on one of our published intervention reports, we'd be curious to see what you get–both methodology and output.
Overall this seems like a sensible, and appropriately skeptical, way of using LLM's in this sort of work.
In regards to improving the actual AI output, it looks like there is insufficient sourcing of claims in what it puts out, which is going to slow you down when you actually try and check the output. I'm looking at the red team output here on water turpidity. This was highlighted as a real contribution by the AI, but the output has zero sourcing on it's claims, which presumably made it much harder to actually check for validity. If you were to get this critique from a real, human, red-teamer, they would make it signficantly more easy to check that the critique was valid and sourced.
One question I have to ask is whether you are measuring how much time and effort is being extended into managing the output of these LLM's and sifting out the actually useful recommendations? When assessing whether the techniques are a success, you have to consider the counterfactual case where that time was replaced by human research time looking more closely at the literature, for example.
Thanks for the feedback!
On your first point, if researchers found any of the critiques raised by the AI to be credible, we would follow up in the same chat to ask for sourcing or additional information (like for the water turbidity point). We found this more successful than asking for heavy citations in the initial output, which led to lower quality critiques (presumably because we were giving too many instructions). The most recent generation of models are better at following complex instructions, so I expect we should revisit this.
Human researchers spent ~75 minutes reviewing AI output per intervention. We haven't rigorously compared this to having researchers look into the literature themselves, but my sense is that currently using AI is worth it for areas where we haven't done as much research (family planning, syphilis treatment) and not worth it in areas where we've done a lot of research (bed nets, vaccinations). Given the pace of improvement we've seen over the past year, I expect a preliminary AI red teaming pass will be worthwhile in almost all cases by the end of 2026.