After evaluating the Metaculus community’s comments in the Forecast With GiveWell series, GiveWell selected five top contributors to share the $3,500 prize pool:
Special recognition also goes to katifish, who made numerous high-quality comments and reached the maximum individual prize payout. We’re also grateful to the Manifund donors who contributed to the prize pool: Austin Chen, Saul Munn, Anton Makiievskyi, and Ryan Kidd.
The Forecast With GiveWell series aims to strengthen GiveWell’s decision-making, surfacing perspectives and analysis. Forecasts and comments can be compared with GiveWell’s public comments and reasoning, helping GiveWell sense-check the rationales behind its funding decisions and incorporate new information. GiveWell shared the following reflections on the winning comments:
Thank you to everyone who commented on our forecasts! We evaluated comments for clarity (10 points), reasoning (10 points), and impact (10 points). These were judged blind, i.e., without seeing the commenters’ names. Three GiveWell researchers evaluated each comment, including the researcher most closely involved in the grant related to the prediction.Overall, we found the comments to be clear and well-reasoned, and they engaged deeply with complex topics related to a variety of GiveWell’s grantmaking areas. The most impactful comments were ones that brought in additional evidence or knowledge, critiqued our approach, critiqued specific parameters we used in our models, or called out areas where our arguments or models were unclear.Our top-rated comment was by NickLaing on the question: “Will the study of supplying eyeglasses to near-vision impaired adults in Kenya and India find a boost to household consumption of the following levels?” This comment was well-informed, clear, and well-argued. It stood out for its argument of specific ways the study could be biased, including conflict of interest risk, implementation methods, data collection and sampling techniques, and lack of blinding. The commenter dug into our cost-effectiveness analysis and offered alternative suggestions for both implementation and evaluation of this program or one like it.The second-highest rated comment was also by NickLaing. It was the most critical comment of them all, in response to, “Will the study of oral rehydration solution and zinc distribution in Sierra Leone find a statistically significant increase in ORS/zinc usage by the recipients?” The arguments here are well-articulated and highlight some of the potential weaknesses of this grant. Although most claims aren’t related to the prediction itself, and we would have liked to see more evidence for some key claims, we appreciated the level of thought that went into this critique and the engagement with GiveWell’s assumptions and decision-making.Two comments by katifish had the third and fourth highest ratings. The first two relate to our ILC India grant predictions: First “Will in-line chlorination devices serve at least 5.8 million people in Andhra Pradesh at the end of the 4th year of Evidence Action's program?” Katifish’s response to this question was very detailed, sense-checking our estimates with evidence from other types of infrastructure projects in India. It was overall a strongly reasoned comment that included ways the commenter could be wrong or what would make them change their mind. One potential misunderstanding in this comment about what Evidence Action’s program would do (it will install chlorinators on existing water systems, not install new water points) was an important flag for us to make sure we are clearly explaining grant activities in our communications.The fourth-highest rated comment, on the prediction, “Will the household water chlorination rate be 65% or higher for households served in the first two years of Evidence Action's program in Madhya Pradesh and Andhra Pradesh?” introduced a few critiques and new ideas that we hadn’t previously incorporated into our work. For example, katifish pointed out that we didn’t fully explain our reasoning for adjusting results from a study we rely on to estimate chlorine levels in household water. This comment also brought up a consideration we haven’t yet incorporated or thought much about, which is how boiling and seasonality might affect residual chlorine measurements.We’re excited to continue to add forecasts to Metaculus to get even more meaningful engagement with our work. External feedback is an important part of what we do, as we strive for transparency and truth-seeking in order to help people as much as possible.Thank you to everyone who participated!
The Forecast With GiveWell questions remain open, and GiveWell and Metaculus would welcome additional forecasts and reasoning shared on these questions.
Congratulations again to our commenting prize winners, NickLaing, katifish, dimaklenchin, sleepyaardvark, and Zaldath!
Note that we awarded a 10.5th place because katifish hit the prize cap of $1,500 with the first $100 of their ninth place comment. Unfortunately the rules we wrote were a bit ambiguous about what should happen in this scenario, so we decided to award the leftover $100 from ninth place as an additional prize.
AI Use Note: Main body text entirely human written. Claude (Opus 4.8) helped develop models of animal life histories in the appendix.
Cross-posted from Good Structures.
Executive Summary
* Animal advocates sometimes make claims like “there are X of this animal...
“How long have you been v*g*n?”
This is one of the most common icebreakers at animal protection events. It’s a baseline assumption, and it mostly holds true: if you’re out advocating for animals not to be tortured or abused, realistically these days you are v**n, or close. And it makes for good conversation. It seems fairly safe to assume when you meet strangers.
But this assumption is hurting the movement in a way which we don’t always notice: someone new comes into the sp...
Summary
Back in November 2023 I posted here to launch Spiro and raise our first $198k. Two and a half years later this is an update and a fundraiser for the next step.
The short version: we've now reached over-5,900 people with TB preventive medicine, including over 3,000 children under five years old. Our early results have held up well an...
