Hide table of contents

Comment Permalink

Manifold Markets ran a prediction tournament to see whether forecaster would be able to predict the winners! For each Cause Exploration Prize entry, we had a market on "Will this entry win first or second place?". Check out the tournament rules and view all predictions here.

I think overall, the markets did okay -- they managed to get the first place entry ("Organophosphate pesticides and other neurotoxicants") as the highest % to win, and one of the other winners was ranked 4th ("Violence against women and girls"). However, they did miss out on the two dark horse winners ("Sickle cell disease" and "shareholder activism"), which could have been one hypothetical way markets would outperform karma. Specifically, none of the Manifold forecasters placed a positive YES bet on either of the dark horse candidates.

I'm not sure that the markets were much better predictors than just EA Forum Karma -- and it's possible that most of the signal from the markets were just forecasters incorporating EA Forum Karma into their predictions. The top 10 predictions by Karma also had 2 of the 1st/2nd place winners:

And if you include honorable mentions in the analysis, EA Forum Karma actually did somewhat better. Manifold Markets had 7/10 "winners" (first/second/honorable), while EA Forum Karma had 9/10.

Thanks again for the team at OpenPhil (especially Chris and Aaron) for hosting these prizes and thereby sponsoring so many great essays! Would love to see that writeup about learnings, especially curious what the decision process was that lead to these winners and honorable mentions.

Showing 3 of 5 replies (Click to show all)

ChrisSmith

Hi jserv! I'll aim to say a bit more about the nuts and bolts of the process in an update before the end of the year, but prize selection was dominated by blind, independent review. I'm following up with you privately on tuberculosis.

Jon Servello3y0

Thank you!

Jon Servello

Mainly that forum karma was highly predictive of winners when you include honourable mentions, even moreso than Manifold Markets. It could be for several reasons; forum votes were taken into consideration, there was a high degree of alignment between forum voters and the Open Philanthropy panel on the known call criteria (slightly more than MM, another related community), karma acted as a proxy for something else that factored into selection like timing of the submission, general community interest, familiarity with the topic, or true quality of the submissions, or it could be chance. As an entrant, I am very interested to know whether any of the above played a role. Also, Open Philanthropy have mentioned this was an experiment, so it would be great to know whether anything unexpected came up during their decision-making.

See in context

Cause Exploration Prizes: Announcing our prizes

by ChrisSmith, Aaron Gertler 🔸

Sep 9 20222 min read 28

221

Cause prioritizationGlobal health & developmentCause Exploration PrizesOpen Philanthropy

Frontpage

Cause Exploration Prizes: Announcing our prizes

Our top prize

Second prizes

Honorable mentions

Good-faith submissions

Future plans

Thank you

28 comments

We were gratified to receive over 150 good-faith submissions to Open Philanthropy’s Cause Exploration Prizes, where we invited people to suggest a new area for us to support or respond to our suggested questions. We hoped that these submissions would help us find new ways to carry out our mission — helping others as much as possible with the resources available to us.

You can read them on the EA Forum. Below, we highlight the submissions to which we are awarding major prizes and honorable mentions.

We’re awarding these prizes to entries that we thought engaged well with our prompts and helped us to better understand the questions and issues they addressed. We have not investigated each and every claim made in these entries, and the awarding of a prize does not imply that we necessarily endorse their claims or arguments as correct.

Our top prize

We are awarding our top prize ($25,000) to: Organophosphate pesticides and other neurotoxicants by Ben Stewart.

Second prizes

We are awarding three second-place prizes ($15,000) for the following submissions. These are listed in no particular order.

Violence against women and girls by Akhil
Sickle Cell Disease by anonymous
Shareholder activism by sbehmer

Honorable mentions

We are awarding $500 to the authors of the following entries. These are listed in no particular order.

We are contacting all prize recipients by email.

Good-faith submissions

Next week we will begin the process of emailing everyone who submitted a good-faith submission in order to offer them participation awards of $200.

Future plans

As we stated in our announcement, this was a trial process for us. We’re grateful to those who sent us feedback and suggestions for how to improve. At this stage, we don’t know if or when we will repeat a process like this. We might write a public update later this year on what we have learned from this exercise and any plans to repeat this or a similar exercise again.

Thank you

We are grateful to Lizka and the other operators of the EA Forum, and to everyone who engaged with or submitted an entry to the Cause Exploration Prizes for making this possible.

221 Reactions

Mentioned in

128"Doing Good Best" isn't the EA ideal

62EA Organization Updates: January 2023

61Why I gave AUD$12,573 to Innovations For Poverty Action

39New positions on Open Philanthropy's Cause Prioritization team (Global Health and Wellbeing)

36EA & LW Forums Weekly Summary (5 - 11 Sep 22’)

Load more (5/7)

More posts like this

190

Open Philanthropy's Cause Exploration Prizes: $120k for written work on global health and wellbeing

ChrisSmith+1 more

268

New cause area: Violence against women and girls

Akhil

116

Cause exploration prize: organophosphate pesticides and other neurotoxicants

Ben Stewart

Comments28

Sorted by

New & upvoted

Click to highlight new comments since: Today at 5:15 PM

Austin3y75

And if you include honorable mentions in the analysis, EA Forum Karma actually did somewhat better. Manifold Markets had 7/10 "winners" (first/second/honorable), while EA Forum Karma had 9/10.

Jon Servello3y6

This makes me interested to know whether EA Forum engagement and the (real or perceived) popularity of these topics among the community contributed to Open Philanthropy's decision-making process.

I am hugely grateful for the opportunity and participation award, but I'm sure like many other contest entrants, I submitted my entry in the belief that my cause area (TB) met and even exceeded the neglectedness, tractability, and cost-effectiveness criteria. The above post suggests to me that maybe other criteria were used. Can entrants expect anything in the way of feedback or some general comments on how entries were selected?

Congratulations to the winners, and thank you to Open Philanthropy for taking this initiative. I would welcome another like it in the future.

ChrisSmith3y11

Jon Servello3y0

Thank you!

Rebecca3y1

Out of curiosity, which parts of Austin’s comment suggests to you something about the criteria used?

Jon Servello3y3

As an entrant, I am very interested to know whether any of the above played a role. Also, Open Philanthropy have mentioned this was an experiment, so it would be great to know whether anything unexpected came up during their decision-making.

Algo_Law3y26

This was a great event which I followed very closely indeed. It generated so much interesting exploration of different areas and I learned so much about the world I live in.

I just want to add that the idea of having 'good-faith' submission prizes was a fantastic addition, and really helped level the playing field for people who otherwise might not have been able to contribute. I heard from a couple of people that they may not have been able to submit without them. I'd love to see more of these in similar contests in future.

Simon Holm3y26

Amazing to see the variation and the effort put into the contributions here - a joy to skim through. Well done everyone!

Ben Stewart3y25

Holy moly thank you! I'm very honoured, especially given there were so many interesting and brilliant submissions. Congratulations to all the other winners!

SiebeRozendal3y24

Could you elaborate what made the top 4 stand out specifically?

Is there any chance that I could get feedback on my submission, which hasn't received a prize? https://forum.effectivealtruism.org/posts/njgRDx5cKtSM8JubL/long-covid-mass-disability-and-broad-societal-consequences

ChrisSmith3y5

Hi Siebe - thank you again for your entry! Unfortunately we don't have capacity to provide feedback on every submission. There were many useful and interesting pieces of work that didn't receive a prize.

SiebeRozendal3y30

Thank you Chris, that's understandable.

How about public feedback on just the top 4 though? Or even just the #1. I find it odd that, in a competition of this scale, no specific reasons are provided for why you picked these winners.

A lot of people put a lot of effort into these reports. Providing reasons why you pick certain winners seems to be like a basic aspect of running a competition in a way that's respectful to participants. This helps participants to compare their own submissions and learn from that. (I think the reward for good faith submissions is a nice contribution to that, and I'm grateful for it, but I don't think it's a replacement)

Rebecca3y1

This is a great point that even just explaining why the submission/s that won did so is useful to other participants

[anonymous]3y15

Looking at the winners/honorable mentions of the prizes - I'm struck by how new most of those mentioned are to the forum.

Of the 26 names above who had a join date next to them, the median date of joining was 2021.

Only 3 people had posted on the forum for more than 5 years.

Of the 20 names I could easily identify as male/female; 9 were female and 11 were male.

Paula Amato3y6

I'm glad to see SCD on the list. But I feel compelled to point out that SCD is 100% preventable with carrier genetic screening and IVF/PGT-M (pre-implantation genetic testing). Thus any strategy to address SCD should definitely include increasing access to IVF/PGT in low resource countries.

Erich_Grunewald 🔸3y2

Thus any strategy to address SCD should definitely include increasing access to IVF/PGT in low resource countries.

That's a pretty bold claim. Are you sure that would be more cost-effective than the newborn screening and treatment intervention proposed in that post? IVF seems pretty expensive compared to the costs of screening and treatment.

Paula Amato3y1

Here is an analysis of Huntington's Disease.

https://pubmed.ncbi.nlm.nih.gov/35618525/

Not sure if a similar one has been done for SCD. But it's not hard to imagine that eradication of a disease in a population is much more cost-effective (especially in the long, long term) than continuing to create billions more people with SCD and treating them with increasingly expensive therapies eg gene therapies (despite what pharmaceutical companies would have you believe).

[anonymous]3y6

Congratulations on running the competition. Reading and judging over 150 essays is no mean feat.

Interested to hear what the next steps will be for the ideas within these submissions, and excited to hear about any new projects or funding streams in these areas.

PeterSlattery3y5

Thank you for working on this, and congratulations to all the winners! Just wanted to mention that I think that it could be good to have a running competition for suggesting new cause areas on the forum, with an annual awards process. Suggesting new causes seems like a valuable activity to prompt and incentivise.

trevor13y3

Karthik Tadepalli3y5

How do you square that with the success of the Center for Pesticide Suicide Prevention in advocating for some pesticides to be banned in dozens of countries? Even if the CPSP wasn't instrumental in all of these cases, it doesn't seem to have been destroyed by the food and farming lobbies.

trevor13y2

Ben Stewart3y3

I think you're right that industry pushback/inertia is a key consideration, and that pesticide suicides are a much more tangible harm to motivate regulation.

However, I think things are much more uncertain than you suggest. Industry have been quite on-board with the related international movement against 'highly hazardous pesticides' and the optics of 'slowly poisoning pregnant women and reducing their kids' potential' may be a compelling narrative for regulatory action.

But I really don't know - I'd be very interested to see what CPSP staff reckon.

trevor13y6

Ben Stewart3y2

Okay fair. And I just remembered something during my research that strengthens your point - there are a few papers critiquing the neurotoxic effect of organophosphates. And if memory serves all of them were authored by and/or sponsored by the pesticide industry. So there is pre-existing industry pushback!

SiebeRozendal3y3

I briefly worked in a research consortium which worked on the EUs precautionary principle, which is under attack by industry: they're basically raising the bar for evidence and definitely push back against regulation a lot.

See eg this: https://corporateeurope.org/en/environment/2018/12/innovation-principle-trap

Karthik Tadepalli3y2

I am not super clear on the delineation between DNT pesticides and suicide-risk pesticides and their relative importance so I'll defer to you.

Karolina Soltys2y2

David Nash and I are organising a lightning talk session where we hope to feature presentations of several new cause ideas from the contest. If you submitted an entry and happen to be in London on 25th October, we'd love you to come along to the event with a short presentation. If you're interested, please get in touch. Here are more details about the event.