This post has 2 parts, a conceptual overview and an empirical experiment. Feedback on the first and participation in the second is greatly desired! I work at the Ethereum Foundation on funding mechanisms.

Concept

We all know that impact evaluation, especially the sort to quantify outcomes (like GiveWell or Robin Hood), is costly and difficult to scale. A design we have been experimenting with over the last year is conditional prediction markets where traders wager money upon the expected quantified impact of a project if it were to be evaluated.

Here is an example showing how the approach can scale evaluation;

1. GiveWell has 100 outcomes in need of quantification

2. They have staff capacity to properly evaluate only 10 outcomes

3. A market is opened where traders predict the value each of the 100 outcomes would receive if evaluated by GiveWell staff

4. GiveWell resolves the market by releasing the results of the 10 outcomes it did evaluate

5. Traders get profit and loss based on whether they moved market prices of those 10 outcomes closer or further away from GiveWell evaluations.

Although only a subset were evaluated, we can use the market prices on all items as a proxy for their value!

We also saw the value of running a data science challenge (like Kaggle) that gets resolved with the same data, as a way to lower the barrier & potentially use the evaluations by the winning models .

I'd love feedback on this approach! Trading on the results of philanthropic evaluation seems like the kind of thing right up the EA alley.

Experiment

Funding open source dependencies is good in theory, messy in practice. Especially with laws like the Cyber Resilience Act coming into effect in the EU from 2027 onwards, it's critical we figure out easier ways to allocate money between various dependencies.

Accordingly, we started a prediction market (and data science competitions) to guess the impact of repositories as judged by experts, at 3 levels;

1. The relative value between 98 open source repos to Ethereum

2. How important dependencies are for each of these 98 repos

3. The relative value between 3,677 open source dependencies for each of the 98 repos

The prices of repositories in these 3 markets collectively decide how $350,000 gets allocated between all 3,800 dependencies. A trial version of this mechanism with 628 rows of evaluator data comparing 45 repos with one another is accessible here

The ask

1. Participate in the data science competitions and win glory + $20,000 in prizes from the Ethereum Foundation and Gitcoin if you land in the top positions on the leaderboard
    
2. Upload your predictions to deep.seer.pm, whose market prices will be used to distribute $350,000 to open source repositories. 
   
   If you have any questions, you can join our telegram group here. A more detailed writeup is also available here.