Nice! Though I guess my concern that someone would conclude the program hadn't work, after failing to reject the null, is still there. But if the proposed study goes ahead, I'm sure your quick input could be very helpful!

I really like that you are digging into this! 2 quick questions:

• I could imagine the impact of talent interventions could be quite long tailed (e.g. perhaps 1 person in a cohort of 60 contributes ~half of the impact), might that make your proposed RDD approach understate impact, since it really just tells you the impact on the marginal accepted applicant (rather than the super impressive applicant who was a slam dunk)?
• I agree that the application scores (assume that is indeed a thing) make the RDD approach straightforward, but how would you measure impact post-program, especially for the comparison group that might be hard to follow up with? I'm imagining here that, say, an AIS bootcamp greatly increases someone's AIS knowledge but not in a legible way that would show up in, say, their resume -- developing an assessment tool that captures this impact might be difficult (but also not something I've given serious thought to, and perhaps you have!)

Lastly just a statistical power flag: you link to LawrenceC's piece on variation in application assessments. That variation would suggest we'd need quite a good sample size to allow for precision in the detectable effect, which might be hard. I guess that's just a note to make sure some sample calculation is done once you have clarity on your outcome of interest to make sure you have a path to measuring the kind of effect that would make these programs cost-effective. The worst case scenario would be a study with 100 people looking to detect an impact of, say, 0.05 SD, when you are in fact only powered to detect 0.3 SD and basically doomed to concluding the program doesn't work, when perhaps it does. 

These are intended as genuine questions to see if your ideas could work, if there is indeed a path to a good study here I'd love to see something proposed to, say, the EA Infrastructure Fund, your ideas seem valuable! (Disclaimer, I work at CEA, which contains the EAIF, but I am not involved in EAIF decisions and these are personal takes)

Good catch! I've updated the title for now to be the title of the podcast episode they are promoting (using my Immense Forum Admin Powers)

I love that! Thanks for the example. 

Congrats on the great sign up results! Do you mind giving some more examples of what you did differently? Those could be helpful for other groups to learn from. 

For instance, you noted "For example: you walk past a table and someone goes “Hey! Do you want to join the EA club?” " as an example of what you wouldn't to-- what would you do to get attention when tabling? 

Sounds like you have a super relevant background! Some quick thoughts:

• Try ~replicating some of GiveWell's output, such as a cost-effectiveness model. Not entirely from scratch, that would take forever, but perhaps go through it and test your own ability to explain each step, which might help you identify gaps in your knowledge (e.g. some of the moral weights thinking can be counter-intuitive in my experience)
• Try picking a Charity Entrepreneurship graduate charity and write your own GW-style evaluation. CE stands out as their charities will likely publish a lot of the data you might need, but they also might not have public GW evaluations yet

Both of those might just help you realize which parts of GW's process you don't really "get", so you know what to work on. 

But I have never worked at GW, so take all of this as "one guy's opinion" :) 

Interesting suggestion! I think the example you outline makes sense. I'd guess part of why you don't see this kind of thing in EA might include:
 

• It might not be a very neglected field: there are already a lot of scholarships and student loan companies/government policies. That might reduce the counterfactual value of the loan if the person might have gotten funding elsewhere.
• The logistics of setting up a system to evaluate these opportunities could be significant, especially relative to the size of the loans. You might need to thoroughly investigate each borrower to be confident that they will not only repay but also are sincere in their donation plans.
• Building on my 2nd point: I think it would be super easy to game a program like this and say "Of course I plan to donate $1m+ once I become a doctor" just so you get the loan. While the repayments might be legally binding, the obligation to donate will not be, so the impact side could weaken.

Anyway, just some quick thoughts, very open to counter takes and thank you for suggesting this idea! I do think there is a good place for some smart finance type stuff in EA for sure, such as loans for social enterprises to kick off, or other clever things like advance market commitments. 

Wow this is so well done. I'm really excited to see what else you have in the pipeline, and more generally for high quality EA content!

Thanks Vasco! I think it depends on how well your particular system is a strict A > B> C> Done flow. If it's as linear as car production, then it is indeed that case that if you make 100 wheels, 20 front axles and 40 windshields per day, you have made 20 cars (your bottleneck is the axles). And making 400 wheels the next day without changing your axle output has zero effect on the number of cars produced, the additional "productivity" is entirely wasted.

Not every process will be quite so linear, but at least for the elements that are, I do think that increasing the output of non-bottlenecks will have zero effect on the output you care about.

In that case it was just waiting for USDA approval of a pilot of the vaccine (it was a vaccine for chickens). Our best guess was that the approval was another, say, 2 months out, so we could just let the vaccine pilot batch take a bit longer to make and it wouldn't affect when the pilot itself started (we could make the vaccine before it was approved, we just couldn't actually use it).