READ ME FIRST: This document is being made public for the purposes of openness and to general comments, critique, and discussion. It is unfinished at this time. If you leave a substantive comment, have suggestions, etc, please leave your name in the comments so you can be credited with your contribution.
In order to make this fair (i.e. not “cheating” by eliciting comments) and to encourage others to collaborate without reservation, I am forgoing my portion of any prize money that may be received from this essay (save for a reasonable time reimbursement for working time on this). Any contributors and collaborators have a say in which charities the remaining portion of the prize money will go. The goal is simply to make the best essay possible to address these issues by encouraging many contributors, regardless of winning or losing a prize money. The important part is making change.
General request for comments: If you suggest an addition, please also suggest 2x that amount I should cut or trim from somewhere else.
Suggestion was deleted
Show more
Show less
Comment details cannot be verified
0 replies
New
Sandy Buchanan
Nov 20, 2022
Approver
Delete space
Suggestion was deleted
Show more
Show less
Comment details cannot be verified
0 replies
New
You're suggesting
Gemini created these notes. They can contain errors so should be double-checked. How Gemini takes notes
Drag image to reposition
11
10
9
8
7
6
5
4
3
2
1
1
2
3
4
5
6
7
8
9
10
Outline
Outline
Document tabs
GiveWell's Uncertainty Problem manuscript
2
Headings you add to the document will appear here.
Deprecated, please use live version here: https://www.metacausal.com/givewells-uncertainty-problem/
GiveWell’s Uncertainty Problem
Background
Conceptual underpinnings
GiveWell’s decision-making framework
Sources of uncertainty in cost-effectiveness modeling
Selection in aggregate
Baseline scenario
What happens if the amount of uncertainty and the true value of the program are related?
Summary
Is this a problem for GiveWell’s models? A review of uncertainty in GiveWell models and demonstration of probabilistic sensitivity analysis
Uncertainty compounds rapidly and complicatedly
A brief infrastructure and workflow interlude
Potential methods and frameworks for addressing uncertainty
Option 1) Just look at the uncertainty
Option 2) Use a lower bound of the uncertainty interval
Option 3) Use a probability-based threshold with a discrete comparator
Option 4) Use a probability-based threshold with a distributional comparator
Alternative options
Which option is best depends on what question(s) you want to answer
Additional benefits of uncertainty-forward models
Relationship to other common EA critiques
How an uncertainty-forward modeling framework changes model decision-making
Error-checking opportunities
Clearer communication with donors about the reliability of estimates