Is there something like GiveWell for government programs?

aoreidmarr

Is there something like GiveWell for government programs?

aoreidmarr

1 min readDec 20, 2021

Comments 6

Sorted by

New & upvoted

Eevee🔹

The federal government regularly uses cost-benefit and cost-effectiveness analyses to evaluate regulations, but I don't know if anyone has aggregated these into an overall CBA or CEA for the entire government. It's worth a try!

schethik

2020 Economists' Comparisons of Welfare Policies: A few economists (Hendren, Sprung-Keyser) aggregated the results of studies that calculate the impact and costs of various welfare policies (US only). They develop a method to compare the impact of these policies. This paper has a lengthy description of this approach and contains the comparisons themselves (reference Table II. in the appendix for the full list and know that "MVPF" is the primary comparison metric). Hendren and, another economist, Finkelstein provide a less lengthy description here. There's also a great 7-minute explainer video by econimate here (I'd start with this!).
Federal Government CBAs: Additionally, as @evelynciara mentioned earlier, the federal government uses cost-benefit analyses (CBAs) to evaluate regulations. Agencies (e.g. the Department of Energy) conduct these CBAs, which are then reviewed by OIRA (a group housed within OMB). You can find spreadsheets with aggregated results for different regulations here and can explore regulations.gov for specific CBAs.
Other: State and local governments will publish CBAs too (and economists frequently perform CBAs on state/local programs obviously), but I'm not aware of any aggregation of these studies beyond the work by Hendren, Sprung-Keyser, Finkelstein, et al.

|||||

MVPFs and CBAs calculate the ratio of benefits-to-costs (or costs-to-benefits). On the other hand, GiveWell generally calculates "$/life saved" and/or "cost-effectiveness in multiples of cash transfers" for each program they analyze. At first glance, the"cost-effectiveness in multiples of cash transfers" metric appears to approximate a ratio of benefits-to-costs. However, this ratio should not be compared to MVPF calculations by Hendren, nor the US CBAs for multiple reasons. One primary reason they shouldn't be compared is that there are issues comparing cost-benefit ratios of programs that affect different income groups (a friend and I made a post about that issue here).

Happy to provide more info if you're interested (for example, I think there are some limitations to the approach used by Hendren/Sprung-Keyser, but overall I think it's awesome).

Eirik

This is super interesting! The Policy Impacts-project at Harvard University, and their method for evaluating the impacts of public policies that you refer to, the Marginal Value of Public Funds (MVPF), is new to me but seems potentially very valuable!

I recommend any policy-interested EA to check out the links I have provided here (very simple reading), and those from @Schethik above.

I see that the Policy Impacts project have also created a Policy Impacts Library, a database of MVPF estimates for different public policies derived from rigorous empirical research.

aoreidmarr

Thanks for this. I agree, the MVPF approach does seem to be somewhat limiting, but it's got me thinking in the right direction. It seems like the US Federal CBA system is pretty ad-hoc. As far as I can tell each department is responsible for producing their own CBAs, and they each use their own methodology for quantifying things. Do you know if that's right, or is there some standardized methodology that I'm not seeing?

schethik

Yes, you're correct that each department produces its own CBAs. However, the Office of Management and Budget (OMB) does help standardize these CBAs to some extent. For example, the federal government sets up "interagency working groups" to standardize specific inputs (e.g. the social cost of carbon) across departments (more info on social cost of carbon standardization). And, when OIRA reviews CBAs, they make sure departments are following some general standard processes too.

aoreidmarr

I see. I've got some more research to do. Thanks again.

Comments

Curated and popular this week

Hard-to-reverse decisions destroy option value

Stefan_Schubert·9y ago·Curated 4h ago·14m read

This post is co-authored with Ben Garfinkel. It is cross-posted from the CEA blog. A PDF version can be found here. Summary: Some strategic decisions available to the effective altruism m...

If you're agentic, work in biosecurity

sharmaayushmaan🔸·5d ago·7m read

Disclaimer: Although I work on the Groups Team at CEA, I’m writing this in a personal capacity, and this post does not constitute an endorsement by CEA. Agency - the realisation that you really can just do things. TL;DR Biosecurity needs people (of any background) who are agentic and have a high execution velocity and track record....

A Complete Overview of Our AI Safety Talent Pipelines (+ Interactive Visual!)

Roman Ross·6d ago·20m read

Summary Making AI safe and good requires many people to be ready to address crucial problems. However, our current “pipelines” for finding, training, and accelerating talented individuals who could work on these problems are “leaky”, meaning there are many ways they can be improved. This post shows my full overview of the AI safety talent pipeline: where people come from, where they go to learn more, and why this is impactful. I also describe some “leaks” in the pipeline: stages that...

Recent opportunities to take action

Marginal Victories: career advising and opportunities for U.S. democracy preservation & political work

Annika Burman 🔸·3d ago·2m read

I'm stepping down as Hive's Executive Director, and we're hiring my successor

SofiaBalderson, Hive·3d ago·3m read

Starting an EA group @ SUNY Binghamton

micahzarin·2d ago·1m read

schethik

2020 Economists' Comparisons of Welfare Policies: A few economists (Hendren, Sprung-Keyser) aggregated the results of studies that calculate the impact and costs of various welfare policies (US only). They develop a method to compare the impact of these policies. This paper has a lengthy description of this approach and contains the comparisons themselves (reference Table II. in the appendix for the full list and know that "MVPF" is the primary comparison metric). Hendren and, another economist, Finkelstein provide a less lengthy description here. There's also a great 7-minute explainer video by econimate here (I'd start with this!).
Federal Government CBAs: Additionally, as @evelynciara mentioned earlier, the federal government uses cost-benefit analyses (CBAs) to evaluate regulations. Agencies (e.g. the Department of Energy) conduct these CBAs, which are then reviewed by OIRA (a group housed within OMB). You can find spreadsheets with aggregated results for different regulations here and can explore regulations.gov for specific CBAs.
Other: State and local governments will publish CBAs too (and economists frequently perform CBAs on state/local programs obviously), but I'm not aware of any aggregation of these studies beyond the work by Hendren, Sprung-Keyser, Finkelstein, et al.

|||||

Happy to provide more info if you're interested (for example, I think there are some limitations to the approach used by Hendren/Sprung-Keyser, but overall I think it's awesome).