All of EdoArad's Comments + Replies

If you or someone else wants to expand on this point, I'd be interested in understanding whether that's really the case or in which ways it got better or worse

1
Judith
2mo
The AIM burgundy red is basically the same as the old CE red for easier re-identification; the other colours we slowly added on last year and this year and what can I say, we do like us some magical, natural, but clearly distinct colours for our different programs :)
1
EdoArad
2mo
wait, was it like that before?? 

Is the issue highly polarized/politicized in Germany? Say, do you expect some people to engage less with your work as a result of "choosing sides"?

In EA Israel we tend to take more politically neutral approaches when engaging publicly, including in relation to the current war or to the large "pro-democratic" public protests (wiki). I'm curious about your attitude to reputational risks and the potential problem of "painting EA with leftist colors".

Germany so far is probably less polarized than other countries (e.g. the US). Currently there is only one far right party with significant reach (AFD) currently polling around 19% nationally.

The main conservative party (CDU) and the conservative leaning economically liberal party (FDP) are currently clearly and credibly distancing themselves from the AFD. We even embedded an interview from one of the most prominent FDP members (Gerhart Baum) in one of our blog posts about how we think about defending democracy.

So while we still might risk our rep... (read more)

Two considerations:
1. does protecting democracy have to be "painting with leftist colors"?
2. even if it was, does the ROI justify it?

On the first, as noted in this EAGxVirtual lightning talk I gave on the US context, the design of the political system is a big upstream cause of authoritarian voter bases and to a larger extent authoritarian politicians. Many of the reforms to the US political system that would in the short to long term reduce the antidemocratic threat are "bi-populist," as I like to call it.

Left- and Right-wing populists are generally for c... (read more)

Usually we use purchase-parity adjustments when we look at the value of money, so there might be a subtle circular reasoning problem here. One issue for example is that there is no value in innovation (say, by designing a cheaper transportation service/product, people could end up paying less for the service so the revenue may decrease [while the profits could get higher]). 

Another issue is that I don't think this could take into account counterfactual impact. 

Didn't read through the whole thing yet, sorry if I missed how you addressed this issues! I love the idea of trying to find simpler ways of estimating impact!

2
Christoph Hartmann
2mo
Hm, not sure. If there is an opportunity for innovation, I'd expect either the incumbent to pursue this to expand the addressable market (and thereby make more revenue / have more impact) or/and a competitor to innvoate, thereby reducing the price and capturing market share / prevent the incumbent from increasing profits (and increasing revenue / imapct for the competitor). On second reading I assume you are referring to the issue that when a product gets cheaper through innvoation it might look like the product would be less impactful because it now gets less share of the total WELLBYs of the customer. I guess, though, what would happen at the same time is that overall life satisfaction of the customer will go slightly up as they now have more disposable income (just saved some money from spending less on that product), and that increased life satisfaction would be distributed across all purchases, including the one that just got cheaper. On a micro level those won't perfectly balance of course but on the coarsness level of this analysis I think we'd be fine - see the section on first dollar vs last dollar spent in Appendix 1. But I'm not an economist or anything like that by training so very curious about your further thoughts! I very likely missed things. I addressed the counterfactual impact a bit in Appendix 1 in the section on absolute vs relative impact.

Also, CCC are using a discount rate of 8% (compared with 4% that's common at least for CE)

How should we compare their (CCC's) cost-benefit estimates to GiveWell's (GW's) results?

I quickly spot two differences:

  1. GW's benefit estimates are in units of doubling of consumption (interventions are compared to direct cash transfers through Give Directly, itself measured in terms of doubling of consumption), whereas CCC's estimates are in units of value of a statistical life.
  2. CCC's analyses are for large-scale government operations, so 
    1. they don't account for counterfactual investments,
    2. they encounter diminishing returns (compared to GW's focus on the
... (read more)
2
Vasco Grilo
3mo
Thanks for commenting, Edo! I think which metric to pick will depend on one's preferred heuristic for contributing to a better world (somewhat related draft; comments are welcome): * Units of doubling of consumption appear more strictly connected to improving nearterm welfare (as proxied by e.g. quality-adjusted life years). * Units of value of a statistical life, roughly proportional to units of consumption, appear more strictly connected to economic growth (as proxied by e.g. gross world product). The papers focus on large scale interventions with massive benefits, so the counterfactual benefits will be pretty small in comparison? There is a chance governments or international organisations go ahead with the interventions funded by GiveWell, but a seemingly much smaller chance that a big government investment has no counterfatual impact due to the potential intervention of other actors? Your BOTEC looks good to me. GiveDirectly's unconditional cash transfers have a benefit-to-cost ratio of 2.4, and GiveWell's cost-effectiveness bar is 10 times that, which suggests a bar for the benefit-to-cost ratio of around 24 (= 10*2.4). So one should expect the cost-effectiveness of the interventions supported by GiveWell to have benefit-to-cost ratios in that ballpark. I agree a more careful comparison would be useful.
4
EdoArad
3mo
Also, CCC are using a discount rate of 8% (compared with 4% that's common at least for CE)

Totally agree with the need for a more balanced and careful analysis!

The main point is that access management is more natively associated with the structure of the model in software settings. Say, you are less likely to release a model without its prerequisites.

But I agree that this could also be messed up in software environments, and that it's mainly an issue of UI and culture. I guess I generally argue for a modeling environment that is "modeling-first" rather than something like "explainable-results-first".

I have a very rough draft based on work with @Michael Latowicki and Charity Entrepreneurship (@Filip_Murar and @weeatquince) where we tried various tools for modeling uncertainty for use in their CEAs. Due to personal issues, I'm not sure when I'll finish it, so here it is (very unpolished and not finished).

Mainly, we list various tools we found with some comments on them and discuss the pros and cons of spreadsheet vs software solutions. The latter is mostly coherent so I'll paste it here:

Spreadsheets vs programming languages

The tools we found for probabi... (read more)

3
JoshuaBlake
3mo
I don't think this evaluation is especially useful, because it only presents one side of the argument. Why spreadsheets are bad, not their advantages or how errors typically occur in programming languages. The bottom line you present (quoted below) is in fact not very action relevant. It's not strong enough to even support that the switching costs are worth it IMO.
7
Filip_Murar
3mo
To add, we at Charity Entrepreneurship have been experimenting with using Dagger/Carlo for our cost-effectiveness analyses of new charity ideas. We've put together this (very rough, work-in-progress) guide on how to use Dagger – sharing in case others find it helpful.
4
MichaelStJules
3mo
I've personally found Guesstimate less error-prone and easier to verify and I'd guess easier to use in general than Google Sheets/Excel. Node names+acronyms and the displayed arrows between nodes are helpful. I'd also imagine Guesstimate would beat programming languages on these, too, with fewer places for error per variable or operation. However, Guesstimate is often not flexible enough, or takes a lot of nodes to do some simple things (e.g. sampling randomly from one of multiple variables). It can get very slow to edit with many (like 100 or more) nodes. It can also be more tedious for simple operations over many variables at a time, like a sum, IIRC. (Of the options you've listed, I've only used Guesstimate and Google Sheets (without the probability stuff). I was also a deep learning engineer for ~2 years.)

[btw, this is a common problem when using spreadsheets rather than when modeling in a software development environment - the software space has a lot of experience in working in (partially-) open source settings]

2
JoshuaBlake
3mo
Could you expand on this please? Isn't this going to be roughly equivalent to "we kept our GitHub repo private"?

Oh! I see that you've also posted Shrimp Neuroanatomy. Great, I'm looking forward future posts on this topic to inform the debate :) 

I don't agree with your take, but many people in EA (and definitely outside it) feel similarly to you so I appreciate you writing this. It's also a good time to criticize invertebrate welfare, as more attention and funding are directed there.

That said, I found the tone of this post too confrontational to promote useful discussion so I've decided neither to upvote or downvote. I know it's a tough ask, but it would also be very useful to directly engage with works in this space (say, recent posts on invertebrate welfare or RP's sequence) and clearly outline ... (read more)

5
EdoArad
3mo
Oh! I see that you've also posted Shrimp Neuroanatomy. Great, I'm looking forward future posts on this topic to inform the debate :) 

I like this as an example of a case where you wouldn't want to combine these two different forms of uncertainty

Yeah. If I understand correctly, everything is resampled rather than cached, so comparing results between two models is only done on aggregate rather than on a sample-by-sample basis

2
Agustín Covarrubias
4mo
We used to have a caching layer meant to fix this, but the objective is for there not to be too much inter-run variability.

If I understand correctly, all the variables are simulated freshly for each model. In particular, that applies to some underlying assumptions that are logically shared or correlated between models (say, sentience probabilities or x-risks).

I think this may cause some issues when comparing between different causes. At the very least, it seems likely to understate the certainty by which one intervention may be better than another. I think it may also affect the ordering, particularly if we take some kind of risk aversion or other non-linear utility into accou... (read more)

2
Agustín Covarrubias
4mo
With "variables are simulated freshly for each model", do you mean that certain probability distributions are re-sampled when performing cause comparisons?

These are beautiful! Thanks for your generous offer to the community, this seems very valuable!

Answer by EdoAradNov 12, 20232
1
0

I have a very limited understanding of this field, so the following is probably a bit wrong but hopefully leading in the right direction.

In the US (and maybe some other countries) there is a regulated energy demand & supply prediction market. This is set up at various points in the electric grid, which is connected to many different suppliers of both renewable and non-renwable sources of electricity.

The coal power plants are required to be able to deliver enough electricity so that there is a low chance of power outage. The bound on that chance is defi... (read more)

Thanks! Yeah, I totally agree - the topic is surprisingly delicate and nonintuitive, and my examples above are too technical. 

By the way, I'd love it if other people would write posts that make the exact same point but better! (or for different audiences). 

I think in this thread there are two ways of defining costs:

  1. Michael considers the cost as the total amount spent
  2. Stan suggests a case where the cost is the amount needed to be spent per unit of intervention

I think this is the major source of disagreement here, right?

This discussion resembles the observation that the cost-effectiveness ratio should mostly be used in the margin. That is, in the end we care about something like  and when we decide where to spend the next dollar we should compute the derivatives with ... (read more)

Things to add:

  1. Graphical example
  2. Explicit discussion of GiveWell's use of cost/effect 
  3. Real-world examples where this causes a significant error 
  4. Expand on the lognormal example, discuss cases where the variance is in the same order of magnitude as the expected value
  5. What this says about how we should model uncertainty
2
Lorenzo Buonanno
5mo
From my experience last year writing the post you linked to (which I now just edited), I would really dumb things down and try to make the graphical example as simple as possible for people with ~high-school math knowledge (like me) Several smart people I asked for advice about that post seemed to find reasoning about this not trivial, gave somewhat contradicting answers, or thought that the solution I had thought of was too obvious to post, despite it being very wrong (as you explained in the comments last year, thank you!) Even after a year, tbh I'm not sure I "grok" it completely (see the appendix I added to the post. But I reassure myself with the fact that costs very rarely span across orders of magnitude, and in those cases, I could try to model E(value) directly rather than E(value/cost) as a proxy).

@Aaron Gertler has some experience fundraising from his MTG stream and may want to chime in :)

I like the distinction of the various kinds of risk aversions :) Economists seem too often to conflate various types of risk aversion with concave utility.

I feel a bit uneasy with REV as a decision model for risk aversion about outcomes. Mainly, it seems awkward to adjust the probabilities instead of the values, if we care more about some kinds of outcomes. It doesn't feel like it should be dependent on the probability for these possible outcomes. 

Why use that instead of applying some function to the outcome's value?

Hi Edo,

There are indeed some problems that arise from adding risk weighting as a function of probabilities. Check out Bottomley and Williamson (2023) for an alternative model that introduces risk as a function of value, as you suggest. We discuss the contrast between REV and WLU a bit more here. I went with REV here in part because it's better established, and we're still figuring out how to work out some of the kinks when applying WLU.

Nice spot! I think it should be 

(found reference here).

Then, we get 

Joshua Greene recently came to Israel to explore extending their work aiming at bridging the Republican-Democrat divide in the US to the Israel-Palestine conflict. A 2020 video here.

I like the new linkpost design! It makes me think of the potential of linkposts on the forum. 

One possible vision would be to have the forum be a centralized place to discuss any source on the web. Some suggestions to that end:

  1. Develop a simple chrome extension to publish a linkpost directly from a page to the forum.
  2. Merge all linkposts that link to the same page. (Say, multiple sections of posts and then a common comment section)
  3. Make sure that no more than x% of the frontpage posts are linkposts.
  4. Let people/orgs claim ownership of linkposts to their content.

The links to the spreadsheets don't work (maybe it isn't publicly shared?)

2[anonymous]6mo
Yeah I've realised my old google account where that was kept has been deleted. I'll figure out whether I can get it back somehow

Do you mind sharing your main consideration against releasing it? Not trying to push back, but rather to understand this as I'm considering working on related topics 

That's really exciting! Do you plan on open-sourcing the project?

Maybe. We're a little unsure about this right now. The code base for this is part of the bigger Cross-Cause Cost-Effectiveness Model which we haven't made a final determination on whether we will release it.

Answer by EdoAradSep 25, 20234
1
0

In case you arn't aware of it, the idea of approval voting is being promoted strongly in the Center for Election Science. They have some relevant resources here.

I also remember an 80k episode with Aaron Hamlin on the topic. Yeah, from 2018.

1
pleaselistencarefullyasourmenuoptionshaverecentlychanged
6mo
Yes—has anyone said why they eliminated all the other methods?

Value monism is the thesis that there is only one intrinsically valuable property.

(I didn't know this term)

Maybe controversial is too strong a word. I think EAs tend to think on the margin and are also less concerned with inequality or with non-existential risks from advanced tech. 

Answer by EdoAradSep 14, 20238
3
0
1

I find Jaron Lanier's thoughts on the impacts of tech development interesting and probably controversial in EA circles. 

Daron Acemoglu is a known growth economist, and he has recently published a new book and had an interesting episode with Rhys Lindmark. 

I also remember following Vinay Gupta a while back, and that he had interesting opinions on civilizational resilience (say, here). He was also involved with Ethereum, and is doing blockchain stuff, not sure what.

Generally, I'm interested in more discussion with people in communities  or ide... (read more)

3
hmijail
6mo
I'd also be very interested in both Lanier and Gupta's views. But, could you elaborate on why you'd expect Lanier to be "probably controversial in EA circles"?

(Sharing this because I'm uncertain and would be interested in thoughts/pushbacks)

Thank you for sharing this!

My answer to the survey's question "Given the growing salience of AI safety, how would you like EA to evolve?":

I think EA is in a great place to influence the direction of AI progress and many orgs and people should be involved with this project. However, I think that many people in this forum think that the most important outcome of the EA community is by influencing this technology, and I think this is mistaken and misleading.

The alternative would be to continue supporting initiatives in this space, including AI safety-specific... (read more)

2
EdoArad
7mo
(Sharing this because I'm uncertain and would be interested in thoughts/pushbacks)

Some ideas held by many EAs - whether right or wrong, and implied by EA philosophy or not - encourage risky behaviour. We could call these ideas risky beneficentrism (RB), and they include:

i. High risk appetite.

ii. Scope sensitivity

iii. Unilateralism

iv. Permission to violate societal norms. Violating or reshaping an inherited morality or other “received wisdom” for the greater good.

iv. Other naive consequentialism. Disregard of other second-order effects

2
Chris Leong
7mo
Thanks, I missed that!

Is the game something like "EA online discussion norms" and the strategy that you are proposing something like "make your writings independent of the EA forum, and allow for competing discussion spaces on your posts"? 

3
NunoSempere
7mo
Mmh, probably, but I was thinking about it less abstractly, e.g., CEA is offering the EA community a forum with such and such characteristics, the EA community responds by xyz, etc. Yes, but not "competing", such that either option is ok, but "separate", such that the EA forum doesn't have to have a role to play. For example, this blogpost of mine: https://nunosempere.com/blog/2023/07/19/better-harder-faster-stronger/ has a rich comment section, and it didn't need the EA forum to have it.

I know the author personally, and I want to signal that Michelle is genuinely interested in doing good, that the proposed technology does seem highly promising, and that an informed take on this question would help her make a better decision. 

Thanks for anyone experienced with the subject matter who is willing to help here!

Sure! Looking into GiveWell's main CEA, and their analysis of AMF (link), the location granularity is by country. However, AMF prioritizes their distribution location based on malaria prevalence rates and operational partners, so the results might be much better than the country's average.

Regarding the first question, I just briefly looked again at the report and indeed I don't see that explicitly taken into account. I only vaguely remember thinking about that, and I'm not sure how that was resolved. 

I think the main causal pathway they used in their report is QALY gains from people quitting smoking, so that sounds like it wouldn't change drastically if the intervention was delayed by, say, a couple of years. So I agree that this is a good question to look into further, and I expect that could indeed reduce the cost effectiveness by 3x-10x. Great catch David!

I think cost-effectiveness analyses may generally be under-evaluating interventions due to the possibility of optimizing some parameters of the intervention. Such optimization could be accounted for in prospective quantitative CEAs by running some optimization algorithm, but I'm not sure how useful/accurate the result of that would be.  

4
Guy Raveh
7mo
Could you give an example for an intervention you think might have been undervalued?

FEM developed a new technology to evaluate our impact: a transmitter that identifies when FEM’s content is on air, and replaces it with music. This means the region surrounding the transmitter is temporarily not exposed to our programmes.

This is SO COOL! 

Really amazing progress all around 👏

That's cool. I had the thought of developing a "personal manager" for myself of some form for roughly similar purposes

It's funny, I think you'd definitely be in the list of people I respect and care about their opinion of me. I think it's just imposter syndrome all the way up.

Personally, one thing that seemed to work a bit for me is to find peers which I highly appreciate and respect and schedule weekly calls with them to help me prioritize and focus, and give me feedback. 

How would you use AI here?

2
Ozzie Gooen
7mo
A very simple example is, "Feed a log of your activity into an LLM with a good prompt, and have it respond with assessments of how well you're doing vs. your potential at the time, and where/how you can improve." You'd be free to argue points or whatever. 

I'm not sure, what do you think you should learn?

Load more