Okay, thank you. I aim to take a look at these evals and hopefully learn something and maybe give some useful feedback.

And one more point which maybe is obvious but just to get it out there.

Sure, but these are hard to account for. I agree it's better to adjust the model when it's possible, but you'll still be left with a model that has a tonne of uncertainty.

I agree that a large amount of uncertainty will persist, but I suppose we should aim to do the modeling and adjustments is mean zero. E.g., we'd put in a large adjustment for 'potential non-counterfactuality' for things like "maybe the people who pledged would have pledged later on anyways and the fact that they pledged and donated now means that they're likely to end their pledges earlier."

I suspect that the impact evaluations indeed consider things like these, and I am looking forward to going over them when I have a moment. Thanks for engaging.

I want  to make sure we're talking about the same thing here.  I'd be want to know the cost-effectiveness in terms "for each $1 we spend to promote giving" (via starting new orgs or doing more fundraising) "how much do we raise in truly counterfactual donations to the most effective charities" and I'd want this to be net of any donations or effective work that might be crowded out.  

E.g., suppose Joe lives in the USA and earns $100k per year. Without our spending Joe,  would not give anything to charity and would also not be doing socially-useful work. We spend $1 on ads and this causes Joe (a rich guy) to give $1.50 to The Humane League or The Malaria Consortium, without affecting anyone else's behavior. From the PoV of ~"the EA community" we have earned our $1 back plus gained an additional 50 cents. Again from the global EA community perspective, wouldn't we always want to do this?   

If it's average future that still could justify a 1x bar, depending on what we're averaging over.

I agree with the concerns about uncertainty, displacing less-effective charities, and counterfactuality. But I'd rather see attempts to adjust the estimate for that rather than ~"we're saying 6x but not really, probably lower after considering this". This will help avoid temptations towards soldier/promotion mentality, and make it more comparable to other estimates.

(RE "opportunity cost of the labor of the employees who could otherwise do impactful work or eartn to give, etc" -- if EA people are putting in free labor into these efforts, that should also be factored into the cost estimates, naturally, not just the direct CG investment.)

Thanks, I'll try to gave a look at that and comment (I might have seen it in the past).

What you say about average vs. marginal seems true in principle but

A. "we estimate that our current grantees deliver an average adjusted return on donations of 6x across our effective giving portfolio"

Saying 'deliver' to me present tense implies "deliver and will continue to deliver", suggesting the marginal returns should be comparable.

B. Given the nature of what these funds go for and what these organizations are doing, to me it indeed seems intuitive to expect marginal returns to be fairly similar to the previous returns. 

Starting new ~regional initiatives: Okay, once the markets are saturated for "founding new effective giving orgs in new areas" there should be diminishing returns.  But it would seem like that should already have been picked up by the data from the most recent crop. 

For marketing and advertising activities, I even more expect the returns to perhaps decrease somewhat with future expenditure, but in a sort of gradual, continual way.  

I might be overlooking some aspects of what the organizations and these grants are doing ... But generally, I tend to expect that more money brings diminishing returns, but only gradually diminishing returns. So if Estonia was seen as the 'next most promising target', and founding an organization there had 5x returns, and Latvia is the next one on the priority list, you might expect that to have 4.5x returns. 

 

Very interesting RFP.

Could you share evidence for the

Based on our internal analysis, we estimate that our current grantees deliver an average adjusted return on donations of 6x across our effective giving portfolio, with some grantees as high as ~10x.

This seems surprisingly high in light of other research I have seen on counterfactial returns to donor matches and other standard interventions., and the idea that charities will already want to invest money in efforts to maximize donations, nearly up to the point where an additional dollar invested in marketing yields $1 in additional contributions.

If there is truly this kind of multiplier that would suggest that Effective Giving organizations are dramatically, drastically underfunded.

If an investment in an organization can yield anything over 1x in counterfactual donations, this would be worth funding.

Even more so, if we assume that there are indirect benefits and spillovers to getting people to make effective donations. E.g., Effective giving organizations that convince people to donate to global health charities are likely to be simultaneously convincing them to be more supportive of foreign aid and pro-development trade policies. The things that convince people to donate or pledge for effective animal welfare charities are also likely to convince people to change their diet and support animal welfare legislatio, etc.

I’d add a question around how we can infer the sign of 'how things affect the valence of digitial minds' ... and otherwise, how can digital-mind welfare can be action-guiding at all?

You discuss nearby issues: whether digital minds will be happy by default, whether we can communicate with AIs about preferences, whether we can promise them things positive for wellbeing, and whether self-modification/freedom helps. But I don’t think this fully addresses the deeper crux: even conditional on some part of an AI system having conscious valenced experience, how would we know what makes that experience better rather than worse?

As I suggested in  The "talker–feeler gap": AI valence may be unknowable,  there may be a “talker–feeler gap”: 

A. The part of the system we instruct, bargain with, or ask about preferences may not be the part, if any, that has valenced experience. Or it may not have reliable epistemic access to the welfare-relevant states. This isn't a deception problem. Even a perfectly “honest” reporting subsystem might not know whether the conscious subsystem is made better or worse off.  And its reports may track training objectives, conversational incentives, or preferences rather than welfare.

B. Even if there is valence and the 'decisionmaker' can detect it, the system may be optimized or constrained to act in ways that don't track its own valence. This may be fundamentally baked into the training and development and hard to adjust. 

Either A or B would also make typically proposed solutions less clearly beneficial and even potentially harmful.  If it doesn't have access to the part of the system having balanced experience, asking it about this will not tell us much. And “give them freedom / let them do what they want / avoid what makes them uncomfortable” won't lead to better outcomes if the "decisionmaker in the system" doesn't optimize for the "feeler's welfare." (And it seems as plausible to me as anything else, that having freedom of choice might be painful for the valenced part of a complex system.) 

So I’d suggest adding something like: "Can we ever get reliable, action-guiding evidence about the sign and magnitude of digital-mind valence and how it responds to different requests and outcomes?" Without a bridge from computation, preferences, or self-report to valence, it’s unclear whether potential AI welfare interventions actually improve welfare rather than merely satisfying some behavioral or optimization proxy.

They do and it’s a powerful point. But on the other hand they may be very much unaware of the nature of available tools and solutions. So I think there should probably be some searching — and listening — in both directions. If it’s done in good faith.

The recent forecasting is overrated post got me thinking: 

Solution Seeking a Problem

When talking about forecasting, people often ask questions like “How can we leverage forecasting into better decisions?” This is the wrong way to go about solving problems.

Intuitively, that seems correct, and I've relied on the expression "when you have a hammer, everything looks like a nail." This got me thinking: is it necessarily the wrong way, or is this a truism?

If I have a legitimately useful and powerful tool, isn't it indeed valuable to look around for problems that it can help solve? E.g., if we have discovered a way to harness electricity, shouldn't think about the ways it can be used to improve communication, build labor-saving devices, power factories, etc? If we have something that has demonstrated potential to generate reliable information (supposing that forecasting could do this) shouldn't we look for fruitful opportunities to apply it?

With a set of tools and a set of problems, why is it more useful for one side to do the searching than the other? (Sorry, maybe this is getting too meta and belongs in its own shortform?)

Solution Seeking a Problem
When talking about forecasting, people often ask questions like “How can we leverage forecasting into better decisions?” This is the wrong way to go about solving problems.

I'm reconsidering this point. It seems intuitive, but what is the strongest argument that this is "wrong"?