[Linkpost] An update from Good Ventures

I've never seen anyone express frustration or disappointment about Dustin, except for Habryka. However, Habryka seems to be frustrated with most people who fund anything / do anything that's not associated with Lightcone and its affiliates, so I don't know if he should count as expressing frustration at Dustin in particular.

There is a huge amount of work I am deeply grateful for that as far as I can tell is not "associated with Lightcone and its affiliates". Some examples: 

• The historical impact of the Future of Humanity Institute
• The historical impact of MIRI 
• Gwern's writing and work
• Joe Carlsmith's writing
• Basically all of Holden's intellectual output (even if I disagree with his leadership of OP and EA in a bunch of very important ways)
• Basically all of Owen Cotton Barratt's contributions (bar the somewhat obvious fuckups that came out last year, though I think they don't outshine his positive contributions)
• John Wentworth's contributions
• Ryan Greenblatt's and Buck Shlegeris's contributions
• Paul Christiano's and Mark Xu's research (I have disagreements with Paul on EA leadership and governance things, but I think his research overall has been great)
• Rohin Shah's many great contributions over the years
• More broadly the Deepmind safety team
• Zach Stein Perlman's work on carefully analyzing lab policies and commitments

There are also many others that I am surely forgetting. There is an enormous number of extremely talented, moral, and smart people involved in the extended rationality/EA/AI-x-risk ecosystem, and I am deeply grateful to many of them. It is rare that my relationship to someone is purely positive and completely devoid of grievance, as I think is normal for relationships, but there are many people for which my assessment of their good vastly outshines the grievances I have.

I'd say from a grantee pov, which is I guess a large fraction of the highly-engaged EA community (eg commenters on a post like this), Dustin/GV/OP have mostly appeared as an aggregate blob - "where the money comes from".  And I've heard so much frustration & disappointment about OP over the years!  (Along with a lot of praise of course.)  That said, I get the spirit of your comment, I wouldn't want to overstate how negative people are about Dustin or OP.

And for the record I've spent considerable energy criticizing OP myself, though not quite so far as "frustration" or "disappointment".

As the author of the comment linked for "criticizing OP for departing grantees too quickly," I'd note that (in my pre-Forum days) I expressed concern that GiveWell was ending the Standout Charity designation too abruptly. So I don't see my post here expressing potential concern about OP transitioning out of these subareas as evidence of singling out OP for criticism.

No, the farm animal welfare budget is not changing, and some of the substreams GV are exiting (or not entering) are on the AI side. So any funding from substratgies that GV is no longer funding within FAW would be reallocated to other strategies within FAW (and as Dustin notes below, hopefully the strategies that GV will no longer fund can be taken forward by others).

Re: attack surface in my early comment, I actually meant attacks from EAs. People want to debate the borders, quite understandably. I have folks in my DMs as well as in the comments. Q: "Why did we not communicate more thoroughly on the forum"
A: "Because we've communicated on the forum before"

I don't think endorse vs. not endorse describes everything here, but it describes some if it. I do think I spend some energy on ~every cause area, and if I am lacking conviction, that is a harder expenditure from a resource I consider finite.

An example of a non-monetary cost where I have conviction: anxiety about potential retribution from our national political work. This is arguably not even EA (and not new), but it is a stressful side hustle we have this year. I had hoped it wouldn't be a recurring thing, but here we are.

An example of a non-monetary cost where I have less conviction: the opportunity cost of funding insect welfare instead of chicken, cow, or pig welfare. I think I could be convinced, but I haven't been yet and I've been thinking about it a long time! I'd much prefer to just see someone who actually feels strongly about that take the wheel. It is not a lot of $s in itself, but it keeps building, and there are an increasing number of smaller FAW areas like this.

I failed to forecast this issue for myself well when we were in an expansionary mindset, and I found that the further we went, the more each area on the margin had some element of this problem. I deferred for a really long time, until it became too much. Concurrently, I saw the movement becoming less and less appealing to other funders, and I believe these are related issues.

I feel quite aligned with you spiritually on this topic: we both want more diversified funding for these causes, and see Good Ventures-as-sole-funder both getting in the way of that outcome and creating a variety of second order problems. I'm sorry to hear that OP may have actively dissuaded you from pursuing diversification. Folks like you, as well as OP staff, will now spend more of their time pursuing other funding sources, and I expect that to be strongly positive for the causes and the general EA project over the long run.

In the short run, I agree we have recently lost valuable time and energy via this path, and I regret that and accept a good deal of responsibility. When FTX showed up on the scene, it felt like plurality was happening, and as soon as they went away, I became quite fixated on this topic again.

Thanks for clarifying! That sounds like a pretty unpleasant experience from a grantee perspective, I'm sorry that happened.

I would be very surprised if digital minds work of all things would end up PR-costly in relevant ways. Indeed, my sense is many of the "weird" things that you made a call to defund form the heart of the intellectual community that is responsible for the vast majority of impact of this ecosystem, and I expect will continue to be the attractor for both funding and talent to many of the world's most important priorities. 

An EA community that does not consider whether the minds we aim to control have moral value seems to me like one that has pretty seriously lost its path. Not doing so because some people will walk away with a very shallow understanding of "consciousness" does not seem to me like a good reason to not do that work.

I think you absolutely have a right to care and value your personal reputation, but I do not think your judgement of what would hurt the "movement/EA brand" is remotely accurate here.

Your second statement is basically right, though my personal view is they impose costs on the movement/EA brand and not just us personally.... I hope to see everything funded by a more diverse group of actors, so that their dollar and non-dollar costs are more distributed.

Do you think that these "PR" costs would be mitigated if there were more large (perhaps more obscure) donors? Also, do you think that "weird" stuff like artificial sentience should be funded at all or just not by Good Ventures?

[edit: see this other comment by Dustin]

>> If they endow a re-granter that funds something weird, they can say "well the whole point of this endowment was to diversify decision-making; it's out of our hands at this point".

I proposed this myself at one point, and the team politely and quite correctly me informed me that projecting this response from critics was naive. We are ultimately responsible for the grants downstream of our decisions in the eyes of the world, regardless of who made intermediate decisions.

As an example of how this has played out in practice, we're known (and largely reviled) locally in San Francisco for supporting the controversial DA Chesa Boudin. In fact, I did not even vote for him (and did not vote in the recall), and the only association is the fact that we made a grant in 2019 to a group that supported him in later years, for totally unrelated work. We made a statement to clarify all this, which helped a little on the margin, but did not substantially change the narrative.

Yes, I think funding an endowment would have been a great thing to do (and something I advocated for many times over the years). My sense is that it's too late now.

I wrote at length about my views on epistemic confidence here https://medium.com/@moskov/works-in-progress-the-long-journey-to-doing-good-better-9dfb68e50868

Nice points, Ozzie! For reference, Alex wrote:

As CEO, I [Alex] work more closely with Cari Tuna and Dustin Moskovitz [@Dustin Moskovitz], our primary funders and founding board members, than I had in the past. Dustin and especially Cari were very involved at the founding of Open Philanthropy — our grant approval process in the very early days was an email to Cari. But their level of day-to-day involvement has ebbed and flowed over time. Cari, in particular, has recently had more appetite to engage, which I’m excited about because I find her to be a wise and thoughtful board president and compelling champion for Open Philanthropy and our work. Dustin has also been thinking more about philanthropy and moral uncertainty recently, as reflected in this essay he posted last month.

It’s worth noting that their higher level of engagement means that some decisions that would have been made autonomously by our staff in the recent past (but not in the early days of the organization) will now reflect input from Cari and Dustin. Fundamentally, it has always been the case that Open Philanthropy recommends grants; we’re not a foundation and do not ultimately control the distribution of Cari and Dustin’s personal resources, though of course they are deeply attentive to our advice and we all expect that to continue to be the case. All things considered, I think Cari and Dustin have both managed to be involved while also offering an appropriate — and very welcome — level of deference to staff, and I expect that to continue.

I asked Alex about the above 3 months ago:

Could you elabotate on the influence of Cari and Dustin on your grantmaking (see what I have highlighted below [above]), ideally by giving concrete examples?

There was no answer.

Not sure if this helps, but I currently believe:
1. Relatively little or no AI suffering will happen, pre-AGI.
2. There's not going to actually be much lock-in on this, post-AGI.
3. When we get to AGI, we'll gain much better abilities to reason through these questions. (making it different from the "figuring out animal welfare" claim.

I think paying AIs to reveal their misalignment and potentially to work for us and prevent AI takeover seems like a potentially very promising intervention.

I'm pretty skeptical of this. (Found a longer explanation of the proposal here.)

An AI facing such a deal would be very concerned that we're merely trying to trick it into revealing its own misalignment (which we'd then try to patch out). It seems to me that it would probably be a lot easier for us to trick an AI into believing that we're honestly presenting it such a deal (including by directly manipulating it's weights and activations), than to actually honestly present such an deal and in doing so cause the AI to believe it.

Further, I think there is a substantial chance that AI moral patienthood becomes a huge issue in coming years and thus it is good to ensure that field has better views and interventions.

I agree with this part.

paying AIs to reveal their misalignment and potentially to work for us and prevent AI takeover

I'm hoping this doesn't happen anytime soon. This assumes that AI would themselves own property and be seen as having legal persons or similar. 

My strong guess is that GPT4 to GPT7 or so won't have much sentience, and won't have strong claims to owning property (unless weird political stuff happens). 

I'm sure it's theoretically possible to make beings that we'd consider as important to humans or more, I just don't expect these LLMs to be those beings. 

I am skeptical that a new large philanthropist would be well-advised by doing their grantmaking via OP (though I do think OP has a huge amount of knowledge and skill as a grantmaker). At least given my current model, it seems hard to avoid continuing conflict about the shared brand and indirect-effects of OP on Good Ventures.

I think any new donor, especially one that is smaller than GV (as is almost guaranteed to the the case), would end up still having their donations affect Good Ventures and my best understanding of the things Dustin is hoping to protect via this change. 

If Dustin can't communicate that he doesn't endorse every aspect of, or can't take responsibility for, everything that is currently funded through OP, I doubt that the difficult-to-track "from which source did OP spend this money" aspect of an additional donor would successfully avoid the relevant conflicts. 

My best guess is that if OP attracts an additional large donor above ~$200M, it seems best for some people to leave OP, establish a new organization that the now donor would actually be confident will be independent from Dustin's preferences, while maintaining a collaborative relationship with OP to share thoughts and insights.

I am not super confident of this, but I don't see a clear line that would allow a new donor to have confidence that OP staff isn't still going to heavily take into account Dustin's reputation and non-financial priorities in the recommendations to the new donor.

To clarify, I did see the invitations to other funders. However, my perception was that those are invitations to find people to hand things off to, rather than to be a continuing partner like with GV. Perhaps I misunderstood. 

I also want to be clear that the status quo you're articulating here does not match what I've heard from former grantees about how able OP staff are to participate in efforts to attract additional funding. Perhaps there has been quite a serious miscommunication.

This isn’t true for any of the other sub-focus areas that will be exited though, which I thought was strange. Given that nothing other than digital minds work was listed, how would any potential donors or people who know potential donors know about OP exiting things like invertebrates or wild animal welfare?

(I’m basing those sub-focus areas on the comments in this thread because of exactly this problem - I’m unclear if there’s more being exited or if those two are definitely part of it)

Due to my borderline forum addiction, you probably have a disproportionate influence on on me haha. Will probably never earn to give though so no harm done here ;).

I'm not thinking just discouraging attempts to diversify funding of one's own org, but also discouraging earning to give, discouraging projects to bring in more donors, etc.

I generally agree, but with reservations.

• I think that Cari and Dustin's funding has obviously created a lot of value. Maybe, even ~60% of all shapley EA value. 
• I personally don't feel like I know much of what Cari and Dustin actually believe about things, other than that they funded OP. They both seem to have been fairly private.
• At this point, I'm hesitant to trust any authority figure that much. "Billionaires in tech, focused on AI safety issues" currently has a disappointingly mixed track record. 
• 'ultimately it's "their money" and their choice to do with their money what they genuinely believe can achieve the most good.  ' -> Sure, but this is also true of all many[1] big donors. I think that all big donors should probably get more evaluation and criticism. I'm not sure how many specific actions to take differently when knowing "it's their choice to do with their money what they genuinely believe can achieve the most good."
• "genuinely believe can achieve the most good" -> Small point, but I'm sure some of this is political. "Achieve the most good" often means, "makes the funder look better, arguably so that they could do more good later on". Some funders pay for local arts museums as a way for getting favor, EA funders sometimes pay for some causes with particularly-good-optics for similar reasons. My guess is that the EA funders generally do this for ultimately altruistic reasons, but would admit that this set of incentives is pretty gnarly. 
  
  [1] Edited, after someone flagged. I had a specific reference class in mind, "all" is inaccurate.