All of A_donor's Comments + Replies

Yes, and greater GDP maps fairly well to greater effectiveness of altruism. I think you're focused on downside risks too strongly. They exist, and they are worth mitigating, but inaction due to fear of them will cause far more harm. Inaction due to heckler's veto is a not a free outcome.

Companies not being loss-limited would not cause them to stop producing x-risks when the literal death of all their humans is an insufficient motivation to discourage them. It would reduce a bunch of other categories of harm, but we've converged to accepting that risk to avoid crippling risk aversion in the economy.

1. Seems essentially fine. There's a reason society converged to loss-limited companies being the right thing to do, even though there is unlimited gain and limited downside, and that's that individuals tend to be far too risk averse. Exposing them to a risk to the rest of their portfolio should be more than sufficient to make this not a concern.
2. Might be a fair point, but remember, this is in the case where some project was predictably net negative and then actually was badly net negative. My guess is at least some funders would be willing to step in and disincentivise that kind of activity, and the threat of it would keep people off the worst projects.

If the distribution mismatch problem is not mitigated (and it seems hard to mitigate), investors are incentivized to fund high-stakes projects while regarding potential harmful outcomes as if they were neutral.

I thought a bunch more about this, and I do think there is something here worth paying attention to.

I am not certain that there is a notable enough pool of the projects in the category you're worried about to offset the benefits of impact markets, but we would incentivise those that exist, and that has a cost.

If we're limited to accredited investors,... (read more)

I should have phrased differently: It's not that hard to pick out highly risky projects retroactively, relative to identifying them prospectively. I also think that the reference class which is most worrying is genuinely not that hard to identify as strongly negative EV.

Impact markets don't solve the problem of funders being able to fund harmful projects. But they don't make it differentially worse (it empowers funders generally, but I don't expect you would argue that grantmakers are net negative, so this still comes out net-positive).

I would welcome atte... (read more)

I think these concerns are largely incorrect, and avoiding implementing this technology due to them would be a tragedy. Here's my reasoning:

1. It's not that hard to see when a project was at risk of having large downsides, oraculars can avoid retrofunding these just as they would avoid normally funding them.^[1]
2. If you assume oraculars who are fine with incentivising doing large amounts of expected harm in order to get credit for something if it goes well (which I think is a flawed assumption), well.. they can do that with normal funding. Retrofunding does not

Quick opinions on the choice branches:

1: What Is The Basic Format Of The Market?

Custom: Awesome Auto Auction, but closest to D: Fractionalized Impact Shares With Assurance Contract.

2: What Is Being Sold?

B: Credit For Funding The Project - Funding a project retroactively should not be considered morally different from funding it in any other way, creating a novel kind of moral accounting seems overcomplicating things and hard to get actual consensus / common knowledge on.

3: How Should The Market Handle Projects With Variable Funding Needs?

Custom: Awesome Au... (read more)

I did a bunch of mechanism design on the details of #1 alongside the Impact Markets people!

Key desiderata:

• Low overhead for participants in the market - Incentivising these people to spend lots of brainpower trying to work out when you buy and sell is massively suboptimal. You want a system that is buy and forget and just works.
• Minimize hard feelings around selling your cert at a loss or seeing equivalent ones go for a very low price - Related to your #II - C
• Impact authors automatically get some fraction of the rewards of a very successful project^[1]
• Ability

This is great! Ties the core threads of thought around this disagreement in a cohesive and well-referenced way, much more digestible than the originals.

I'd suggest cross-posting to LessWrong, and including links to the relevant posts on LW/EAF so that the "Mentioned in" feature of this forum generates backlinks to your work from those posts.

The Parable of the Talents, especially the part starting at:

But I think the situation can also be somewhat rosier than that.

Ozy once told me that the law of comparative advantage was one of the most inspirational things they had ever read. This was sufficiently strange that I demanded an explanation.

Ozy said that it proves everyone can contribute. Even if you are worse than everyone else at everything, you can still participate in global trade and other people will pay you money. It may not be very much money, but it will be some, and it will be a measure

Talk to HaonChan and Jehan on the Bountied Rationality Discord. They're trying to build this.

I was considering writing something like this up a a while back, but didn't have enough evidence directly, was mostly working of too few examples as a grantmaker and general models. Glad this concern is being broadcast.

I did come up with a proposal for addressing parts of the problem over on the "Please pitch ideas to potential EA CTOs" post. If you're a software dev who wants to help build a tool which might make the vultures less able to eat at least parts of EA please read over the proposal and ping me if interested.

Additional layer: Have the researchers have a separate "Which infrastructure / support has been most valuable to you?" category of karma, and use that to help direct funding towards the most valuable parts of the infrastructure ecosystem to support alignment. This should be one way, with researchers able to send this toward infrastructure, but not the reverse since research is the goal.

The purpose of preserving alignment is not to get back to AI as quickly as possible, but to make it more likely that when we eventually do climb the tech tree we are more likely to be able to align advanced AIs. Even if we have to reinvent a large number of technologies, having alignment research ready represents a (slightly non-standard) form of differential technological development rather than simply speeding up the recovery overall.

Agreed that civilization restart manuals would be good, would be happy to have the alignment archives stored alongside those. Would prefer not to hold up getting a MVP of this much smaller and easier archive in place waiting for that to come together though.

My guess is these are great for longevity,  but maybe prohibitively expensive^[1] if you want to print out e.g. the entire alignment forum plus other papers. 

Could be good for a smaller selected key insights collection, if that exists somewhere?

1. ^{^}

   Likely reference class is gravestones. I'm getting numbers like:  "Extra characters are approximately $10 thereafter" and "It costs around £1.95 per letter or character", even with a bulk discount that's going to add up.

AI alignment is rapidly scaling funding, and this means grantmakers will be stretched thin and less able to reliably avoid giving money to people who are not mission aligned, not producing useful research, or worst of all just want to extract money from the system. This has the potential to cause massive problems down the road, both by producing distracting low-quality research and by setting up expectations which will cause drama if someone is defunded later on while there's still a lot of money flowing to others.

An attack-resistant EigenKarma^[1]-like net... (read more)

Yes: "Regrantors will be compensated for their work based on the quality and volume of their grantmaking."

I'd be very interested  in joining as a regranter, though it may make sense to wait a few years, by which point I will have donated most of my crypto pool and gained a bunch of experience. You can see my current strategy at Being an individual alignment grantmaker.

Edit: Does screening for conflicts of interest mean not allowing regranters to grant to people they know? If yes, I can see the reasoning, but if I was operating under this rule it would have blocked several of my most promising grants, which I found through personal connections. I would pro... (read more)