Yes, and greater GDP maps fairly well to greater effectiveness of altruism. I think you're focused on downside risks too strongly. They exist, and they are worth mitigating, but inaction due to fear of them will cause far more harm. Inaction due to heckler's veto is a not a free outcome.
Companies not being loss-limited would not cause them to stop producing x-risks when the literal death of all their humans is an insufficient motivation to discourage them. It would reduce a bunch of other categories of harm, but we've converged to accepting that risk to avoid crippling risk aversion in the economy.
Seems essentially fine. There's a reason society converged to loss-limited companies being the right thing to do, even though there is unlimited gain and limited downside, and that's that individuals tend to be far too risk averse. Exposing them to a risk to the rest of their portfolio should be more than sufficient to make this not a concern.
Might be a fair point, but remember, this is in the case where some project was predictably net negative and then actually was badly net negative. My guess is at least some funders would be willing to step in and disincentivise that kind of activity, and the threat of it would keep people off the worst projects.
If the distribution mismatch problem is not mitigated (and it seems hard to mitigate), investors are incentivized to fund high-stakes projects while regarding potential harmful outcomes as if they were neutral.
I thought a bunch more about this, and I do think there is something here worth paying attention to.
I am not certain that there is a notable enough pool of the projects in the category you're worried about to offset the benefits of impact markets, but we would incentivise those that exist, and that has a cost.
If we're limited to accredited investors, as Scott proposed, we have some pretty strong mitigation options. In particular, we can let oraculars pay to mark projects as having been strongly net negative, and have this detract from the ability of those who funded that project to earn on their entire portfolio. Since accounts will be hard to generate and only available to accredited investors, generating a new account for each item is not an available option.
I think I can make some modifications to the Awesome Auto Auction to include this fairly simply, and AAA does not allow selling as an action which removes the other risk of people dumping their money / provides a natural structure for limiting withdrawals (just cut off their automatic payments until the "debt" is repayed).
Would this be sufficient mitigation? And if not, what might you still fear about this?
If you run a fully controlled (Web2) impact market for 6-12 months, and the market funds great projects/posts and there's no sign of trouble, will you then launch a decentralized impact market that no one can control (in which people can sell the impact of recruiting additional retro funders, and the impact of establishing that very market)?
I don't see much benefit to a Web3 one assuming we can do microtransactions on a Web2, so I'd be fine with either not doing the Web3 or only doing it after several years of having a Web2 without any of those restrictions and nothing going badly wrong (retaining the option to restrict new markets for it at any time).
I should have phrased differently: It's not that hard to pick out highly risky projects retroactively, relative to identifying them prospectively. I also think that the reference class which is most worrying is genuinely not that hard to identify as strongly negative EV.
Impact markets don't solve the problem of funders being able to fund harmful projects. But they don't make it differentially worse (it empowers funders generally, but I don't expect you would argue that grantmakers are net negative, so this still comes out net-positive).
I would welcome attempts to cause the culture of big grantmakers to more reliably make sure the recipients stay focused on the major challenges, but that is a separate project.
The classes of problem you list are all important questions of what should be funded, and it would be great to have better models of the EV of funding them, but none of them are impact-market specific. It's already true that the funder who is most enthusiastic about a project can fund it unilaterally, and that this will sometimes be EV-negative. People can already recruit new funders who are risk-tolerant.
We're making grantmakers generally more powerful, and that's not entirely free of negative effects[1], but it does seem very likely net-positive.[2]
I do think it makes sense to not rush into creating a decentralized unregulatable system on general principles of caution, as we certainly should watch the operation of a more controllable one for some time before moving towards that.
The community as a whole cannot come to consensus on each of the huge number of important decisions to be made, the bandwidth of common knowledge is far, far to low, and we are faced with too many choices for that to be a viable option. Having several of the most relevant people strongly on board is about as good a sign as you could expect currently. I'm open to more opinions coming in, and would be very interested in seeing you debate with people on the other side and try to double crux on this or get more people on board with your position, but turning this into a committee is going to stall the project.
There are some other effects around cultural effects of making money flows more legible which seem possibly concerning, but I'm not super worried about negative EV projects being run.
I think these concerns are largely incorrect, and avoiding implementing this technology due to them would be a tragedy. Here's my reasoning:
It's not that hard to see when a project was at risk of having large downsides, oraculars can avoid retrofunding these just as they would avoid normally funding them.[1]
If you assume oraculars who are fine with incentivising doing large amounts of expected harm in order to get credit for something if it goes well (which I think is a flawed assumption), well.. they can do that with normal funding. Retrofunding does not add to it unless the early funders irrationally expect oraculars to buy up bad EV bets which paid off.
As for "weak consensus", we have Scott Alexander, Paul Christiano, and Eliezer Yudkowsky coming down on the side of "Yes, retrofunding is great". I'm not sure how that could be seen as anything other than strong consensus of key thought leaders, not to mention the many other people who've thought carefully about this and decided it is one of the most important interventions to improve the future.
In reply to "hindsight bias"; retrofunders have strictly more information than normal funders, and can be explicitly made aware of this failure mode. I expect major oraculars to be capable of correcting for hindsight bias well enough to pass on any really risky projects. This is a mid-sized crux, necessary but not sufficient to reverse the sign of Impact Markets' value due to this failure mode.
B: Credit For Funding The Project - Funding a project retroactively should not be considered morally different from funding it in any other way, creating a novel kind of moral accounting seems overcomplicating things and hard to get actual consensus / common knowledge on.
A: Yes as this provides good incentives, but again with Awesome Auto Auction presenting a somewhat cleaner way than the default, as the founders would not liquidate their shares and drop the price / eat a chunk of donor money, but instead receive trickle income if their project kept climbing in value.
Custom: People who are basically just okay with donating the money, like me, but don't have infinite runway so would be happy to be topped up by major funders who like the bets we pick. I really want impact markets to exist for my own use case.
I really hate all these options, and I’m posting this partly to see if other people have better ideas.
You bet I have an idea! And I bet you can guess what it is! Awesome Auto Auction solves this neatly, the founder gets funding which scales with the value of the cert, but not in a form they can sell so there is no risk of them losing the credit for their project.
Happy to go into far more detail on any of these if requested.
I did a bunch of mechanism design on the details of #1 alongside the Impact Markets people!
Key desiderata:
Low overhead for participants in the market - Incentivising these people to spend lots of brainpower trying to work out when you buy and sell is massively suboptimal. You want a system that is buy and forget and just works.
Minimize hard feelings around selling your cert at a loss or seeing equivalent ones go for a very low price - Related to your #II - C
Impact authors automatically get some fraction of the rewards of a very successful project[1]
Ability to buy into certificates for flexible amounts, with minimal hassle and handling scaling well
We converged on the below model if there is a payments platform like FTX to automate microtransactions (crypto or not), and a different one which has slightly less nice properties but requires less payouts per pay-in. I can write that one up too if wanted.
Awesome Auto-Auction (better name welcome)
The core innovation is the queue of impact slices, with a "head" of buyouts moving along the queue as new money comes in.
Ownership of certificates in an auto-auction is divided into a series of slices, from earliest (the initial entry creating the auction), through all previous holders of slices of ownership, to the current owners’ slices.[2]
There is no action required to sell or decisions about price, the only available move is to buy and the only decision is how much you want to buy. When someone buys into a cert, the incoming money is split into three streams:
Creator - Directly to the impact creator
Capital repayment - Returning the capital to current holders of the cert and buying out their slice, with no opportunity for profit
Royalty - Giving profit to investors who have already been bought out, in proportion to the amount they raised price by (incentivising people to be early investors and raise the price)
Let’s look at an example of how this plays out with a $1000 purchase into a cert, with a 25% / 30% / 45% split.[3]
Some set % goes to the original creator (100% if they have not yet got their asking price, optionally held by an assurance contract until met), some goes to paying back the money current owners put in to buy their slices (with the oldest current owner getting priority), and some as a trickle income profit to already bought-out investors who funded the cert earlier in proportion to how much they raised the price (as a %) divided by the total amount that bought out investors raised the price (the %s added together).
I made a spreadsheet which you can copy to play around with split proportions and buy events and see how the money ends up distributed among the three groups at different investment levels.
This might sound complicated, but can be made very straightforward to operate and understand with a UI which lets buyers predict how much would be paid out by their purchase at any future level of total investment in the cert. This could work something like:
This mechanism seems to thoroughly succeed at all the desiderata, and would I think make impact markets much lower friction than other proposed market mechanisms (and as a bonus transitions smoothly into #5E!).
Edit: Also might well help with some of the concerns raised in the reddit thread, as I think the action "sell" has weird emotional connotations in this domain which this mechanism might bypass.
I would be happy to have calls with anyone who might want to turn it into a reality.
I come down on the "doing good should be a route to increasing wealth" side of #4, with the caveat that I'd like to see this happen not as a big sell off type event where a final funder directly buys impact from someone who has already got plenty back, but as an ongoing royalty stream which any funder can top up if they liked the project and is split fairly among investors and creators.
If using crypto, each individual’s NFT represents the combination of all their blocks of ownership, and indicates both how large a share of the NFT they currently own and their share of past ownership visually in an unobtrusive manner (likely a thin bar, with two colors).
The split can either be determined by the creator as they add the cert to the market, or we could just pick reasonable defaults and everyone sticks to that.
This is great! Ties the core threads of thought around this disagreement in a cohesive and well-referenced way, much more digestible than the originals.
I'd suggest cross-posting to LessWrong, and including links to the relevant posts on LW/EAF so that the "Mentioned in" feature of this forum generates backlinks to your work from those posts.
But I think the situation can also be somewhat rosier than that.
Ozy once told me that the law of comparative advantage was one of the most inspirational things they had ever read. This was sufficiently strange that I demanded an explanation.
Ozy said that it proves everyone can contribute. Even if you are worse than everyone else at everything, you can still participate in global trade and other people will pay you money. It may not be very much money, but it will be some, and it will be a measure of how your actions are making other people better off and they are grateful for your existence.
Might prove reassuring. Yes, EA has lots of very smart people, but those people exist in an ecosystem which almost everyone can contribute to. People do and should give kudos to those who do the object level work required to keep the attention of the geniuses on the parts of the problems which need them.
As some examples of helpful things available to you:
Being an extra pair of hands at events
Asking someone who you think is aligned with your values and might have too much on their plate what you can help them with (if you actually have the bandwidth to follow through)
Making yourself available to on-board newcomers to the ideas in 1-on-1 conversations
Yes, and greater GDP maps fairly well to greater effectiveness of altruism. I think you're focused on downside risks too strongly. They exist, and they are worth mitigating, but inaction due to fear of them will cause far more harm. Inaction due to heckler's veto is a not a free outcome.
Companies not being loss-limited would not cause them to stop producing x-risks when the literal death of all their humans is an insufficient motivation to discourage them. It would reduce a bunch of other categories of harm, but we've converged to accepting that risk to avoid crippling risk aversion in the economy.
I thought a bunch more about this, and I do think there is something here worth paying attention to.
I am not certain that there is a notable enough pool of the projects in the category you're worried about to offset the benefits of impact markets, but we would incentivise those that exist, and that has a cost.
If we're limited to accredited investors, as Scott proposed, we have some pretty strong mitigation options. In particular, we can let oraculars pay to mark projects as having been strongly net negative, and have this detract from the ability of those who funded that project to earn on their entire portfolio. Since accounts will be hard to generate and only available to accredited investors, generating a new account for each item is not an available option.
I think I can make some modifications to the Awesome Auto Auction to include this fairly simply, and AAA does not allow selling as an action which removes the other risk of people dumping their money / provides a natural structure for limiting withdrawals (just cut off their automatic payments until the "debt" is repayed).
Would this be sufficient mitigation? And if not, what might you still fear about this?
I don't see much benefit to a Web3 one assuming we can do microtransactions on a Web2, so I'd be fine with either not doing the Web3 or only doing it after several years of having a Web2 without any of those restrictions and nothing going badly wrong (retaining the option to restrict new markets for it at any time).
I should have phrased differently: It's not that hard to pick out highly risky projects retroactively, relative to identifying them prospectively. I also think that the reference class which is most worrying is genuinely not that hard to identify as strongly negative EV.
Impact markets don't solve the problem of funders being able to fund harmful projects. But they don't make it differentially worse (it empowers funders generally, but I don't expect you would argue that grantmakers are net negative, so this still comes out net-positive).
I would welcome attempts to cause the culture of big grantmakers to more reliably make sure the recipients stay focused on the major challenges, but that is a separate project.
The classes of problem you list are all important questions of what should be funded, and it would be great to have better models of the EV of funding them, but none of them are impact-market specific. It's already true that the funder who is most enthusiastic about a project can fund it unilaterally, and that this will sometimes be EV-negative. People can already recruit new funders who are risk-tolerant.
We're making grantmakers generally more powerful, and that's not entirely free of negative effects[1], but it does seem very likely net-positive.[2]
I do think it makes sense to not rush into creating a decentralized unregulatable system on general principles of caution, as we certainly should watch the operation of a more controllable one for some time before moving towards that.
The community as a whole cannot come to consensus on each of the huge number of important decisions to be made, the bandwidth of common knowledge is far, far to low, and we are faced with too many choices for that to be a viable option. Having several of the most relevant people strongly on board is about as good a sign as you could expect currently. I'm open to more opinions coming in, and would be very interested in seeing you debate with people on the other side and try to double crux on this or get more people on board with your position, but turning this into a committee is going to stall the project.
And your cataloging of them does seem useful, there are things we can adjust to minimize them.
There are some other effects around cultural effects of making money flows more legible which seem possibly concerning, but I'm not super worried about negative EV projects being run.
I think these concerns are largely incorrect, and avoiding implementing this technology due to them would be a tragedy. Here's my reasoning:
As for "weak consensus", we have Scott Alexander, Paul Christiano, and Eliezer Yudkowsky coming down on the side of "Yes, retrofunding is great". I'm not sure how that could be seen as anything other than strong consensus of key thought leaders, not to mention the many other people who've thought carefully about this and decided it is one of the most important interventions to improve the future.
In reply to "hindsight bias"; retrofunders have strictly more information than normal funders, and can be explicitly made aware of this failure mode. I expect major oraculars to be capable of correcting for hindsight bias well enough to pass on any really risky projects. This is a mid-sized crux, necessary but not sufficient to reverse the sign of Impact Markets' value due to this failure mode.
Quick opinions on the choice branches:
1: What Is The Basic Format Of The Market?
Custom: Awesome Auto Auction, but closest to D: Fractionalized Impact Shares With Assurance Contract.
2: What Is Being Sold?
B: Credit For Funding The Project - Funding a project retroactively should not be considered morally different from funding it in any other way, creating a novel kind of moral accounting seems overcomplicating things and hard to get actual consensus / common knowledge on.
3: How Should The Market Handle Projects With Variable Funding Needs?
Custom: Awesome Auto Auction solves this elegantly, by letting the project absorb some fraction of the value of the cert if it continues to grow.
4: Should Founders Be Getting Rich Off Impact Certificates?
A: Yes as this provides good incentives, but again with Awesome Auto Auction presenting a somewhat cleaner way than the default, as the founders would not liquidate their shares and drop the price / eat a chunk of donor money, but instead receive trickle income if their project kept climbing in value.
5: How Do We Kickstart The Existence Of An Impact Market?
B: Committed Pot Of Money (aka The Original ACX Grants Plan) plus common knowledge that other funders can easily join later and will do so if they like what's happening
6: Should The Market Use Cryptocurrency?
The best of both world proposal you mention seems clearly the best.
7: How Should The Market Price IPOs?
Custom: Awesome Auto Auction. Combines most of the advantages of both, I think?
8: How Should The Oracular Funder Buy Successful Projects?
A. By Buying A Certain Number Of Shares, with Awesome Auto Auction I think removing the main concern of Monopsony? But either way, B does not fit the structure.
9: What Should The Final Oracular Funder’s Decision Process Be?
A: Fund Based On Regular Charitable Decision-Making Procedure seems like a reasonable default, but letting the oracular decide seems correct.
10: Who Are We Expecting To Have As Investors?
Custom: People who are basically just okay with donating the money, like me, but don't have infinite runway so would be happy to be topped up by major funders who like the bets we pick. I really want impact markets to exist for my own use case.
11: Conclusion: What Kind Of Impact Market Should We Have?
You bet I have an idea! And I bet you can guess what it is! Awesome Auto Auction solves this neatly, the founder gets funding which scales with the value of the cert, but not in a form they can sell so there is no risk of them losing the credit for their project.
Happy to go into far more detail on any of these if requested.
I did a bunch of mechanism design on the details of #1 alongside the Impact Markets people!
Key desiderata:
We converged on the below model if there is a payments platform like FTX to automate microtransactions (crypto or not), and a different one which has slightly less nice properties but requires less payouts per pay-in. I can write that one up too if wanted.
Awesome Auto-Auction (better name welcome)
The core innovation is the queue of impact slices, with a "head" of buyouts moving along the queue as new money comes in.
Ownership of certificates in an auto-auction is divided into a series of slices, from earliest (the initial entry creating the auction), through all previous holders of slices of ownership, to the current owners’ slices.[2]
There is no action required to sell or decisions about price, the only available move is to buy and the only decision is how much you want to buy. When someone buys into a cert, the incoming money is split into three streams:
Let’s look at an example of how this plays out with a $1000 purchase into a cert, with a 25% / 30% / 45% split.[3]
Some set % goes to the original creator (100% if they have not yet got their asking price, optionally held by an assurance contract until met), some goes to paying back the money current owners put in to buy their slices (with the oldest current owner getting priority), and some as a trickle income profit to already bought-out investors who funded the cert earlier in proportion to how much they raised the price (as a %) divided by the total amount that bought out investors raised the price (the %s added together).
I made a spreadsheet which you can copy to play around with split proportions and buy events and see how the money ends up distributed among the three groups at different investment levels.
This might sound complicated, but can be made very straightforward to operate and understand with a UI which lets buyers predict how much would be paid out by their purchase at any future level of total investment in the cert. This could work something like:
This mechanism seems to thoroughly succeed at all the desiderata, and would I think make impact markets much lower friction than other proposed market mechanisms (and as a bonus transitions smoothly into #5E!).
Edit: Also might well help with some of the concerns raised in the reddit thread, as I think the action "sell" has weird emotional connotations in this domain which this mechanism might bypass.
I would be happy to have calls with anyone who might want to turn it into a reality.
I come down on the "doing good should be a route to increasing wealth" side of #4, with the caveat that I'd like to see this happen not as a big sell off type event where a final funder directly buys impact from someone who has already got plenty back, but as an ongoing royalty stream which any funder can top up if they liked the project and is split fairly among investors and creators.
If using crypto, each individual’s NFT represents the combination of all their blocks of ownership, and indicates both how large a share of the NFT they currently own and their share of past ownership visually in an unobtrusive manner (likely a thin bar, with two colors).
The split can either be determined by the creator as they add the cert to the market, or we could just pick reasonable defaults and everyone sticks to that.
This is great! Ties the core threads of thought around this disagreement in a cohesive and well-referenced way, much more digestible than the originals.
I'd suggest cross-posting to LessWrong, and including links to the relevant posts on LW/EAF so that the "Mentioned in" feature of this forum generates backlinks to your work from those posts.
The Parable of the Talents, especially the part starting at:
Might prove reassuring. Yes, EA has lots of very smart people, but those people exist in an ecosystem which almost everyone can contribute to. People do and should give kudos to those who do the object level work required to keep the attention of the geniuses on the parts of the problems which need them.
As some examples of helpful things available to you:
Talk to HaonChan and Jehan on the Bountied Rationality Discord. They're trying to build this.