Update: Replies say this is not a good idea. I haven't deleted the post however so that people could still read the replies.
Idea came to me when thinking about this post
There are x-risks we know of. What about the x-risks we face that we don't even know about? Would it be worth placing a financial bounty to incentivise their research and disclosure? Here's my proposal. Keen on feedback or thoughts. Keen on anyone who wants to seriously take this forward.
How it works
Some org or set of orgs set this up. They maintain a list of known x-risks, well-defined. They maintain a list of criteria for what makes an x-risk worth knowing about. People can submit a report disclosing any new x-risk that doesn't exist on the list, but satisfies the criteria for being worth looking into. They are paid a bounty for the same.
Now let's get into the details.
List of known x-risks
Some formalisation and rigid definition would be required here, to avoid exploitation of loopholes in the bounty. This list will be maintained. There could also be a second more private list of x-risk too dangerous to even mention publicly. (Although I do sincerely hope this list is empty today, you never know.)
Criteria for an x-risk to be "worth knowing about"
I assume the main criteria would be:
- Timeframe. in which the x-risk could be triggered. 30 years? 200 years?
- Impact. Extinction or just massive population reduction? Extinction with painless death or mass suffering? This depends on the org's subective stances on suffering-based ethics, long-termism, etc. An org for instance might not prioritise the knowledge of x-risk that cause massive population reduction (nukes) as much as they prioritise the knowledge of x-risks that cause extinction (AI safety), because they believe a reduced population can still regrow itself. Another org might find futures involving mass suffering or torture to be worse than even extinction.
- Likelihood. Obviously x-risks only matter if they cross some minimum threshold of likelihood. I'm not sure if head-on estimating likelihood is the best approach, but maybe it'll be necessary to define objective criteria for the bounty. Need to look at key cruxes for odds of triggering of the risk.
More criteria can be thought of and maintained.
Bounty size can scale with likelihood and impact of the disclosed x-risk. For instance it might even be worth paying $100M for disclosure of an x-risk that has >5% annual odds of occuring. Onus is on disclosing party to estimate likelihood and impact, although this can be further debated between all parties in an iterated fashion.
One could try determining optimal bounty sizes. This will require estimating the incremental odds of a disclosure happening by increased bounty size. It will also require estimating opportunity costs of that money being stuck as a bounty, instead of being allocated elsewhere.
I assume many people are not motivated by money in the first place, and would write the report even for fairly low renumeration. Some people ofcourse are motivated by money, if they are devoting their idle time to it.
Large bounties can however increase public consciousness because they make good headlines. This in turn could increase the number of people who are aware of x-risks and thinking about them. Like how many college students try solving P v NP or the Clay math problems for fun.
Some research might require funding before being shown to the panel, and someone might be willing to privately fund if they know they can recoup the cost from the bounty. One could even imagine open-ended search in this fashion. An org could get private funding for finding x-risks even before they have found one - as long as the private funder knows they can receive payout from the bounty if one is found. This requires a very large bounty to be effective though.
An alternate to private funding with single bounty is an iterated approach. The disclosing party can obtain funding for their research directly from the bounty-providing org. This will now not be structured as a simple bounty and will have its own considerations. I haven't discussed them here but they're definitely worth discussing.
Cost of capital
Money allocated to the bounty has to be idle, to ensure credibility of the bounty. At best it could be passively invested as stocks or crypto, while waiting for someone to claim it. Some EA orgs may already have idle assets which they haven't yet allocated. Offering such a bounty could be a good idea for such orgs. Once these orgs have less idle assets, then they will have take the tough call of how much to keep as bounty and how much to divert elsewhere. This will depend on the time-value from both opportunities, where time-value will be measured both in monetary and non-monetary terms.
There could be some x-risks that are also infohazards. In this case, the panel will have to interact privately with the disclosing party. If the bounty is paid out for such a disclosure - either the bounty payout itself should be made private knowledge (which may be hard), or people will get to know the bounty was paid out but not know for what. The diclosing party should be able to disclose anonymously, so that info about the disclosing party does not lead to guesses on what the disclosure was. The panel therefore needs to be highly trustworthy. There should also be proper information on how the diclosing party can maintain their anonymity during disclosure. Be it using digital means (Tor, linux, etc) or physical (private addresses to visit or drop letters to).
Most importantly, the panel also needs to be competent at handling the infohazard after its discovery. Some discoveries might be intractable, so the only approach to those x-risks is to hope that no one else ever discovers them, and that's it. Others may benefit from research that could reduce the risk. Others could require lobbying governments or contacting intelligence agencies to take some secretive action that reduces the risk.
I agree that the odds of such world-damning information coming to light is fairly low, but due to epistemic humility I would still think this whole scenario is worth planning out properly. Members of the panel must have all the requisite skills to handle this - or atleast to contact all the people with requisite skills.
There could be one panel that undertakes all the above functions such as maintaining the x-risk and criteria lists, interacting with disclosing parties and allocating bounties, making broader decisions on how big the bounty should be, handling infohazards, etc. Or these responsibilites could be distributed. Ideally the infrastructure is designed such that multiple ethical viewpoints can be supported, and multiple organisations can be involved. Both in appointing panel members, and adding more money to the bounty.