The definition of existential risk as ‘humanity losing its long term potential’ in Toby Ord precipice could be specified further. Without (perhaps) loss of generality, assuming finite total value in our universe, one could specify existential risks into two broad categories of risks such as:
Extinction risks (X-risks): Human share of total value goes to zero. Examples could be extinction from pandemics, extreme climate change or some natural event.
Agential risks (A-risks): Human share of total value could be greater than in the X-risks scenarios but keeps being strictly dominated by the share of total value holded by misaligned agents. Examples could be misaligned institutions, AIs or loud alienscontrolling most of the value in the universe and with whom there would be little gain from trade to be hoped for.
Note: This post was crossposted from Planned Obsolescence by the Forum team, with the author's permission. The author may not see or respond to comments on this post.
Subtitle: If acceleration compounds, the naive extrapolation implies superintelligence within a year
This post discusses a public METR report on AI R&D acceler...
TL;DR:
* When we started Rethink Priorities (RP) in 2018, our mission was, simply put, to close the knowledge gaps that stopped people from improving the world as much as they could. That hasn’t changed much, but the landscape we’re doing it in is shifting.
* In 2027, we plan to grow our core GHD, Animal Welfare, and Worldview Investigations teams to take on emerging priorities across causes, including those that the forthcoming transformative AI is creating.
...
This was cross-posted from the Animal Charity Evaluators blog.
We update our list of recommendations annually. This year, we announced our recommendations on November 10.
Most people who care about animals arrive at the same impas...
The definition of existential risk as ‘humanity losing its long term potential’ in Toby Ord precipice could be specified further. Without (perhaps) loss of generality, assuming finite total value in our universe, one could specify existential risks into two broad categories of risks such as: