I don't have any specific pathway I think is particularly likely. Some pathways could be stuff like "simulator AI latches onto evil AI storytrope" or "AI controllers start threatening each other" or "psychotic AI controller on drugs decides to do Y where Y is a concept sampled from things the AI controller knows about". The specific pathway is hard to predict and there is a general principle underlying all of them which is more relevant to pay attention to.
The abstract principle at play is that a system which has low complexity for X also has low complexity for notX.
If there aren't other reasons that the system has a low complexity for notX, then the dominant effect on the complexity of notX is directly downstream (through inversion) of the systems complexity for X.
Ending up with notX when you wanted X is more likely the more foolish you go about things, and the collection of the general public is not a wise careful entity.
The linked argument seems to talk about default outcomes, and I think it makes sense for x-risk. For s-risks I guess it depends on how one expects that the default outcomes look like, it could make sense depending on the outlook.
My view sees the severe s-risks (strong value pessimization) as tail risks, which one could hyperstition into becoming more probable. I'm sympathetic to seeing some milder/less-severe s-risks being non trivial probability (although still not default).
Is there a TOC for why raising public awareness of S-risks decreases S-risks? I would unconfidently expect that raising public awareness of s-risks increases the probability of s-risks (through waluigi effect style pathways).
Taking this on as a cause area and talking about it can easily (and I would predict) backfire (IE increase probability of said event occurring).
If Alice is not thinking about whether to do X (say creating hell), then Bob actively trying to prevent Alice from doing X can increase the total risk of X. Bob will both bring X to consideration for Alice when it wasn't before, but also increase Alice's likelihood of endorsing X, due to ingroup-outgroup dynamics where beliefs are often flipped/inverted.
In order to justify active opposition to X, you would need X to be not only possible, but a likely or default outcome. Something which has a very strong causal factor of occurrence, such that activism does not become the main causal factor. Such considerations should be taken more seriously before adopting tail-risks as an EA-cause-area.
Most of the "recent developments" which are supposed to point to an update in the target audience were quite predictable in advance (pretty much all of these things were predicted / assigned substantial probability by many people years in advance). To the extent that these cause an update in the reader, it is worth asking yourself; how could I have thought that faster. And more importantly, once you figure out what models would have let you predict the present, what further do these models predict into the future?
hypothesis that springs to mind, might or might not be useful for engaging with it productively. might be wrong depending on what class of people you have been having meetings with.
when you select for people working on AI risks, you select for people who are generally less respective of status quo and social norms. you select for the kind of person who will generally do something less, because of the reason that its a social norm. in order for this reference class of person to do a thing, they kind of who through their personal reasoning methods reaches the conclusion "It would be worth the effort for me to change myself to become the kind of person who shows up on time consistently, compared to other things I could be spending my effort on". they might just figure its a better use of their effort to think about their research all day.
I would guess that most people on this earth don't show up on time because they reasoned it through that this is a good idea, they do it because it has been drilled into them through social norms, and they valued those social norms higher.
(note: this comment is not intended to be an argument that showing up on time is a waste of time)
In my observed experience there are lots of young people (many whom I know personally) who want to help with AI-alignment and are in my opinion capable of doing so, they just need to spend a year or two trying and learning things to get the necessary skills.
These are people who usually lack prior achievement and therefore are not able to access various EA-adjacent grants to buy themselves the slack needed to put in the time and focus on a singular goal. When really what they need is not salary-sized grants but just a guarantee that they will have food, housing and a supportive environment and community, while they take a pause from formal education or jobs meant to keep them afloat.
I personally know one such person who got helped out of prior dependence through CEEALAR and has started to become productive. My own (and Orthogonals) stay at CEEALAR strongly positive as well.
Ive heard that grant makers are often operationally constrained on giving out smaller hit based grants to for example individuals. By giving to CEEALAR, grant makers would outsource this operational cost and would be able to bootstrap people doing AI-alignment, in a hits based manner for low cost compared to alternatives such as individual grants. Which I do think is one of the most cost effective ways to help with AI-alignment. Myself and many others I know doing good work in Alignment would not exist in the space if not for personal hit-based grants.
I'm very confused and sad that CEEALAR has not recieved more funding. Not only do I wish that CEEALAR could stay afloat and expand, but I also think we would STRONGLY benefit from similar institutions aimed at low cost housing for motivated, but potentially unproven individuals, at other geographical locations, say east and west coast in the US, as well as somewhere in Europe. If CEEALAR was able to consistently get funded, that would grant confidence for people to start similar organizations elsewhere.
I don't have any specific pathway I think is particularly likely. Some pathways could be stuff like "simulator AI latches onto evil AI storytrope" or "AI controllers start threatening each other" or "psychotic AI controller on drugs decides to do Y where Y is a concept sampled from things the AI controller knows about". The specific pathway is hard to predict and there is a general principle underlying all of them which is more relevant to pay attention to.
The abstract principle at play is that a system which has low complexity for X also has low complexity for notX.
If there aren't other reasons that the system has a low complexity for notX, then the dominant effect on the complexity of notX is directly downstream (through inversion) of the systems complexity for X.
Ending up with notX when you wanted X is more likely the more foolish you go about things, and the collection of the general public is not a wise careful entity.
The linked argument seems to talk about default outcomes, and I think it makes sense for x-risk. For s-risks I guess it depends on how one expects that the default outcomes look like, it could make sense depending on the outlook.
My view sees the severe s-risks (strong value pessimization) as tail risks, which one could hyperstition into becoming more probable. I'm sympathetic to seeing some milder/less-severe s-risks being non trivial probability (although still not default).
Is there a TOC for why raising public awareness of S-risks decreases S-risks? I would unconfidently expect that raising public awareness of s-risks increases the probability of s-risks (through waluigi effect style pathways).
To clarify, do you mean that high consumption of Veblen goods would imply worse morality?
Taking this on as a cause area and talking about it can easily (and I would predict) backfire (IE increase probability of said event occurring).
If Alice is not thinking about whether to do X (say creating hell), then Bob actively trying to prevent Alice from doing X can increase the total risk of X. Bob will both bring X to consideration for Alice when it wasn't before, but also increase Alice's likelihood of endorsing X, due to ingroup-outgroup dynamics where beliefs are often flipped/inverted.
In order to justify active opposition to X, you would need X to be not only possible, but a likely or default outcome. Something which has a very strong causal factor of occurrence, such that activism does not become the main causal factor. Such considerations should be taken more seriously before adopting tail-risks as an EA-cause-area.
Most of the "recent developments" which are supposed to point to an update in the target audience were quite predictable in advance (pretty much all of these things were predicted / assigned substantial probability by many people years in advance). To the extent that these cause an update in the reader, it is worth asking yourself; how could I have thought that faster. And more importantly, once you figure out what models would have let you predict the present, what further do these models predict into the future?
hypothesis that springs to mind, might or might not be useful for engaging with it productively. might be wrong depending on what class of people you have been having meetings with.
when you select for people working on AI risks, you select for people who are generally less respective of status quo and social norms. you select for the kind of person who will generally do something less, because of the reason that its a social norm. in order for this reference class of person to do a thing, they kind of who through their personal reasoning methods reaches the conclusion "It would be worth the effort for me to change myself to become the kind of person who shows up on time consistently, compared to other things I could be spending my effort on". they might just figure its a better use of their effort to think about their research all day.
I would guess that most people on this earth don't show up on time because they reasoned it through that this is a good idea, they do it because it has been drilled into them through social norms, and they valued those social norms higher.
(note: this comment is not intended to be an argument that showing up on time is a waste of time)
In my observed experience there are lots of young people (many whom I know personally) who want to help with AI-alignment and are in my opinion capable of doing so, they just need to spend a year or two trying and learning things to get the necessary skills.
These are people who usually lack prior achievement and therefore are not able to access various EA-adjacent grants to buy themselves the slack needed to put in the time and focus on a singular goal. When really what they need is not salary-sized grants but just a guarantee that they will have food, housing and a supportive environment and community, while they take a pause from formal education or jobs meant to keep them afloat.
I personally know one such person who got helped out of prior dependence through CEEALAR and has started to become productive. My own (and Orthogonals) stay at CEEALAR strongly positive as well.
Ive heard that grant makers are often operationally constrained on giving out smaller hit based grants to for example individuals. By giving to CEEALAR, grant makers would outsource this operational cost and would be able to bootstrap people doing AI-alignment, in a hits based manner for low cost compared to alternatives such as individual grants. Which I do think is one of the most cost effective ways to help with AI-alignment. Myself and many others I know doing good work in Alignment would not exist in the space if not for personal hit-based grants.
I'm very confused and sad that CEEALAR has not recieved more funding. Not only do I wish that CEEALAR could stay afloat and expand, but I also think we would STRONGLY benefit from similar institutions aimed at low cost housing for motivated, but potentially unproven individuals, at other geographical locations, say east and west coast in the US, as well as somewhere in Europe. If CEEALAR was able to consistently get funded, that would grant confidence for people to start similar organizations elsewhere.