IIRC Tristan Harris has also made this claim. Maybe his 80k podcast or The Social Dilemma has some clues. Edit: maybe he just said something like 'Youtube's algorithm is trained to send users down rabbit hole'
Re why AI isn't generating much revenue - have you considered the productivity paradox? It's historically normal that productivity slows down before steeply increasing when a new general purpose technologies arrives.
See "Why Future Technological Progress Is Consistent with Low Current Productivity Growth" in "Artificial Intelligence and the Modern Productivity Paradox"
Instructions for that: http://www.eccentrictraining.com/6.html
That's really interesting, thanks! Do you (or someone else) have a sense of how much variation in priorities can be explained by the big 5?
Makes sense. I guess then the question is if the work of everyone except the x-risk focused NGOs helps reduce r x-risk much. I tend to think yes since much of pandemic preparedness also addresses the worst case scenarios. But that seems to be an open question.
Thanks, great analysis! Just registering that I still expect bio risk will be less neglected than in the past. The major consideration for me is institutional funding, due to its scale. Like you say:
We believe that an issue of the magnitude of COVID-19 will likely not be forgotten soon, and that funding for pandemic preparedness will likely be safe for much longer than in the aftermath of previous pandemics. In particular it may persist long enough to become institutionalised and therefore harder to cut.
Aside from future institutional funding, we also have to take the into account the current funding and new experience because they contribute to our cumulative knowledge and preparedness.
Important question, and nicely researched!
A caveat is that some essential subareas of safety may be neglected. This is not a problem when subareas substitute each other: e.g. debate substitutes for amplification so it's okay if one of them is neglected. But there's a problem when subareas complement each other: e.g. alignment complements robustness so we probably need to solve both. See also When causes multiply.
It's ok when a subarea is neglected as long as there's a substitute for it. But so far it seems that some areas are necessary components of AI safety (perhaps both inner and outer alignment are).
This was also discussed on LessWrong:
Kudos btw for writing this. Consciousness is a topic where it can be really hard to make progress and I worry that people aren't posting enough about it for fear of saying something wrong.
I agree that physical theories of consciousness are pan psychist if they say that every recurrent net is conscious (or that everything that can be described as GWT is conscious). The main caveats for me are:
Does anyone really claim that every recurrent net is conscious? It seems so implausible. E.g. if I initialize my net with random parameters, it just computes garbage. Or if I have a net with 1 parameter it seems too simple. Or if the number of iterations is 1 (as you say), it's just a trivial case of recurrence. Or if it doesn't do any interesting task, such as prediction...
(Also, most recurrent nets in nature would be gerrymandered. I could imagine there are enough that aren't though, such as potentially your examples).
NB, recurrence doesn't necessarily imply recurrent processing (the term from recurrent processing theory). The 'processing' part could hide a bunch of complexity?