Thank you, Neil. It's highly relevant
Thank you, Neil. It's highly relevant
tl;dr: If your reason for cramming AI knowledge into your brain is stress, then don't do it. You can still be useful, but walk away from the front lines where people are directly responsible.
Disclaimer: 1) This is an obvious problem that has already been noticed and addressed by many LessWrong users 2) This is not an original solution but rather a specific framing of the problem and some food for thought 3) I could be gravely mistaken, and your best bet might be putting your all into research after all. However, you might just want to emerge from lurking and actually do something if that is the case. I wouldn't put all my chips on solely advancing the front lines.
The heroes who will save the world will be alignment researchers: when, dot by dot, the stars will grow dark and a dark dawn will rise, we will all have to buy them a beer. If you are not among this small band of heroes who will guarantee us the best future possible, you may feel an urgent need to promptly join them: I beg of you only to consider whether this is a good idea.
If you think that you should become an alignment researcher in a matter of months, [1] I will not try to stop you. But it's probably worth a few days worth of cranial compute to establish whether you are exploiting yourself in the best way you could.
I'll set the parameters of the problem. "Becoming an alignment researcher" is a spectrum: the more you learn about alignment, the more capable you are at navigating the front lines of the alignment project. Certainly, understanding the tenets of alignment is a laudable goal; but at what point will you be faced with diminishing returns? If you are not planning on single-handedly solving the alignment problem, are there not better uses of your time?[2]
There are many instrumental goals that serve the terminal goal of completing the alignment project, and they might be more worthy of your time:
If you feel stressed right now and have thus decided to spend your time scrantically [6] reading LessWrong posts about AGI projections and alignment solutions while breathing heavily. . . just don't. Don't become an alignment scientist today because you are stressed. Don't sacrifice doing what you love, because there's a good chance you can help us by doing what you love. Solve the "you are human" problem first and then perhaps solve one of the others, so that you are not directly involved on the front lines where responsibility is direct. You are just as responsible for the universe as the rest of us: but you are responsible for results, not effort, and that could mean walking away from the front lines.
Ah, and if you're too stressed: breathe three times using the whole capacity of your lungs, smile at a mirror, then eat some chocolate. I bid you an excellent day. Spend some time looking at flowers or something. Then you can return to heroics.
It's not enough to become an alignment researcher. You must become a useful alignment researcher, which is of course an even harder target to attain.
I'm conflicted because there might be too many people writing and reading on the blog instead of spending hours in solitude attempting to actually find a solution to alignment. I deeply respect people who do the latter, and we need more of them. More on that later.
"Weaker risks" does not mean they should not be addressed: if climate change starts hampering alignment research, like by slowing down development in poor countries, it should be proportionally paid attention to. How important minor risks are kind of depends on what your AGI forecast model is (there are a dozen on LessWrong). But the point is, almost all the existential risk is concentrated in AGI and so the importance of all other problems should be correlated with their relevance to the alignment problem.
The cool aspect of this problem is that you can't be stressed by it. The other problems are external in nature: but the whole point of this problem is that you must be at peace for it to be solved, meaning that you can't rush your way through it, half-ass it, or have a breakdown while doing it. Take a walk outside or something.
False hope is a dangerous thing and I do not mean to supply that here. If we all recycle our pizza boxes, the world won't be saved. But taking away some distractions and burdens that alignment researchers may be plagued with seems like an excellent use of time. And being a good person is just generally a good thing: i.e. don't drop everything including your morals and give your all to the alignment project. There's a lot to say about arrogance of this kind: think of Raskolnikov from Crime and Punishment. AGI is not an excuse for you to forget basic duties.
Rushing around LessWrong with its abundance of footnotes and references with the goal of learning something and clarifying the picture for you, will accomplish nothing but fragment your mind and increase your stress levels. Digest all knowledge.
I'm not so sure these days[1]. The heroes who save the world may well be those that get Microsoft/Open AI, Google Deepmind and Anthropic to halt their headlong Earth-threatening suicide-race toward AGI. Or those who help create a global moratorium or taboo on AGI that gives us the years (or decades) of breathing space needed for Alignment to be solved. Or those who help craft, enact, and enforce strict global limits on compute and data aimed at preventing AGI-level training runs, and those who adhere to them. Without these, I just don't think there is time for Alignment to be solved. They are now the bottleneck through which the future flows, not Alignment research. (More.)
Also, it might even be the case that Alignment is impossible. In this case, then I guess Alignment researchers could be instrumental in providing the theoretical proofs for this, and thus keep the world safe by providing justification for the continuation of an indefinite moratorium on AGI.
Although to be clear, we owe historical alignment researchers a huge debt of gratitude for raising awareness of AI x-risk