Great read, thanks!

I admit it took me a second to understand what you mean by irrevocable risks (it's more the consequences of failure that are irrevocable).

Nice piece!

I think an area that not many people are working on is how to raise more funding (from the public, business, government etc.), especially in specific cause areas.

Your point on replaceability is valid, but it may not be so simple in say AI research as it would be in working for a fossil fuels company or a financial institute. In a less researched field such as AI, your chance of making a significant advance that wouldn't have otherwise been made is significantly higher than in the above examples.

I agree with your point after this though: "because then you get to sound alarm bells if risk becomes elevated, while networking with people in an important space. The risk associated with speeding up a technology's development might be outweighed by the networking benefits from participating in that field."