I'm a Computer Science student from Ireland who's interested in AI safety research.
Good point. It's important to note that black swans are subjective and depend on the person. For example, a Christmas turkey's slaughter is a black swan for it but not for its butcher.
I disagree because I think these kinds of post hoc explanations are invalidated by the hindsight fallacy. I think the FTX crash was a typical black swan because it seems much more foreseeable in retrospect than it was before the event.
To use another example, the 2008 financial crisis made sense in retrospect, but the Big Short movie shows that, before the event, even the characters shorting the mortgage bonds had strong doubts about whether they were right and most other people were completely oblivious.
Although the FTX crisis makes sense in retrospect, I have to admit that I had absolutely no idea that it was about to happen before the event.
Thanks! I used that format because it was easy for me to write. I'm glad to see that it improves the reading experience too.
I really like this post and I think it's now my favorite post so far on the recent collapse of FTX.
Many recent posts on this subject have focused on topics such as Sam Bankman Fried's character, what happened at FTX and how it reflects on EA as a whole.
While these are interesting subjects, I got the impression that a lot of the posts were too backward-looking and not constructive enough.
I was looking for a post that was more reflective and less sensational and focused on what we can learn from the experience and how we can adjust the strategy of EA going forward and I think this post meets these criteria better than most of the previous posts.
This reminds of Nick Bostrom's story, "The Fable of the Dragon-Tyrant". Maybe somebody will write a story like this about ageing instead of smallpox in the future.
I think microgrants are a great idea! Because they're small, you can make lots of investments to different people with relatively little risk and cost.
One way of doing automated AI safety research is for AI safety researchers to create AI safety ideas on aisafetyideas.com and then use the titles as prompts for a language model. Here is GPT-3 generating a response to one of the ideas:
This question would have been way easier if just I estimated the number of AI safety researchers in my city (1?) instead of the whole world.
Here is a model that involves taking thousands of trials of the product of six variables randomly set between 10% and 90% (e.g. 0.5^6 = 0.015 = 1.5%).
As other people have noted, conjunctive models tend to produce low probabilities (<5%).
Great post. This is possibly the best explanation of the relationship between capabilities and safety I've seen so far.