Edit: The match seems to now be over - the funds were expended in around 35 minutes. But many of these charities are great and should be supported regardless!
***
On October 22nd starting at 10:00am Pacific Time, Every.org is matching donations to select animal welfare organizations at a 2:1 rate, up to $2,000 per donor per nonprofit (so if you give up to $2,000, they will contribute up to $4,000 and the nonprofit will get up to $6,000). The first donation a donor makes to each el...
Crossposted to Less Wrong
‘Like most of life’s dangers, the dangers of artificial intelligence are not easily identified from the armchair.’
J Dmitri Gallow
Epistemic status
I’m not an expert in this field, and herein I make some sweeping generalisations educated guesses about the overall...
A lot of AI safety people I know are excited about Anthropic and I don't fully understand why. My instinct is to distrust company because they're the one pushing the AI arms race, autonomously hacking three companies, and IPO'ing, but am likely missing something because of how respected they are within this space. Are there examples where they have counterfactually produced some result, policy, or finding that has slowed down capabilities progress more than they have themselves pushed capabilities?
Anthropic has done many expensive actions to credibly signal that it cares about AI safety.
I think folks could reasonably say that they are engaged in race dynamics that makes them net negative. But it also doesn't seem crazy to look at the actions above, and admire what they've done (especially compared to other actors in the space).
Anthropic has produced a lot of alignment research. On certain theories* of where AI danger comes from, that research has been useful enough that Anthropic's overall impact is net positive.
*Those theories are wrong. In a sentence: Anthropic's alignment research is almost entirely centered around how to produce desired behaviors in the short term, with no understanding of how to make an ASI continue to be aligned once you are no longer smart enough to detect misalignment.
This becomes much easier to explain when you allow for the possibility that people are biased or irrational or have conflicts of interest. People want to work on cool problems, they want to be close to the action, they want to get rich, and they want to think of their friends as good people; then they reason backward from that bottom line to determine that Anthropic must be the good guys.
I don't think they're respected within this space? But of course 'this space' is vague.
By 'this space' I meant AI Safety. I at least see a lot of Anthropic roles being published on 80k and have seen AI safety clubs direct people towards Anthropic fellowships as they would MATS.
I guess the discussion should not turn so much around shaming and blaming particular companies, but changing the structural incentives driving forward the current wave of AI development. Under current circumstances, any company with any CEO with any personnel would largely be incentivised to take the path that Anthropic, OpenAI etc. have taken.