In the wake of this community's second major brush with a sketchy tech CEO named Sam, I'd like to offer two observations.
- Don't panic: 90% of EAs or alignment-friendly tech CEOs are good people. I don't know of anything bad Dario Amodei has done (knock on wood). This doesn't reflect a fundamental problem in EA or at AI companies.
- Corollary: 10% aren't.
After SBF, EAs panicked and began the ritual self-flagellation about what they should have done differently. For people who dropped the guy like a hot potato and recognized he was a psychopath: Not much. You think someone should've investigated him? Why? You investigate someone before you give them money, not before they give you money. I don't launch a full-scale investigation of my boss every time he sends me a paycheck.
But a substantial number of EAs spent the next couple of weeks or months making excuses not to call a spade a spade, or an amoral serial liar an amoral serial liar. This continued even after we knew he'd A) committed massive fraud, B) used that money to buy himself a $222 million house, and C) referred to ethics as a "dumb reputation game" in an interview with Kelsey Piper.
This wasn't because they thought the fraud was good; everyone was clear that SBF was very bad. It's because a surprisingly big number of people can't identify a psychopath. I'd like to offer a lesson on how to tell. If someone walks up to you and says "I'm a psychopath", they're probably a psychopath.
"But Sam Altman never said that. Could we really have predicted this a year ag—"
Clearly the quote was a joke, I don't think he actually did anything bad for AI safe—
While Sam Altman publicly claimed to support AI regulation, he was also spending millions of dollars killing any meaningful regulations.
Anyone would've done the same things, the incentives to race are—
"If my company doesn't do this thing that might destroy the world, I will lose a lot of money" is not a good defense for doing things that might destroy the world. In fact, doing unethical things for personal benefit is bad.
But did he do anything clearly unethical before—
For starters, he was fired from Y Combinator for abusing his position for personal profit.
Stop overthinking it: some people and organizations are unethical. Recognizing this and pointing out they're bad is helpful because it lets you get rid of them. It's not about getting angry at them; it's just that past behavior predicts future behavior. I wish Sam Altman the very best, so long as it's far away from any position of power where he can influence the future of humanity.
(In related advice, if a company builds an AI that is obviously violent and threatening towards you, you should probably go "Hmm, maybe this company will build AIs that are violent and threatening towards me" instead of finding a 12d chess explanation for why OAI is actually good for AI safety.)
Off the cuff answers that may change as I reflect more:
I think incentives matter, but I feel like if they're all that matters, then we're doomed anyway because "Who will step up as a leader to set good incentives?" In other words, the position "incentives are all that matters" seems self-defeating, because to change things, you can't just sit on the sidelines and criticize "the incentives" or "the system." It also seems too cynical: just because, e.g., lots of money is at stake, that doesn't mean people who were previously morally motivated and cautious about their motivations and trying to do the right thing, will suddenly go off the rails.
To be clear, I think there's probably a limit for everyone and no person is forever safe from corruption, but my point is that it matters where on the spectrum someone falls. Of the people that are low on corruptibility, even though most of them don't like power or would flail around helplessly and hopelessly if they had it, there are probably people who have the right mix of traits to create, maintain and grow pockets of sanity (well-run, well-functioning organizations, ecosystems, etc.).