We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.
AI systems with human-competitive intelligence can pose profound risks to society and humanity, as shown by extensive research and acknowledged by top AI labs. As stated in the widely-endorsed Asilomar AI Principles, Advanced AI could represent a profound change in the history of life on Earth, and should be planned for and managed with commensurate care and resources. Unfortunately, this level of planning and management is not happening, even though recent months have seen AI labs locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one – not even their creators – can understand, predict, or reliably control.
Contemporary AI systems are now becoming human-competitive at general tasks, and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization? Such decisions must not be delegated to unelected tech leaders. Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable. This confidence must be well justified and increase with the magnitude of a system's potential effects. OpenAI's recent statement regarding artificial general intelligence, states that "At some point, it may be important to get independent review before starting to train future systems, and for the most advanced efforts to agree to limit the rate of growth of compute used for creating new models." We agree. That point is now.
Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium.
AI labs and independent experts should use this pause to jointly develop and implement a set of shared safety protocols for advanced AI design and development that are rigorously audited and overseen by independent outside experts. These protocols should ensure that systems adhering to them are safe beyond a reasonable doubt. This does not mean a pause on AI development in general, merely a stepping back from the dangerous race to ever-larger unpredictable black-box models with emergent capabilities.
AI research and development should be refocused on making today's powerful, state-of-the-art systems more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal.
In parallel, AI developers must work with policymakers to dramatically accelerate development of robust AI governance systems. These should at a minimum include: new and capable regulatory authorities dedicated to AI; oversight and tracking of highly capable AI systems and large pools of computational capability; provenance and watermarking systems to help distinguish real from synthetic and to track model leaks; a robust auditing and certification ecosystem; liability for AI-caused harm; robust public funding for technical AI safety research; and well-resourced institutions for coping with the dramatic economic and political disruptions (especially to democracy) that AI will cause.
Humanity can enjoy a flourishing future with AI. Having succeeded in creating powerful AI systems, we can now enjoy an "AI summer" in which we reap the rewards, engineer these systems for the clear benefit of all, and give society a chance to adapt. Society has hit pause on other technologies with potentially catastrophic effects on society. We can do so here. Let's enjoy a long AI summer, not rush unprepared into a fall.
Signatories include Yoshua Bengio, Stuart Russell, Elon Musk, Steve Wozniak, Yuval Noah Hariri, Andrew Yang, Connor Leahy (Conjecture), and Emad Mostaque (Stability).
Edit: covered in NYT, BBC, WaPo, NBC, ABC, CNN, CBS, Time, etc. See also Eliezer's piece in Time.
Edit 2: see FLI's FAQ.
Edit 3: see FLI's report Policymaking in the Pause.
Edit 4: see AAAI's open letter.
I signed and strongly support this open letter.
Let me add a little global perspective (as a US citizen who's lived in 4 countries outside the US for a total of 14 years, and who doesn't always see the US as the 'good guy' in geopolitics).
The US is 4% of the world's population. The American AI industry is (probably) years ahead of any other country, and is pushing ahead with the rationale that 'if we don't keep pushing ahead, a bad actor (which usually implies China) will catch up, and that would be bad'. Thus, we impose AI X-risk on the other 96% of humans without the informed consent, support, or oversight.
We used the same arms-race rationale in the 1940s to develop the atomic bomb ('if we don't do it, Germany will') and in the 1950s to develop the hydrogen bomb ('if we don't do it, the Soviet Union will'). In both cases, we were the bad actor. The other countries were nowhere close to us. We exaggerated the threat that they would catch up, and we got the American public to buy into that narrative. But we were really the ones pushing ahead into X-risk territory. Now we're promoting the same narrative for AI development. 'The AI arms race cannot be stopped', 'AGI is inevitable', 'the genie is out of the bottle', 'if not us, then China', etc, etc.
We Americans have a very hard time accepting that 'we might be the baddies'. We are uncomfortable acknowledging any moral obligations to the rest of humanity (if they conflict in any way with our geopolitical interests). We like to impose our values on the world, but we don't like to submit to any global oversight by others.
I hope that this public discussion about AI risks also includes some soul-searching by Americans -- not just the AI industry, but all of us, concerning the way that our country is, yet again, pushing ahead with developing extremely dangerous technology, without any sense of moral obligation to others.
Having taught online courses for CUHK-Shenzhen in China for a year, and discussed quite a bit about EA, AI, and X risk with the very bright young students there, I often imagine how they would view the recent developments in the American AI industry. I think they would be appalled by our American hubris. They know that the American political system is too partisan, fractured, slow, and dysfunctional to impose any effective regulation on Big Tech. They know that American tech companies are legally obligated (by 'fiduciary duty' to shareholders) to prioritize quarterly profits over long-term human survival. They know that many Bay Area tech bros supporting AI are transhumanists, extropians, or Singularity-welcomers who look forward to humanity being replaced by machines. They know that many Americans view China as a reckless, irresponsible, totalitarian state that isn't worth listening to about any AI safety concerns. So, I imagine, any young Chinese students who's paying attention would take an extremely negative view of the risks that the American AI industry is imposing on the other 7.7 billion people in the world.
Not an expert either, but safest to say the corporate-law question is nuanced and not free from doubt. It's pretty clear there's no duty to maximize short-term profits, though.
But we can surmise that most boards that allow the corporation to seriously curtail its profits -- at least its medium-term profits -- will get replaced by shareholders soon enough. So the end result is largely the same.