I know very little about this topic, so please forgive me if this is an ignorant question.
It seems to me that it would be a good idea for the U.S. federal government to ban all private companies from working with sufficiently large models/building AGI (and encourage other countries to do the same). It should then create a new agency dedicated to creating aligned AGI and hire all the bright minds who were just put out of work. This agency's activity should be autonomous; the point is to get everyone on the same team, not to police the research process.
Here's my argument:
- AGI development is a winner-takes-all situation.
- Because it is a winner-takes-all situation, in the status quo, companies are highly incentivized to prioritze capabilities over safety/alignment; they don't want anyone else to beat them to AGI.
- Merging all these companies will improve their capabilities, and there will be less external competition, so the agency will have less fear of a competitor beating it to AGI; thus, it can spend relatively more time on alignment than capabilities. This increase the odds of creating an aligned AGI.
- As a bonus, this would probably increases the odds that the AGI is aligned with someone who is benevolent and cares about the general public.
I've never heard anyone suggest that AI be nationalized before, so I feel like I must be missing something. What are some of the problems with this proposal?
Thanks for the thoughtful response, Ryan.
Why do you think the increase in racing between nations would outweigh the decrease in racing between companies? I have the opposite intuition, especially if the government strikes a cosmopolitan tone: "This isn't an arms race; this is a global project to better humanity. We want every talented person from every nation to come work on this with us. We publicly commit to using AGI to do ___ and not ___."
I have trouble understanding why a nation would initiate violent conflict with the U.S. over this. What might that ... (read more)