Comment Permalink

I've only just CTL-F'd the report so I could have missed something, but I guess the key question for me is what does a multilateral project mean in terms of security/diffusion of the technology?

My intuition is that preventing diffusion of the tech in a multilateral project would be hard, if not impossible and I see this as consideration as something that could kill the desirability of such a project by itself, even if there are several other strong arguments in favour.

I know you mention this in the potential future work section, but I do think it is worthwhile editing in a paragraph or two on why you think we might want to consider this model anyway (it's impossible to address everyone's pet objection, but my guess is that this will prove to be one of the major objections that people make).

rosehadsharMar 242

Sorry for the slow response here! Agree that diffusion is an important issue. A few thoughts:

Some forms of diffusion might be actively good, for reducing concentration of power. So it's not clear that we want to straightforwardly prevent tech diffusion
Ways you could reduce tech diffusion within something like Intelsat:
- Limited membership helps
- You could do things like require companies it contracts with to comply with strong infosec, require members not to allow frontier development without strong infosec, require member governments to provide gov-level info

... (read more)

rosehadshar

Mar 24

Sorry for the slow response here! Agree that diffusion is an important issue. A few thoughts: * Some forms of diffusion might be actively good, for reducing concentration of power. So it's not clear that we want to straightforwardly prevent tech diffusion * Ways you could reduce tech diffusion within something like Intelsat: * Limited membership helps * You could do things like require companies it contracts with to comply with strong infosec, require members not to allow frontier development without strong infosec, require member governments to provide gov-level infosec to frontier developers in their countries * Intelsat for satellites involved sharing all the technical information. For AGI, it could involve sharing only some forms of information (e.g. weights don't get shared with everyone, but encrypted chunks of the weights are distributed among founder members) * h/t Will: having many countries part of the multilateral project removes their incentives to try to develop frontier AI themselves (and potentially open-source)

See in context

Intelsat as a Model for International AGI Governance

by rosehadshar, Forethought, William_MacAskill

Mar 131 min read 3

40

AI safetyPolicyHistoryAI governanceInternational relations

Frontpage

This is a linkpost for https://www.forethought.org/research/intelsat-as-a-model-for-international-agi-governance

If there is an international project to build artificial general intelligence (“AGI”), how should it be designed? Existing scholarship has looked to historical models for inspiration, often suggesting the Manhattan Project or CERN as the closest analogues. But AGI is a fundamentally general-purpose technology, and is likely to be used primarily for commercial purposes rather than military or scientific ones.

This report presents an under-discussed alternative: Intelsat, an international organization founded to establish and own the global satellite communications system. We show that Intelsat is proof of concept that a multilateral project to build a commercially and strategically important technology is possible and can achieve intended objectives—providing major benefits to both the US and its allies compared to the US acting alone. We conclude that ‘Intelsat for AGI’ is a valuable complement to existing models of AGI governance.

40 Reactions

Comments3

Sorted by

New & upvoted

Click to highlight new comments since: Today at 4:40 PM

Chris LeongMar 152

I've only just CTL-F'd the report so I could have missed something, but I guess the key question for me is what does a multilateral project mean in terms of security/diffusion of the technology?

rosehadsharMar 242

Sorry for the slow response here! Agree that diffusion is an important issue. A few thoughts:

Some forms of diffusion might be actively good, for reducing concentration of power. So it's not clear that we want to straightforwardly prevent tech diffusion
Ways you could reduce tech diffusion within something like Intelsat:
- Limited membership helps
- You could do things like require companies it contracts with to comply with strong infosec, require members not to allow frontier development without strong infosec, require member governments to provide gov-level infosec to frontier developers in their countries
- Intelsat for satellites involved sharing all the technical information. For AGI, it could involve sharing only some forms of information (e.g. weights don't get shared with everyone, but encrypted chunks of the weights are distributed among founder members)
- h/t Will: having many countries part of the multilateral project removes their incentives to try to develop frontier AI themselves (and potentially open-source)

rosehadsharMar 242

Sorry for the slow response here! Agree that diffusion is an important issue. A few thoughts:

Some forms of diffusion might be actively good, for reducing concentration of power. So it's not clear that we want to straightforwardly prevent tech diffusion
Ways you could reduce tech diffusion within something like Intelsat:
- Limited membership helps
- You could do things like require companies it contracts with to comply with strong infosec, require members not to allow frontier development without strong infosec, require member governments to provide gov-level infosec to frontier developers in their countries
- Intelsat for satellites involved sharing all the technical information. For AGI, it could involve sharing only some forms of information (e.g. weights don't get shared with everyone, but encrypted chunks of the weights are distributed among founder members)
- h/t Will: having many countries part of the multilateral project removes their incentives to try to develop frontier AI themselves (and potentially open-source)