Wiki Contributions


What is the EU AI Act and why should you care about it?

Excellent overview, and I completely agree that the AI Act is an important policy for AI governance.

One quibble: as far as I know, the Center for Data Innovation is just a lobbying group for Big Tech - I was a little surprised to see it listed in "public responses from various EA and EA Adjacent organisations".

Lessons for AI governance from the Biological Weapons Convention

Hi Aryan,

Cool post, very interesting! I'm fascinated by this topic - the PhD thesis I'm writing is on nuclear, bio and cyber weapons arms control regimes and what lessons can be drawn for AI. So obviously I'm very into this, and want to see more work done on this. Really excellent to see you exploring the parallels. A few thoughts:

  • Your point on 'lock-in' seems crucial. It currently seems to me that there are 'critical junctures' (Capoccia) in which regimes get set and then its very hard to change them. So e.g. the failure to control nukes or cyber in early years. ABM is a complex example - very very hard to get back on the table, but Rumsfeld +others managed it after 30 years of battling.
  • My impression is that the BWC (and CWC) - the meetings/conferences etc - are often seen as arms control regimes that are pretty good at keeping up with technical developments - maybe a point in favour of centralisation.
  • Just on the details of the BWC, seems worth mentioning a few things. (Nitpicky: when the UK proposed a BWC, it said verification wasn't technically possible at the time [1]). First, the Nixon Administration thought BW were militarily useless and had already unilaterally disarmed, so verification was less of a priority [2]. Second, one of the reasons to want a Verification Protocol in the 90s was the revelation that the Soviets cheated over the 70s-80s, building the biggest BW program ever. Third, the Bush Admin rejected the Verification Protocol in 2001 (pre 9/11!), its first year - at the same time as it was ripping up START III, Kyoto, and the ABM Treaty. This is all to suggest that state interest, and elites' changing conceptions of state interest, can create space for change.

[1] http://www.cbw-events.org.uk/EX1968.PDF 

[2] https://www.belfercenter.org/publication/farewell-germs-us-renunciation-biological-and-toxin-warfare-1969-70


What EA projects could grow to become megaprojects, eventually spending $100m per year?

Interesting first point, but I disagree. To me, the increased salience of climate change in recent years can be traced back to the 2018 Special Report on Global Warming of 1.5 °C (SR15), and in particular the meme '12 years to save the world'. Seems to have contributed to the start of School Strike for Climate, Extinction Rebellion and the Green New Deal. Another big new scary IPCC report on catastrophic climate change would further raise the salience of this issue-area.

I was thinking that $100m would be for all four of these topics, and that we'd  get cause-prioritisation VOI across all four of these areas. $100m for impact and VOI across all four seems pretty good to me (however I'm a researcher not a funder!)

On solar geo, I'm not an expert on it and am not arguing for it myself, merely reporting that its top of the 'asks' list for orgs like Silver Lining.

I actually rather like the framing in Xu & Ram - I don't think we know enough about >5 °C scenarios, so describing them as "unknown, implying beyond catastrophic, including existential threats" seems pretty reasonable to me. In any case, I cited that more to demonstrate the lack of research thats been done on these scenarios.

Most research/advocacy charities are not scalable

I think its a really good point that there's something very different between research/policy orgs and orgs that deliver products and services at scale. I basically agree, but I'd slightly tweak this to
"It is very hard for a charity to scale to more than $100 million per year without delivering a physical product or service."

Because  digital orgs/companies who deliver a digital service (GiveDirectly, Facebook/Google/etc) obviously can scale to $100 million per year. 

What EA projects could grow to become megaprojects, eventually spending $100m per year?

Hell yeah! Get JGL to star - https://www.eaglobal.org/speakers/joseph-gordon-levitt/

What EA projects could grow to become megaprojects, eventually spending $100m per year?

Do you mean just the fourth bullet, or do you think this about all four? 

The 1980s nuclear winter and asteroid papers (I'm thinking especially Sagan et al, and Alvarez et al) were very influential in changing political behaviour - Gorbachev and Reagan explicitly acknowledged that on nuclear, the asteroid evidence contributed to the 90s asteroid films and the (hugely successful!) NASA effort to track all 'dino-killers'. On the margin now, I think more scary stuff would be motivating. There's also VOI in resolving how big a concern nuclear winter is (eg some recent papers are skeptical) - if it turned out to not be as existential as we thought, that would change cause prioritisation for GCRs.

On geoengineering (sorry 'climate interventions'(!)), note 'getting more climate modelling' is a key aim for e.g. Silver Lining

On the fourth one, on the margin, I think more research - especially if it were the basis for an IPCC special report - would be influential. There's also VOI for our cause priotisation. It just is really remarkable how understudied it is!

What EA projects could grow to become megaprojects, eventually spending $100m per year?

Megaprojects cost $1 billion or more. Ben Todd was using the (admittedly somewhat confusing) term 'EA megaproject' by which he meant a new project that could usefully spend $100m a year. So these concerns about megaprojects don't apply.
How about we use the term '$100m-scale project'? (I considered 'kiloproject' but that's really niche.)

What EA projects could grow to become megaprojects, eventually spending $100m per year?

Here's the interesting, frustrating evaluation report:  https://www.macfound.org/media/article_pdfs/nuclear-challenges-synthesis-report_public-final-1.29.21.pdf[16].pdf
Looks to me like a classic hits-based giving bet - you mostly don't make much impact, then occassionaly (Nixon arms control, H.W. Bush's START and Nunn-Lugar, maybe Obama JCPOA/New START) get a home run.

Load More