Effective Altruism Forum
EA Forum

Comment Permalink
Ben Stewart4
1
0

I don't know. My guess is that they give very slim odds to the Trump admin caring about carbon neutrality, and think that the benefit of including a mention in their submission to be close to zero (other than demonstrating resolve in their principles to others). 

On the minus side, such a mention risks a reaction with significant cost to their AI safety/security asks. So overall, I can see them thinking that including a mention does not make sense for their strategy. I'm not endorsing that calculus, just conjecturing.

Reply
See in context
by Agustín Covarrubias 🔸
1 min read 7

47

AI safetyPolicyExistential riskAI governanceAnthropicUS policy
Frontpage
This is a linkpost for https://www.anthropic.com/news/anthropic-s-recommendations-ostp-u-s-ai-action-plan

Anthropic just published their submission to the Request for Information for a new US AI Action Plan (OSTP RFI). 

It's 10 pages total and focuses on strengthening the US AISI (and broadly government capacity to test models), strengthening export controls (with some very concrete proposals), enhancing lab security through standards, scaling up energy infrastructure (asking for building infrastructure for 50 GW of power, or  about 4% of the entire US grid capacity), accelerating AI adoption in government, and improving government sensemaking around economic impacts.

I recommend reading it. It's quite insightful regarding the priorities of Anthropic's policy team right now.  

47

2
0

Reactions

2
0
Comments7


Sorted by
New & upvoted
Click to highlight new comments since:
Peter Wildeford12
0
0
1

Here's my summary of the recommendations:

  • National security testing
    • Develop robust government capabilities to evaluate AI models (foreign and domestic) for security risks
    • Once ASL-3 is reached, government should mandate pre-deployment testing
    • Preserve the AI Safety Institute in the Department of Commerce to advance third-party testing
    • Direct NIST to develop comprehensive national security evaluations in partnership with frontier AI developers
    • Build classified and unclassified computing infrastructure for testing powerful AI systems
    • Assemble interdisciplinary teams with both technical AI and national security expertise
       
  • Export Control Enhancement
    • Tighten semiconductor export restrictions to prevent adversaries from accessing critical AI infrastructure
    • Control H20 chips
    • Require government-to-government agreements for countries hosting large chip deployments
      • As a prerequisite for hosting data centers with more than 50,000 chips from U.S. companies, the U.S. should mandate that countries at high-risk for chip smuggling comply with a government-to-government agreement that 1) requires them to align their export control systems with the U.S., 2) takes security measures to address chip smuggling to China, and 3) stops their companies from working with the Chinese military. The “Diffusion Rule” already contains the possibility for such agreements, laying a foundation for further policy development.
    • Review and reduce the 1,700 H100 no-license required threshold for Tier 2 countries
      • Currently, the Diffusion Rule allows advanced chip orders from Tier 2 countries for less than 1,700 H100s —an approximately $40 million order—to proceed without review. These orders do not count against the Rule’s caps, regardless of the purchaser. While these thresholds address legitimate commercial purposes, we believe that they also pose smuggling risks. We recommend that the Administration consider reducing the number of H100s that Tier 2 countries can purchase without review to further mitigate smuggling risks.
    • Increase funding for Bureau of Industry and Security (BIS) for export enforcement
       
  • Lab Security Improvements
    • Establish classified and unclassified communication channels between AI labs and intelligence agencies for threat intelligence sharing, similar to Information Sharing and Analysis Centers used in critical infrastructure sectors
    • Create systematic collaboration between frontier AI companies and intelligence agencies, including Five Eyes partners
    • Elevate collection and analysis of adversarial AI development to a top intelligence priority, as to provide strategic warning and support export controls
    • Expedite security clearances for AI industry professionals
    • Direct NIST to develop next-generation security standards for AI training/inference clusters
    • Develop confidential computing technologies that protect model weights even during processing
    • Develop meaningful incentives for implementing enhanced security measures via procurement requirements for systems supporting federal government deployments.
    • Direct DOE/DNI to conduct a study on advanced security requirements that may become appropriate to ensure sufficient control over and security of highly agentic models

 

  • Energy Infrastructure Scaling
    • Set an ambitious national target: build 50 additional gigawatts of power dedicated to AI by 2027
    • Streamline permitting processes for energy projects by accelerating reviews and enforcing timelines
    • Expedite transmission line approvals to connect new energy sources to data centers
    • Work with state/local governments to reduce permitting burdens
    • Leverage federal real estate for co-locating power generation and next-gen data centers

 

  • Government AI Adoption
    • across the whole of government, the Administration should systematically identify every instance where federal employees process text, images, audio, or video data, and augment these workflows with appropriate AI systems.
    • Task OMB to address resource constraints and procurement limitations for AI adoption
    • Eliminate regulatory and procedural barriers to rapid AI deployment across agencies
    • Direct DoD and Intelligence Community to accelerate AI research, development and procurement
    • Target largest civilian programs for AI implementation (IRS tax processing, VA healthcare delivery, etc.)

 

  • Economic Impact Monitoring
    • Enhance data collection mechanisms to track AI adoption patterns and economic implications
    • The Census Bureau’s American Time Use Survey should incorporate specific questions about AI usage, distinguishing between personal and professional applications while gathering detailed information about task types and systems employed.
    • Update Census Bureau surveys to gather detailed information on AI usage and impacts
    • Collect more granular data on tasks performed by workers to create a baseline for monitoring changes
    • Track the relationship between AI computation investments and economic performance
    • Examine how AI adoption might reshape the tax base and cause structural economic shifts
Reply
huw-7
1
8

Nowhere in their RFP do they place restrictions on what kinds of energy capacity they want built. They are asking for a 4% increase in U.S. energy capacity—this is a serious amount of additional CO2 emissions if that capacity isn’t built renewably. But that’s just what they’re asking for now; if they’re serious about building & scaling AGI, they would be asking for much bigger increases, without a strong precedent of carbon-neutrality to back it up. That seems really bad?

Also to pre-empt—the energy capacity has to come before you build an AI powerful enough to ‘solve climate change’. So if they fail to do that, the downside is that they make the problem significantly worse. I think the environmental downsides of attempting to build AGI should be a meaningful part of one’s calculus.

Reply
Ben Stewart9
1
0

Object-level aside, I suspect they’re aware their audience is the hypersensitive-to-appearances Trump admin, and framing things accordingly. Even basic, common sense points regarding climate change could have a significant cost to the doc’s reception.

Reply
huw2
0
1

Is the assumption here that they would lobby behind the scenes for carbon-neutrality? Because this just sounds like capitulation without a strong line in the sand to me

Reply
Ben Stewart4
1
0

I don't know. My guess is that they give very slim odds to the Trump admin caring about carbon neutrality, and think that the benefit of including a mention in their submission to be close to zero (other than demonstrating resolve in their principles to others). 

On the minus side, such a mention risks a reaction with significant cost to their AI safety/security asks. So overall, I can see them thinking that including a mention does not make sense for their strategy. I'm not endorsing that calculus, just conjecturing.

Reply
JoshYou5
0
0

This is worth considering, but FWIW, 50 GW would be around 10% of US electricity if it runs continuously (the US consumes at a rate of about 500 GW if you divide total consumption by one year). If the new capacity is as clean as the overall electric grid that would be about 2.5% of US emissions (25% of US emissions come from electricity) and 0.35% of global emissions (US emissions are 1/7 of global emissions). 

I'm not going to do this math now but I think if the new capacity is 100% natural gas then that's about as carbon-intense as the US electric grid as a whole, or maybe somewhat worse (the US has a lot of clean energy, but it also has coal plants which are >2x more carbon intense than gas). 100% natural gas would be the worst case, because there is no scenario where the US builds new coal plants.

Reply
NickLaing3
2
5

Completely agree that climate analysis should be a huge part of the scaling AGI equation. I don't buy the "But AGI might solve climate" argument. It might solve everything, but the uncertainty is so huge I don't think we should account for that in any equation - I think we should calculate the "knowns:" and largely ignore the wildly unpredictable "unknowns" here.

Reply
More from Agustín Covarrubias 🔸
292
Kelsey Piper's recent interview of SBF
Agustín Covarrubias 🔸
· · 2m read
155
155
54
UK announces early warning system for pandemics
Agustín Covarrubias 🔸
· · 1m read
0
0
66
Among the A.I. Doomsayers - The New Yorker
Agustín Covarrubias 🔸
· · 1m read
0
0
View more
Curated and popular this week
LintzA
 ·  · 15m read
 · 
Cross-posted to Lesswrong Introduction Several developments over the past few months should cause you to re-evaluate what you are doing. These include: 1. Updates toward short timelines 2. The Trump presidency 3. The o1 (inference-time compute scaling) paradigm 4. Deepseek 5. Stargate/AI datacenter spending 6. Increased internal deployment 7. Absence of AI x-risk/safety considerations in mainstream AI discourse Taken together, these are enough to render many existing AI governance strategies obsolete (and probably some technical safety strategies too). There's a good chance we're entering crunch time and that should absolutely affect your theory of change and what you plan to work on. In this piece I try to give a quick summary of these developments and think through the broader implications these have for AI safety. At the end of the piece I give some quick initial thoughts on how these developments affect what safety-concerned folks should be prioritizing. These are early days and I expect many of my takes will shift, look forward to discussing in the comments!  Implications of recent developments Updates toward short timelines There’s general agreement that timelines are likely to be far shorter than most expected. Both Sam Altman and Dario Amodei have recently said they expect AGI within the next 3 years. Anecdotally, nearly everyone I know or have heard of who was expecting longer timelines has updated significantly toward short timelines (<5 years). E.g. Ajeya’s median estimate is that 99% of fully-remote jobs will be automatable in roughly 6-8 years, 5+ years earlier than her 2023 estimate. On a quick look, prediction markets seem to have shifted to short timelines (e.g. Metaculus[1] & Manifold appear to have roughly 2030 median timelines to AGI, though haven’t moved dramatically in recent months). We’ve consistently seen performance on benchmarks far exceed what most predicted. Most recently, Epoch was surprised to see OpenAI’s o3 model achi
Rory Fenton
 ·  · 6m read
 · 
Cross-posted from my blog. Contrary to my carefully crafted brand as a weak nerd, I go to a local CrossFit gym a few times a week. Every year, the gym raises funds for a scholarship for teens from lower-income families to attend their summer camp program. I don’t know how many Crossfit-interested low-income teens there are in my small town, but I’ll guess there are perhaps 2 of them who would benefit from the scholarship. After all, CrossFit is pretty niche, and the town is small. Helping youngsters get swole in the Pacific Northwest is not exactly as cost-effective as preventing malaria in Malawi. But I notice I feel drawn to supporting the scholarship anyway. Every time it pops in my head I think, “My money could fully solve this problem”. The camp only costs a few hundred dollars per kid and if there are just 2 kids who need support, I could give $500 and there would no longer be teenagers in my town who want to go to a CrossFit summer camp but can’t. Thanks to me, the hero, this problem would be entirely solved. 100%. That is not how most nonprofit work feels to me. You are only ever making small dents in important problems I want to work on big problems. Global poverty. Malaria. Everyone not suddenly dying. But if I’m honest, what I really want is to solve those problems. Me, personally, solve them. This is a continued source of frustration and sadness because I absolutely cannot solve those problems. Consider what else my $500 CrossFit scholarship might do: * I want to save lives, and USAID suddenly stops giving $7 billion a year to PEPFAR. So I give $500 to the Rapid Response Fund. My donation solves 0.000001% of the problem and I feel like I have failed. * I want to solve climate change, and getting to net zero will require stopping or removing emissions of 1,500 billion tons of carbon dioxide. I give $500 to a policy nonprofit that reduces emissions, in expectation, by 50 tons. My donation solves 0.000000003% of the problem and I feel like I have f
 ·  · 8m read
 · 
In my past year as a grantmaker in the global health and wellbeing (GHW) meta space at Open Philanthropy, I've identified some exciting ideas that could fill existing gaps. While these initiatives have significant potential, they require more active development and support to move forward.  The ideas I think could have the highest impact are:  1. Government placements/secondments in key GHW areas (e.g. international development), and 2. Expanded (ultra) high-net-worth ([U]HNW) advising Each of these ideas needs a very specific type of leadership and/or structure. More accessible options I’m excited about — particularly for students or recent graduates — could involve virtual GHW courses or action-focused student groups.  I can’t commit to supporting any particular project based on these ideas ahead of time, because the likelihood of success would heavily depend on details (including the people leading the project). Still, I thought it would be helpful to articulate a few of the ideas I’ve been considering.  I’d love to hear your thoughts, both on these ideas and any other gaps you see in the space! Introduction I’m Mel, a Senior Program Associate at Open Philanthropy, where I lead grantmaking for the Effective Giving and Careers program[1] (you can read more about the program and our current strategy here). Throughout my time in this role, I’ve encountered great ideas, but have also noticed gaps in the space. This post shares a list of projects I’d like to see pursued, and would potentially want to support. These ideas are drawn from existing efforts in other areas (e.g., projects supported by our GCRCB team), suggestions from conversations and materials I’ve engaged with, and my general intuition. They aren’t meant to be a definitive roadmap, but rather a starting point for discussion. At the moment, I don’t have capacity to more actively explore these ideas and find the right founders for related projects. That may change, but for now, I’m interested in
Recent opportunities in AI safety
57
Quick nudge to apply to the LTFF grant round (closing on Saturday)
calebp
· · 1m read
7
7
13
Last week to apply for the Futurekind AI Fellowship! (deadline: April 1)
Jay Luong
· · 1m read
0
0
31
Australians for AI Safety Launches New Election Campaign — Here's How You Can Help
Luke Freeman 🔸
· · 3m read
5
5
View more