Anthropic just published their submission to the Request for Information for a new US AI Action Plan (OSTP RFI)

It's 10 pages total and focuses on strengthening the US AISI (and broadly government capacity to test models), strengthening export controls (with some very concrete proposals), enhancing lab security through standards, scaling up energy infrastructure (asking for building infrastructure for 50 GW of power, or  about 4% of the entire US grid capacity), accelerating AI adoption in government, and improving government sensemaking around economic impacts.

I recommend reading it. It's quite insightful regarding the priorities of Anthropic's policy team right now.  

41

2
0

Reactions

2
0
Comments6


Sorted by Click to highlight new comments since:

Here's my summary of the recommendations:

  • National security testing
    • Develop robust government capabilities to evaluate AI models (foreign and domestic) for security risks
    • Once ASL-3 is reached, government should mandate pre-deployment testing
    • Preserve the AI Safety Institute in the Department of Commerce to advance third-party testing
    • Direct NIST to develop comprehensive national security evaluations in partnership with frontier AI developers
    • Build classified and unclassified computing infrastructure for testing powerful AI systems
    • Assemble interdisciplinary teams with both technical AI and national security expertise
       
  • Export Control Enhancement
    • Tighten semiconductor export restrictions to prevent adversaries from accessing critical AI infrastructure
    • Control H20 chips
    • Require government-to-government agreements for countries hosting large chip deployments
      • As a prerequisite for hosting data centers with more than 50,000 chips from U.S. companies, the U.S. should mandate that countries at high-risk for chip smuggling comply with a government-to-government agreement that 1) requires them to align their export control systems with the U.S., 2) takes security measures to address chip smuggling to China, and 3) stops their companies from working with the Chinese military. The “Diffusion Rule” already contains the possibility for such agreements, laying a foundation for further policy development.
    • Review and reduce the 1,700 H100 no-license required threshold for Tier 2 countries
      • Currently, the Diffusion Rule allows advanced chip orders from Tier 2 countries for less than 1,700 H100s —an approximately $40 million order—to proceed without review. These orders do not count against the Rule’s caps, regardless of the purchaser. While these thresholds address legitimate commercial purposes, we believe that they also pose smuggling risks. We recommend that the Administration consider reducing the number of H100s that Tier 2 countries can purchase without review to further mitigate smuggling risks.
    • Increase funding for Bureau of Industry and Security (BIS) for export enforcement
       
  • Lab Security Improvements
    • Establish classified and unclassified communication channels between AI labs and intelligence agencies for threat intelligence sharing, similar to Information Sharing and Analysis Centers used in critical infrastructure sectors
    • Create systematic collaboration between frontier AI companies and intelligence agencies, including Five Eyes partners
    • Elevate collection and analysis of adversarial AI development to a top intelligence priority, as to provide strategic warning and support export controls
    • Expedite security clearances for AI industry professionals
    • Direct NIST to develop next-generation security standards for AI training/inference clusters
    • Develop confidential computing technologies that protect model weights even during processing
    • Develop meaningful incentives for implementing enhanced security measures via procurement requirements for systems supporting federal government deployments.
    • Direct DOE/DNI to conduct a study on advanced security requirements that may become appropriate to ensure sufficient control over and security of highly agentic models

 

  • Energy Infrastructure Scaling
    • Set an ambitious national target: build 50 additional gigawatts of power dedicated to AI by 2027
    • Streamline permitting processes for energy projects by accelerating reviews and enforcing timelines
    • Expedite transmission line approvals to connect new energy sources to data centers
    • Work with state/local governments to reduce permitting burdens
    • Leverage federal real estate for co-locating power generation and next-gen data centers

 

  • Government AI Adoption
    • across the whole of government, the Administration should systematically identify every instance where federal employees process text, images, audio, or video data, and augment these workflows with appropriate AI systems.
    • Task OMB to address resource constraints and procurement limitations for AI adoption
    • Eliminate regulatory and procedural barriers to rapid AI deployment across agencies
    • Direct DoD and Intelligence Community to accelerate AI research, development and procurement
    • Target largest civilian programs for AI implementation (IRS tax processing, VA healthcare delivery, etc.)

 

  • Economic Impact Monitoring
    • Enhance data collection mechanisms to track AI adoption patterns and economic implications
    • The Census Bureau’s American Time Use Survey should incorporate specific questions about AI usage, distinguishing between personal and professional applications while gathering detailed information about task types and systems employed.
    • Update Census Bureau surveys to gather detailed information on AI usage and impacts
    • Collect more granular data on tasks performed by workers to create a baseline for monitoring changes
    • Track the relationship between AI computation investments and economic performance
    • Examine how AI adoption might reshape the tax base and cause structural economic shifts
huw
-6
1
7

Nowhere in their RFP do they place restrictions on what kinds of energy capacity they want built. They are asking for a 4% increase in U.S. energy capacity—this is a serious amount of additional CO2 emissions if that capacity isn’t built renewably. But that’s just what they’re asking for now; if they’re serious about building & scaling AGI, they would be asking for much bigger increases, without a strong precedent of carbon-neutrality to back it up. That seems really bad?

Also to pre-empt—the energy capacity has to come before you build an AI powerful enough to ‘solve climate change’. So if they fail to do that, the downside is that they make the problem significantly worse. I think the environmental downsides of attempting to build AGI should be a meaningful part of one’s calculus.

Object-level aside, I suspect they’re aware their audience is the hypersensitive-to-appearances Trump admin, and framing things accordingly. Even basic, common sense points regarding climate change could have a significant cost to the doc’s reception.

Is the assumption here that they would lobby behind the scenes for carbon-neutrality? Because this just sounds like capitulation without a strong line in the sand to me

This is worth considering, but FWIW, 50 GW would be around 10% of US electricity if it runs continuously (the US consumes at a rate of about 500 GW if you divide total consumption by one year). If the new capacity is as clean as the overall electric grid that would be about 2.5% of US emissions (25% of US emissions come from electricity) and 0.35% of global emissions (US emissions are 1/7 of global emissions). 

I'm not going to do this math now but I think if the new capacity is 100% natural gas then that's about as carbon-intense as the US electric grid as a whole, or maybe somewhat worse (the US has a lot of clean energy, but it also has coal plants which are >2x more carbon intense than gas). 100% natural gas would be the worst case, because there is no scenario where the US builds new coal plants.

Completely agree that climate analysis should be a huge part of the scaling AGI equation. I don't buy the "But AGI might solve climate" argument. It might solve everything, but the uncertainty is so huge I don't think we should account for that in any equation - I think we should calculate the "knowns:" and largely ignore the wildly unpredictable "unknowns" here.

Curated and popular this week
 ·  · 4m read
 · 
Forethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar. We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window. More details on our website. Why we exist We think that AGI might come soon (say, modal timelines to mostly-automated AI R&D in the next 2-8 years), and might significantly accelerate technological progress, leading to many different challenges. We don’t yet have a good understanding of what this change might look like or how to navigate it. Society is not prepared. Moreover, we want the world to not just avoid catastrophe: we want to reach a really great future. We think about what this might be like (incorporating moral uncertainty), and what we can do, now, to build towards a good future. Like all projects, this started out with a plethora of Google docs. We ran a series of seminars to explore the ideas further, and that cascaded into an organization. This area of work feels to us like the early days of EA: we’re exploring unusual, neglected ideas, and finding research progress surprisingly tractable. And while we start out with (literally) galaxy-brained schemes, they often ground out into fairly specific and concrete ideas about what should happen next. Of course, we’re bringing principles like scope sensitivity, impartiality, etc to our thinking, and we think that these issues urgently need more morally dedicated and thoughtful people working on them. Research Research agendas We are currently pursuing the following perspectives: * Preparing for the intelligence explosion: If AI drives explosive growth there will be an enormous number of challenges we have to face. In addition to misalignment risk and biorisk, this potentially includes: how to govern the development of new weapons of mass destr
jackva
 ·  · 3m read
 · 
 [Edits on March 10th for clarity, two sub-sections added] Watching what is happening in the world -- with lots of renegotiation of institutional norms within Western democracies and a parallel fracturing of the post-WW2 institutional order -- I do think we, as a community, should more seriously question our priors on the relative value of surgical/targeted and broad system-level interventions. Speaking somewhat roughly, with EA as a movement coming of age in an era where democratic institutions and the rule-based international order were not fundamentally questioned, it seems easy to underestimate how much the world is currently changing and how much riskier a world of stronger institutional and democratic backsliding and weakened international norms might be. Of course, working on these issues might be intractable and possibly there's nothing highly effective for EAs to do on the margin given much attention to these issues from society at large. So, I am not here to confidently state we should be working on these issues more. But I do think in a situation of more downside risk with regards to broad system-level changes and significantly more fluidity, it seems at least worth rigorously asking whether we should shift more attention to work that is less surgical (working on specific risks) and more systemic (working on institutional quality, indirect risk factors, etc.). While there have been many posts along those lines over the past months and there are of course some EA organizations working on these issues, it stil appears like a niche focus in the community and none of the major EA and EA-adjacent orgs (including the one I work for, though I am writing this in a personal capacity) seem to have taken it up as a serious focus and I worry it might be due to baked-in assumptions about the relative value of such work that are outdated in a time where the importance of systemic work has changed in the face of greater threat and fluidity. When the world seems to
Sam Anschell
 ·  · 6m read
 · 
*Disclaimer* I am writing this post in a personal capacity; the opinions I express are my own and do not represent my employer. I think that more people and orgs (especially nonprofits) should consider negotiating the cost of sizable expenses. In my experience, there is usually nothing to lose by respectfully asking to pay less, and doing so can sometimes save thousands or tens of thousands of dollars per hour. This is because negotiating doesn’t take very much time[1], savings can persist across multiple years, and counterparties can be surprisingly generous with discounts. Here are a few examples of expenses that may be negotiable: For organizations * Software or news subscriptions * Of 35 corporate software and news providers I’ve negotiated with, 30 have been willing to provide discounts. These discounts range from 10% to 80%, with an average of around 40%. * Leases * A friend was able to negotiate a 22% reduction in the price per square foot on a corporate lease and secured a couple months of free rent. This led to >$480,000 in savings for their nonprofit. Other negotiable parameters include: * Square footage counted towards rent costs * Lease length * A tenant improvement allowance * Certain physical goods (e.g., smart TVs) * Buying in bulk can be a great lever for negotiating smaller items like covid tests, and can reduce costs by 50% or more. * Event/retreat venues (both venue price and smaller items like food and AV) * Hotel blocks * A quick email with the rates of comparable but more affordable hotel blocks can often save ~10%. * Professional service contracts with large for-profit firms (e.g., IT contracts, office internet coverage) * Insurance premiums (though I am less confident that this is negotiable) For many products and services, a nonprofit can qualify for a discount simply by providing their IRS determination letter or getting verified on platforms like TechSoup. In my experience, most vendors and companies