Hide table of contents

Advanced AI could unlock an era of enlightened and competent government action. But without smart, active investment, we’ll squander that opportunity and barrel blindly into danger.

Executive summary

See also a summary on Twitter / X.

The US federal government is falling behind the private sector on AI adoption. As AI improves, a growing gap would leave the government unable to effectively respond to AI-driven existential challenges and threaten the legitimacy of its democratic institutions.

A dual imperative

Government adoption of AI can’t wait. Making steady progress is critical to:

  • Boost the government’s capacity to effectively respond to AI-driven existential challenges
  • Help democratic oversight keep up with the technological power of other groups
  • Defuse the risk of rushed AI adoption in a crisis

But hasty AI adoption could backfire. Without care, integration of AI could:

  • Be exploited, subverting independent government action
  • Lead to unsafe deployment of AI systems
  • Accelerate arms races or compress safety research timelines

Summary of the recommendations

1. Work with the US federal government to help it effectively adopt AI

Simplistic “pro-security” or “pro-speed” attitudes miss the point. Both are important — and many interventions would help with both. We should:

  • Invest in win-win measures that both facilitate adoption and reduce the risks involved, e.g.:
    • Build technical expertise within government (invest in AI and technical talent, ensure NIST is well resourced)
    • Streamline procurement processes for AI products and related tech (like cloud services)
    • Modernize the government’s digital infrastructure and data management practices
  • Prioritize high-leverage interventions that have strong adoption-boosting benefits with minor security costs or vice versa, e.g.:
    • On the security side: investing in cyber security, pre-deployment testing of AI in high-stakes areas, and advancing research on mitigating the risks of advanced AI
    • On the adoption side: helping key agencies adopt AI and ensuring that advanced AI tools will be usable in government settings

2. Develop contingency plans and build capacity outside the US federal government

Current trends suggest slow government AI adoption. This makes it important to prepare for two risky scenarios:

  • State capacity collapse: the US federal government is largely ineffective or extremely low-capacity and vulnerable
    • We should build backstops for this scenario, e.g. by developing private or non-US alternatives for key government functions, or working directly with AI companies on safety and voluntary governance
  • Rushed, late-stage government AI adoption: after a crisis or sudden shift in priorities, the US federal government rapidly ramps up integration of advanced AI systems
    • We should try to create a safety net for this scenario, e.g. by preparing “emergency teams” of AI experts who can be seconded into the government, or by identifying key pitfalls and recommending (ideally lightweight) guardrails for avoiding them

These scenarios might render a lot of current risk mitigation work irrelevant, seem worryingly probable, and will get little advance attention by default — more preparation is warranted.

What does “government adoption of AI” mean?

“AI” can refer to a wide range of things — from natural language processing to broad, generative AI systems like GPT-4. This piece focuses on frontier AI systems and their applications, including today’s large language models and more advanced tools that will become available in the coming years.

“Adoption” of AI can happen at different layers of an organization. On the “micro” level, it could mean making sure individual employees are using state-of-the-art assistants (or more specialized AI tools) to improve and speed up their task performance. Individuals’ use of AI is important and in scope, but not the sole focus of this piece; especially as AI capabilities improve, integrating AI could involve deeper or structural changes, e.g. automating institutional processes or more fundamentally reshaping how decisions are made and services are provided.

What could deeper AI integration of AI look like in the US federal government?

We could see:

  • The Bureau of Industry and Security (BIS) deploying systems that forecast and respond to potential supply chain disruptions and automatically monitor for compliance with export controls
  • The Cybersecurity and Infrastructure Security Agency (CISA) continuously testing for vulnerabilities and maintaining key digital infrastructure
  • The National Institute of Standards and Technology (NIST) rapidly testing new AI systems
  • The Securities and Exchange Commission (SEC) employing systems that scan for early signs of market manipulation and optimize enforcement efforts to recover more illegal profits
  • Secure oversight and reporting systems that identify instances of government waste or corruption and surface that information (with specific recommendations) to relevant congressional committees or Executive Branch officials
  • Legislative platforms and tools that pool a huge amount of information from different stakeholders to identify potential compromises, flag contradictory regulation, and quickly verify whether proposed laws or deals conflict with existing statutes

... as well as the emergence of entirely new, AI-native government functions that, for instance, help address the problem of overlapping or convoluted governance mechanisms (what is known as “kludgeocracy”).

This piece considers AI adoption within the US federal government quite broadly, paying somewhat more attention to the bodies involved in governing emerging technologies and to the civilian side of the government, which is adopting the technology especially slowly. Much of this discussion could also apply to other democratic nations, state governments, and even non-governmental organizations that serve the public interest.

The government is falling behind the private sector on AI adoption

AI is starting to transform our society. State-of-the-art AI models keep getting better, and the costs of running them are plummeting. More and more companies are using AI in core business functions,[1] and organizations that leverage AI are starting to report real value and efficiency gains.[2] AI-native startups are dominating the latest cohorts of incubators like YCombinator. Platforms like Stack Overflow are seeing major decreases in traffic as users shift to AI tools.

Source.

But different groups are integrating AI at different rates, and the resulting “uplift” will be uneven. Unsurprisingly, the tech industry is among the groups in the lead.

The US federal government appears to be falling increasingly behind. Slow adoption of new technologies is the norm for governments, and current evidence suggests that AI is no different. For instance:

  • Private-sector job listings are four times more likely to be AI-related than public-sector job listings (and the divide is widening)[3]

  • In surveys, public-sector professionals report using AI less than most Americans[4]

  • And various government officials have expressed the sentiment that adoption is slow

AI adoption is uneven within the government, too. Use of AI is heavily concentrated in the Department of Defense, which accounts for 70-90% of federal AI contracts, despite historically representing a minority of federal IT spending.[5] 

There are some signs that US government integration of AI is ramping up:[6]

  • AI adoption was a priority for the Biden Administration, and the number of “AI use cases” disclosed by civilian agencies more than doubled between 2023 and 2024[7]

  • In January, OpenAI announced “ChatGPT Gov”, a tailored version of ChatGPT that US government agencies can deploy in their existing (Azure) cloud environments, which could help agencies manage security and other requirements[8]

  • The Department of Government Efficiency (DOGE)[9] has already used AI[10] and reportedly planned to develop a chatbot for the U.S General Services Administration (GSA) to boost staff productivity (and may replicate this effort in other agencies or lean into an “AI-first” strategy)

These efforts have had mixed success, and still leave the government behind the private sector.

Moreover, I expect this AI capacity gap to widen over time:

  • Many of the original causes of slow AI adoption will persist, continuously hindering progress
    • The need for well organized and accessible data makes AI even harder for governments to adopt than other technologies (given sensitive information, complex and heterogenous data management policies, and an abundance of non-digitized data). That particular issue may be addressed over time, but many of the other reasons for slow adoption are not new, and will, by default, remain.[11] These include:

      • Scarce technical talent

      • Insufficient funding and convoluted procurement processes

      • Outdated IT infrastructure and interoperability issues, along with high security needs[12]

      • Poorly aligned incentives (staff and agencies are very hesitant to incur the risk of trying something new given low rewards for modernization, cost savings, or otherwise improved performance)

      • Burdensome legal requirements

    • Meanwhile, AI and “tech-ready” companies may pull even further ahead due to persistent advantages (like early access to state-of-the-art AI models) and the adapt-or-die pressure of market forces
  • Compounding effects and accelerating AI progress will widen the AI capability divide between leaders and laggards over time
    • Successfully leveraging AI would translate into profits and accumulated expertise, which in turn make it easier to leverage AI in the future — a compounding effect that we’ve seen before and are seeing already with AI
    • And if AI capabilities progress at a greater-than-linear speed, then lagging by a fixed amount of time will translate to a growing capability gap
  • As AI capabilities improve, the private sector may be able to integrate increasingly advanced AI systems and tools that may be especially difficult to adopt in government settings

It’s possible that this gap won’t widen, especially if adopting AI becomes a top national priority, if there’s a major plateau in AI progress (which could even the playing field), or if the government begins to more directly control AI development. But a growing divide between the government and the private sector seems likely.

A dual imperative for the US federal government

Government AI adoption can’t wait

There are three broad reasons for accelerating the US government’s AI adoption:

  1. Increasing the government’s ability to respond to existential challenges
  2. Maintaining the relevance of democratic institutions
  3. Defusing the time bomb of rushed AI adoption

1. Increasing the government’s ability to respond to existential challenges

Integrating AI could improve business-as-usual government efficiency. A significant fraction of government work is the kind that’s been productively automated or augmented with AI tools in the private sector.[13] If government agencies manage to integrate AI into their work,[14] they would likely become more responsive and efficient. Moreover, sophisticated AI-powered governance tools could address today’s regulatory failures or circumvent difficult tradeoffs.[15] 

Upgrading the US government will become more important given rapid AI progress.

Normal levels of government competence may be insufficient for handling AI-driven challenges, given their technological complexity and the sheer speed at which they may emerge:

  • The government may need new technical tools to verify the safety of advanced AI models, audit AI systems without compromising private information, prevent cyber-attacks, or supervise the activities of AI agents
  • The speed and unpredictability of change will make it even harder to understand what’s happening and what policy responses are appropriate — and impose a higher administrative burden that may leave the government with less capacity to spare for AI-specific issues
  • Without better compliance tools, AI companies and AI systems might start taking increasingly consequential actions without regulators’ understanding or supervision[16]

(Of course, AI adoption isn’t the only thing that determines how effective the US government’s response is. Poor judgement of key decision-makers, polarization, selfish choices, myopic incentives, and other factors matter, too. But AI use will play an increasingly large role.)

If the US government is unable to keep up with society’s adoption of AI, the results could be catastrophic. For instance, we might see:

  • Devastating global pandemics
  • Great power war and global instability
    • Given the inherent destabilizing effects of transformative technologies like AGI, strong international coordination may be needed to prevent large-scale conflict or loss of influence of democratic countries
  • Serious societal issues
    • If not managed carefully, unpredictable interactions between automated systems, dramatic changes in the economy, and other change may break down social and economic institutions
  • Human disempowerment by advanced AI
    • Leaving AI companies to regulate themselves (by failing to pass and enforce sensible laws around AI development or failing to coordinate on AI safety internationally) increases the likelihood that uncontrollable models will be released

Capacity-building for these issues has to happen in advance — we can’t just wait for them to arise. Integrating new technology in government systems takes time. Even if procurement is streamlined, agencies would need to train their staff (and likely hire and vet new staff with appropriate expertise) and update complex and sensitive systems.[17]

2. Maintaining the relevance of democratic institutions

The military is adopting AI in large part because of the technology’s strategic relevance in shaping the international balance of power.[18] Similar dynamics might play out within the US.

For instance, groups that integrate capable AI systems may accumulate massive profits, and gain social and political power through that money — or through AI-assisted lobbying and advocacy.[19] The government will need to keep up.

Moreover, AI and technology companies could leverage their positions as providers of the technology to gain special advantages and evade oversight,[20] influence policy to advance their own agendas,[21] and (in the extreme) even subvert democracy.[22]  Companies like Microsoft may already be enjoying similar dynamics at the expense of US interests.[23] Without experienced staff, the ability to oversee partnerships and evaluate alternative products, or the resources to develop and maintain systems in-house, the government may grow significantly more vulnerable to private influence.

The balance and separation of powers within the US government could also shift. If most government bodies have little AI expertise, groups that leverage the tech (e.g. by partnering with private companies) may leap ahead, broadening their reach. This doesn’t have to involve deliberate acts of subversion; it could happen simply because AI-boosted groups are able to handle more and start taking on greater responsibility. Alternatively, other government actors may unknowingly start relying on AI systems whose behavior has been shaped — perhaps in subtle, hard-to-catch ways — to advance a particular agenda.

3. Defusing the time bomb of rushed automation

Gradual adoption is significantly safer than a rapid scale-up. Agencies would have more time to build up more internal AI expertise, develop proprietary tools, invest in appropriate safeguards, iteratively test automation and AI systems, and use early efficiency gains to boost future capacity for managing AI systems.

Moving slowly today raises the risk of rushed adoption later on. Pressure to automate will probably keep increasing. And in a crisis — e.g. after a conspicuous failure, or a jump in the salience of AI adoption for the administration in power — agencies might cut corners and have less time for security measures, testing, in-house development, etc.

And background risks will increase over time. Frontier AI development will probably concentrate, leaving the government with less bargaining power. Larger technological gaps between private companies and government agencies will worsen the dynamics described above, lowering government ability to oversee private partners. The best AI systems will become more capable (and likely more agentic), making them more dangerous to deploy without robust testing and systems for managing their work. And a broadly more volatile international and economic environment may make failures especially costly. So earlier adoption seems safer.

But the need for speed shouldn’t blind us to the need for security. Steady AI adoption could backfire if it desensitizes government decision-makers to the risks of AI in government, or grows their appetite for automation past what the government can safely handle.

Government adoption of AI will need to manage important risks

Integrating AI in the government carries major risks:

  1. AI adoption could provide an opening for subversion of democratic processes and harm to national interests
    • People with more control over government AI systems or their deployment may be able to influence what they do (possibly locking in certain behaviors in a way that’s very hard to change), use them to gain access to sensitive data (or otherwise weaken data security), or shape how and by whom the systems are used
    • Increasing reliance on AI may weaken meaningful government oversight of AI companies (especially if critical areas are automated, the technology becomes more complex, and the AI market becomes concentrated)
  2. AI systems may be deployed unsafely, leading to catastrophic system failures
    • Even before AI systems pose serious risks of takeover, AI applications might trade reliability for efficiency in areas where this isn’t appropriate
    • For instance, automated processes in military contexts may escalate conflicts
    • Other sources of danger include hard-to-predict interactions between automated systems, new kinds of security vulnerabilities, or “rogue” AI systems that escape human control
  3. Government AI integration could encourage AI race dynamics, or speed up the development of dangerous AI systems
    • Other nations may interpret AI adoption by the US government as a threat, and accelerate their own adoption, potentially triggering an increasingly reckless automation race
    • Increased use of AI by the US government would likely boost investment in AI development, potentially leaving less time for society to prepare for the most dangerous systems

Proper care and preparation can mitigate these risks, but we can’t eliminate them entirely.[24] 

Recommendations

1. Help the US government safely adopt advanced AI

It’s natural to focus on the broad question of whether we should speed up or slow down government AI adoption. But this framing is both oversimplified and impractical — there’s no universal lever that controls the rate of adoption across the federal government.[25]

Perhaps more importantly, taking such a binary stance could lead to poor decisions. Blanket moves to accelerate adoption might override critical safety measures for negligible gains. Conversely, broad restrictions aimed at reducing risk could block valuable and relatively safe use cases.

Instead, we should do what we normally do when juggling different priorities: evaluate the merits and costs of specific interventions, looking for "win-win" opportunities and improvements whose risk-reducing benefits outweigh their adoption-inhibiting costs (and vice versa).

We should focus on "win-win" interventions & key improvements whose benefits outweigh their costs, not take a "safety" or "speed" side.

A) “Win-win” opportunities

Safety measures and government AI adoption don't have to be at odds. Clear policies can increase uptake — especially in risk-averse environments like government. Guidelines that are poorly tailored to reality both stifle integration of new technologies and harm compliance.[26] And reasonable precautions (and investments into transparency) can prevent backlash against the adoption of a technology. Moreover, proposals for improving both safety and speed of AI adoption will generally be the easiest to implement.

Top recommendations:

  1. Streamline AI procurement policies, and remove non-critical regulatory and procedural barriers to rapid AI deployment
    1. Expedite procurement for AI and related technologies (like cloud services), for instance by fast-tracking procurement of vetted systems (and generally harmonizing processes across agencies and use cases and allowing authorization to be ported over from one agency to another),[27] developing standard terms for AI contracts, expanding flexible procurement vehicles like OTAs or TMF, and more[28]

    2. Clarify and streamline policies around deploying AI; clearly delineate between low-risk and high-risk cases (across different use cases, AI systems, and data) to make sure policies are appropriate for the stakes involved (and avoid vague categorizations like “rights impacting”);[29] audit requirements to identify how they might be ill-suited for future advanced AI tools (including agentic systems or increased automation); avoid introducing new policies simply because AI is involved

  2. Invest in technical capacity in the federal government; talent is critical for deploying or building AI tools in-house, acquiring the best AI products without overpaying, and independently testing AI systems
    1. Hire and retain technical talent, including by raising salaries for skilled technical employees to make positions competitive with the private sector, expediting hiring (and security clearance) processes, expanding special hiring authority, recognizing skilled technical staff and providing them with growth opportunities, and strengthening the broader US AI workforce (including by keeping top talent in the US)
    2. Nurture agencies with AI expertise, like National Institute of Standards and Technology (NIST),[30] and resources like the National AI Initiative Office[31]

    3. Boost AI expertise among existing staff, including by incentivizing experimentation with low-risk AI use, investing in training programs that boost AI literacy across government, and developing knowledge-sharing channels
  3. Modernize the government’s digital infrastructure and data management practices
    1. Work towards secure, standardized data infrastructure across agencies, including by fixing data fragmentation and quality issues, ensuring chief data officers have the resources they need, improving data standards, and developing controls managing AI systems’ data access
    2. Improve or replace legacy IT systems (which could significantly cut costs), build cybersecurity capacity, address IT acquisitions issues, invest in secure/confidential computing infrastructure (and broader security improvements), and invest in interoperability of technologies deployed across different agencies — including by exploring use of AI for IT modernization
    3. Plan for advanced AI systems, for instance by developing controlled testing environments and sandboxes or more reliable infrastructure for AI agents[32]

  4. Keep government decision-makers informed on AI progress and ensure advanced AI tools will be available for government use
    1. Track progress in capabilities and trends in adoption of AI, encourage incident-reporting, invest in forecasting AI development
    2. Promote development of technology that makes AI tools usable in government setting, for instance by investing in mechanisms for supporting verifiable claims about AI systems, AI evaluation as a science, privacy tools, and more
    3. Explore legal or other ways to avoid extreme concentration in the frontier AI market (barring exceptional concerns around security)

B) Risk-reducing interventions

If safety-oriented measures are too burdensome, they will simply be dropped by agencies when they do not have the capacity to comply — particularly in crisis situations, when they’re most needed.[33] Clumsy, gratuitous “safeguards” can actively increase the overall risk by leaving less room for other defenses or by giving the appearance of safety when the underlying problems have not actually been solved.[34]

So we should focus on the strongest, highest priority safeguards. The best safeguards will be:

  • as easy to implement as possible
  • scalable (or technology-agnostic), to ensure they remain in place as AI improves
  • and robust to crisis scenarios

Top recommendations:

  1. Make it easier for agencies to comply with safety measures, especially in high-stakes areas
    1. Increase the resources available for safeguards, including by boosting staff capacity, funding, and compute, by giving key safeguards priority status in bureaucratic processes, and by supporting resource-sharing across agencies
    2. Try to automate safeguards as much as possible, e.g. by building continuous verification mechanisms into the systems that get deployed[35]

  2. Build capacity for testing advanced AI models and automated systems that operate in high-stakes areas (and mandate such testing later on)
    1. Invest in in-house testing capacity, including by investing in talent (as discussed above), by ensuring the National Institute of Standards and Technology (NIST) is appropriately funded, by partnering with AI companies (and with third-party groups with AI evaluation expertise) to test earlier models, and by acquiring cloud computing and other resources needed to securely test advanced AI models
  3. Promote work on AI security, to decrease the risk of deploying unsafe models in high-stakes areas
    1. Fund research in control, interpretability, and other key areas related to AI security
    2. Invest in R&D projects for defensive AI tools and other beneficial technology
    3. More generally invest in the field of AI control and safety, including by helping the field coordinate, developing resources for public-sector researchers in the area, and more — possibly even starting a “Manhattan Project” for AI safety
  4. Mitigate the risk of complicating relations between the US federal government and other states or private companies that may arise via deeper government AI adoption
    1. Avoid over-focusing on military uses of AI and push back on perceptions of AI as “military-only” technology, and mutually beneficial use cases of AI in international contexts
    2. Limit the influence AI companies can exert on AI governance, for instance by exploring ways to separate incentives of government procurement and regulation decision-makers
  5. Other
    1. Develop “crisis plans” for catastrophic system failures, for instance by building viable backup systems (which could also involve making sure that qualified staff will be able to step in if automated systems fail)
    2. Establish clear “red lines” between appropriate and inappropriate (or unsafe) deployment of AI in government (including requirements about certain forms of information-sharing, ensure information about inappropriate deployment would be shared (e.g. via whistleblowing or secure incident-reporting mechanisms), and agree on what the responses should be if those lines are crossed

C) Adoption-accelerating interventions

Loudly advocating for increased government use of AI may prompt superficial investment in “AI” tools that are weak or not actually helpful, or encourage decision-makers to blind themselves to security concerns. Instead, we should:

  • Try to accelerate AI adoption in key agencies
    • On the civilian side this includes:
      • DOGE, OMB, and other agencies that coordinate government resources
      • NIST, CISA, and BIS, which are all heavily involved in (international) AI security work
      • National Security Council (for national security decision-making) and National Economic Council (for tracking economic matters)
      • Energy (National Labs), Treasury, the Federal Reserve, and other agencies involved in managing key infrastructure and critical for managing and tracking AI diffusion
      • State, especially for intelligence analyses, negotiations, better enforcement of treaties, etc.
  • And focus on scalable, future-oriented use cases

The best ways to do this — besides the “win-win” opportunities above — might involve:

  1. Informing, planning, and strategic work
    1. Ensure key decision-makers are informed about current and forecasted AI capabilities (and the challenges they may need to deal with)
    2. Audit key agencies’ needs and try to identify potential core use cases, including in crisis scenarios, likely bottlenecks to adoption, and promising next steps
  2. Ensuring that advanced AI tools will be usable in government settings, and developing custom tools for government agencies
    1. Invest in making advanced AI products usable in government settings, including by going through authorization processes, investing in “security by design” and other properties relevant for high-stakes contexts, and mitigating other barriers to adoption
    2. Try to speed up the development of new AI applications for government buyers and contexts
  3. Getting key agencies “AI-ready”
    1. As discussed above, ensure they have necessary resources (funding, talent, and compute), as well as strong digital systems and data management practices
    2. Get them started; work with them to decompose core responsibilities and define success criteria, investing in robustness checks to avoid issues like adversarial specification gaming, help them evaluate or develop and iterate on early AI tools
    3. Ensure that they will have access to frontier AI systems in the future, especially in crises

2. Contingency planning for slow government AI adoption

Besides trying to steer government adoption of AI, we should probably prepare for scenarios where government adoption remains very slow.

A) State collapse: a largely ineffective or vulnerable US government

If the US government never ramps up AI adoption, it may be unable to properly respond to existential challenges. At least in AI safety,[36] it might makes sense to invest more heavily in:

  • Proposals for AI governance that do not rely on US federal government action[37]

    • Build independent third-party AI auditors and evaluators that can substitute for the government (e.g. by being authorized to operate on its behalf or operate entirely without US government support)

    • Explore private AI governance, or what more AI-ready state governments can do

    • Create AI governance institutions in countries like the Netherlands or the UK, or multilateral governance institutions

  • Technical rather than policy-based interventions for preparing for looming challenges
    • Map out scenarios in which AI safety regulation is ineffective and explore potential strategies, e.g. by trying to solve resulting cooperation problems
    • Work directly with AI companies to develop and implement internal safety protocols and governance systems, and help companies coordinate on those priorities
    • Generally work on technical solutions (e.g. paying the “alignment tax” or differentially promoting defensive technologies)

B) Rushed AI adoption

Hasty integration of AI in the US government would go better if we prepared for it in advance (even if that preparation happens outside the federal government).

Many of the interventions that could help to avoid this situation would also help with de-risking rushed automation. Besides those, it could help to:

  • Build emergency AI capacity outside of the government
    • Coordinate with AI experts to create “standby” response teams that can be quickly seconded into government roles (e.g. via the Intergovernmental Personnel Act)
    • Develop standardized protocols for rapid but as-safe-as-possible AI integration (including toolkits on improving infrastructure)
    • Invest in compute and energy resources
  • Develop tools that the US federal government would be able to rapidly deploy
    • Create monitoring and testing systems that satisfy the needs of relevant agencies and can be deployed very quickly
    • Try to build provably secure or guaranteed-safe tools for government officials, and custom tools for specific high-priority use cases (e.g. chip verification mechanisms)
  • Other
    • Develop training programs for government officials
    • Analyze potential high-stakes failure modes
    • Continuously test state-of-the-art AI tools and systems — and generally try to vet AI companies — to help inform (rushed) decision-makers in procurement processes

Conclusion

AI is a growing force. In the near future, it’s likely to massively accelerate the pace of change and trigger existentially relevant challenges that the US government will need to respond to. I’m worried that the US government’s adoption of AI isn’t on track to keep up with its crucial role.

Improving government adoption seems like a neglected lever for reducing existential risks.

Acknowledgements

I'm very grateful to Nikhil Mulani, Max Dalton, Rose Hadshar, Owen Cotton-Barratt, Fin Moorhouse, and others for conversations and comments on earlier drafts. 

  1. ^

     Surveys suggest that AI is rapidly diffusing into workplaces; in 2023-2024, the annualized growth rate in uptake was around 70-140%.

  2. ^

     Field tests show significant boosts to performance. For instance, a recent experiment showed that R&D professionals working with AI “teammates” performed as well as two-person teams that didn’t use AI. (The size of the boost seems to vary by type of task, employee skills, and more.)

    Real-world data is limited, but some studies that attempt to measure productivity gains have been compiled by Epoch.

  3. ^

     An analysis of job listings shows that from 2017 to 2023, the percent of all job postings represented by AI jobs grew from 0.5% to 2% in the private sector but remained fairly flat in the public sector, at around 0.25%. (The authors suggest that the difference in pay might be an important factor in the public sector’s inability to attract and retain AI talent; average posted salaries are around 50% higher in the private sector.)

  4. ^

     A late-2024 AWS survey of “public-sector IT decision-makers” (839 respondents across the federal, education, nonprofit and healthcare sector) found that only 12% reported that their company has already adopted generative AI. (Apparently 30% expect this to happen within the next 2 years. Two thirds have found it difficult for their organization to adopt generative AI; current and future barriers that were cited include lack of clarity on how it might be useful, concern about the cost of integrating AI with legacy systems, concern about public trust, and concern about data security and privacy.) The general US public, meanwhile, appears to use AI for work fairly frequently; a NBER report found that in a nationally representative US survey, over 28% of adults use generative AI for work. And among “knowledge workers” 52% reported using AI weekly.

    Comparing results from different surveys is difficult, but other data-points seem to confirm the general trend. See e.g. another late-2024 survey of public-sector AI use from the Hoover Institution, and a review of general-public surveys here.

  5. ^

     A report from Brookings found that over the last few years, most of the growth in the number (and total value) of federal AI-related contracts was concentrated in the Department of Defense (DOD). By August 2023, federal agencies had committed to a total of $675 million in contracts (which might be paid out over the course of several years), of which 82% is from DOD contracts — leaving $118 million for all other agencies. This trend was even more extreme among new  contracts; in FY2023, 88-96% of federal AI contracts were due to the DOD.

    It’s not just contracts. In surveys, DOD staff report more use of AI. And a Stanford white paper noted that while agencies requested an average of $270K to support each AI office in their 2025 congressional budget justifications, the DOD proposed a budget of $435 M. 

    Meanwhile, the DOD has historically accounted for less than half of federal IT spending — generally between 40% - 50%, although data wasn’t provided in recent years. (The DOD tends to dominate federal contract spending, driven primarily by weapons systems.)

  6. ^

     Adoption is increasing outside the US too. In the UK, Anthropic is partnering with the government to explore how Claude could enhance public services. The UK government has also announced an AI Opportunities Action Plan, which, among other things, proposes building a UK AI cluster.

  7. ^

     It’s worth keeping in mind that some of these use cases are just being tested — and many are not “sophisticated.” When canvassing agency use of AI in 2020, Stanford computer scientists evaluated the techniques deployed in each use case. For many, there was insufficient information provided. Of the rest, only around 30% were rated as “high in sophistication.” (“To illustrate the scale used, we considered: (a) logistic regression using structured data to be of lower sophistication; (b) a random forest with attention to hyperparameter tuning to be of medium sophistication; and (c) use of deep learning to develop “concept questioning” of the patent examination manual to be of higher sophistication.”)

  8. ^

     OpenAI is also working toward higher levels of FedRAMP accreditation for ChatGPT Enterprise, but that process tends to take a long time. It’s also worth noting, however, that OpenAI is closely partnered with Microsoft, which might streamline integration given Microsoft’s dominance in US government IT.

  9. ^

     Some have suggested that DOGE could be a good opportunity for boosting the US government’s technical and AI capacity. If executed poorly, though, DOGE could have the opposite effect — e.g. by reducing already limited critical AI expertise in the federal government (see also commentary from Alex Stapp) or by cutting the funding agencies have for AI adoption

  10. ^

     Note that there are a number of concerns (related to data security and other issues) about how DOGE’s use of AI

  11. ^

     In a review of the AI “Compliance Plans” agencies had published (as of October 2024), authors found five common themes across different agencies’ discussions of the barriers they were encountering for AI innovation and internal governance. These were:

    Funding (especially for “AI governance”)

    Shortage of AI talent and expertise

    Challenges with access to computing infrastructure

    “Difficulties in accessing and validating data sources for AI models, alongside data privacy and security concerns”

    “Regulatory ambiguity”

  12. ^

     As one concrete example, the GSA reportedly recently sought to deploy the popular Cursor code editor, but pivoted to a different assistant because Cursor isn’t planning to achieve FedRAMP authorization (a lengthy and fairly costly process) in the near future.

  13. ^

     A report on “An Efficiency Agenda for the Executive Branch” includes relevant discussion: “An Accenture analysis estimated that 39 percent of working hours for the public service sector have a ‘higher potential for automation’ or ‘augmentation.’ [...] [A study on jobs more exposed to AI] found 86 ‘fully exposed’ job categories in total, largely within the realm of administrative and knowledge labor. These occupational categories have substantial overlap with the jobs and task sets commonly seen within the federal workforce, suggesting that the federal bureaucracy is itself highly exposed to AI-enabled labor savings.”

    For more on which areas might be automated earlier, see “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality

  14. ^

     See more on specific use cases in Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies, which distinguishes between policy setting/research/monitoring, enforcement, delivering services, and internal management. As a concrete example, the Treasury Department is already using AI to monitor and enforce corporate compliance.

  15. ^

     For instance, structured transparency technologies could facilitate measures that today would require impractical violations of privacy (e.g. monitoring), or conversely reduce the privacy costs of current security measures.

  16. ^

     Analogous things have arguably happened before; the financial crisis of 2008 is at least partly the result of increasing complexity in the financial sector, without a sufficient increase in monitoring and understanding from regulators.

  17. ^

     Recently DOGE wanted to build and use a tool that would help it navigate the GSA’s main portal. Wired reported that, while given today’s technology this might seem like a simple project to people unfamiliar with the GSA’s systems, it’s more realistic to think of it as a multi-year endeavor: “every database would need to be mapped, its columns and metadata described and categorized, ensuring the system understood what data lived where. None of this would happen automatically. It would be a manual, painstaking process.” (A simpler version of this tool, the GSAi, began development during the Biden administration; it was announced in March 2025.)

  18. ^

     See for instance the DOD’s “AI Adoption Strategy,” which stresses goals like “ensure U.S. warfighters maintain decision superiority on the battlefield for years to come” and “competitive advantage in fielding the emerging technology.”

  19. ^

     For instance, it seems that Standard Oil used its massive economic and political leverage to resist regulation for many years.

  20. ^

     For instance, the government may be unable to verify AI companies’ claims about their testing practices or the safety of their AI models.

  21. ^
  22. ^

     See a forthcoming paper from Tom Davidson, Lukas Finnveden and Rose Hadshar: “AI-Enabled Coups: how a small group could use AI to seize power”

  23. ^

     Microsoft holds around 85% of the share in US government office productivity software, and appears uniquely insulated from government accountability. (Network effects make this very difficult to change.)

    It appears that the US government’s interests have already been harmed by this; Microsoft’s systems have been repeatedly hacked by the Chinese government and other actors, jeopardizing the security of sensitive data and systems across dozens of agencies. (It’s not clear if Microsoft’s approach to security is improving.)

  24. ^

     Note: if we did manage to eliminate the risks from poorly implemented AI adoption, automation could enable persistent and very harmful government decisions. This is out of scope here.

  25. ^

     Still, I overall expect that near-future AI adoption will be slower than the “ideal” pace that would minimize the total risks involved.

  26. ^

     This phenomenon is pretty widespread, and we’re already seeing it with AI uses. “Shadow AI use” — employee use of AI without managers’ knowledge — is emerging in the private sector at least. And regulatory ambiguity is already cited as a barrier for compliance with OMB directives. This response to the OMB also discusses related issues.

  27. ^

     See:

    NAIAC’s recent recommendation: establishing an AI model evaluation, testing, and assessment framework to help address the issue of “considerable variation in federal agency capacity to integrate modern AI systems without significant upgrades or overhauls.”

    Google’s response to the OSTP’s AI Action Plan request, which emphasizes the need to allow authorization to be ported from one agency to another

    GAO’s 2024 analysis of federal software licenses, which found a number of issues with tracking the usage and inventory

  28. ^

     Other promising directions here have been discussed in various responses to the OSTP’s AI Action Plan Request. These include relevant recommendations from Anthropic:

    Tasking the OMB to “rapidly address resource constraints, procurement limitations, and programmatic obstacles to federal AI adoption, incorporating provisions for substantial AI acquisitions in the President’s Budget”

    Leveraging “existing frameworks to enhance federal procurement for national security purposes, particularly the directives in the October 2024 National Security Memorandum (NSM) on Artificial Intelligence and the accompanying Framework to Advance AI Governance and Risk Management in National Security”

    Creating “a joint working group between the Department of Defense and the Office of the Director of National Intelligence to develop recommendations for the Federal Acquisition Regulatory Council (FARC) on accelerating procurement processes for AI systems while maintaining rigorous security and reliability standards. The FARC should then consider appropriate amendments to the Federal Acquisition Regulation based on these recommendations to create a procurement environment that balances innovation with responsible governance.”

  29. ^

     See for instance concerns about use of “rights impacting” and “safety impacting” in OMB’s guidance

  30. ^

     which houses AISI, the AI Risk Management Framework, and more

  31. ^

     See a bit more about resourcing the NAIIO in NAIAC’s recent recommendation to President Trump

  32. ^

     Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies discusses sandboxes in Part III.

  33. ^

     We’re already seeing this with government AI adoption. Reviews of compliance with OMB directives and AI-oriented executive orders have found very uneven results.

    The capacity of internal supervisory bodies that enforce compliance with safety-oriented policies should also be taken into account. See for instance the “emerging crisis in mass adjudication” discussed here.

  34. ^

     A paper that analyzed policies requiring human oversight of government algorithms found that staff were often unable to provide the required oversight. It argues that as a result, “human oversight policies legitimize government uses of faulty and controversial algorithms without addressing the fundamental issues with these tools.”

  35. ^

     For more in this vein, see a discussion of automated monitoring and accountability in “Government by Algorithm” (which in turn references this paper on a technological toolkit for “Accountable Algorithms”)

  36. ^

     We should also explore how we can better prepare for other existential challenges we might be facing, like risks from engineered bioweapons.

  37. ^

     It might also make sense to explore whether other government responsibilities can be done outside government (either now or in crisis situations).

39

0
1

Reactions

0
1

More posts like this

Comments18
Sorted by Click to highlight new comments since:

Thank you for this article. I've read some of the stuff you wrote in your capacity at CEA, which I quite enjoyed, your comments on slow vs. quick mistakes changed my thinking. This is the first thing I've read since you started at Forethought. I have some comments, which are mostly critical, I tried using ChatGPT and Claude to make my comment more even-handed but they did a bad job so you're stuck with reading my overly critical writing. Some of my criticism may be misguided due to me not having a good understanding of the motivation behind writing the article so it might help me if you explained more about the motivation. Of course you're not obligated to explain anything to me or to respond at all, I'm just writing this because I think it's generally useful to share criticisms.

I think this article would benefit from a more thorough discussion of the downside risks of its proposed changes—off the top of my head:

  • Increasing government dependency on AI systems could make policy-makers more reluctant to place restrictions on AI development because they would be hurting themselves by doing so. This is a very bad incentive.
    • The report specifically addresses how the fact that Microsoft Office is so embedded in government means the company can get away with bad practices, but seemingly doesn't connect this to how AI companies might end up in the same position.
  • Government contracts to buy LLM services increases AI company revenue, which shortens timelines.
  • The government does not always work in the interests of the people (in fact it frequently works against them!) so making the government more effective/powerful is not pure upside.

The article does mention some downsides, but with no discussion of tradeoffs, and it says we should focus on "win-wins" but doesn't actually say how we can avoid the downsides (or, if it did, I didn't get that out of the article).

To me the article reads like you decided the conclusion and then wrote a series of justifications. It is not clear to me how you arrived at the belief that the government needs to start using AI more, and it's not clear to me whether that's true.

For what it's worth, I don't think government competence is what's holding us back from having good AI regulations, it's government willingness. I don't see how integrating AI into government workflow will improve AI safety regulations (which is ultimately the point, right?[^1]), and my guess is on balance it would make AI regulations less likely to happen because policy-makers will become more attached to their AI systems and won't want to restrict them.

I also found it odd that the report did not talk about extinction risk. In its list of potential catastrophic outcomes, the final item on the list was "Human disempowerment by advanced AI", which IMO is an overly euphemistic way of saying "AI will kill everyone".

By my reading, this article is meant to be the sort of Very Serious Report That Serious People Take Seriously, which is why it avoids talking about x-risk. I think that:

  1. you won't get people to care about extinction risks by pretending they don't exist;
  2. the market is already saturated with AI safety people writing Very Serious Reports in which they pretend that human extinction isn't a serious concern;
  3. AI x-risk is mainstream enough at this point that we can probably stop pretending not to care about it.

There are some recommendations in this article that I like, and if I think it should focus much more on them:

investing in cyber security, pre-deployment testing of AI in high-stakes areas, and advancing research on mitigating the risks of advanced AI

Without better compliance tools, AI companies and AI systems might start taking increasingly consequential actions without regulators’ understanding or supervision

[Without oversight], the government may be unable to verify AI companies’ claims about their testing practices or the safety of their AI models.

Steady AI adoption could backfire if it desensitizes government decision-makers to the risks of AI in government, or grows their appetite for automation past what the government can safely handle.

I also liked the section "Government adoption of AI will need to manage important risks" and I think it should have been emphasized more instead of buried in the middle.

Some line item responses

I don't really know how to organize this so I'm just going to write a list of lines that stood out to me.

invest in AI and technical talent

What does that mean exactly? I can't think of how you could do that without shortening timelines so I don't know what you have in mind here.

Streamline procurement processes for AI products and related tech

I also don't understand this. Procurement by whom, for what purpose? And again, how does this not shorten timelines? (Broadly speaking, more widespread use of AI shortens timelines at least a little bit by increasing demand.)

Gradual adoption is significantly safer than a rapid scale-up.

This sounds plausible but I am not convinced that it's true, and the article presents no evidence, only speculation. I would like to see more rigorous arguments for and against this position instead of taking it for granted.

And in a crisis — e.g. after a conspicuous failure, or a jump in the salience of AI adoption for the administration in power — agencies might cut corners and have less time for security measures, testing, in-house development, etc.

This line seems confused. Why would a conspicuous failure make government agencies want to suddenly start using the AI system that just conspicuously failed? Seems like this line is more talking about regulating AI than adopting AI, whereas the rest of the article is talking about adopting AI.

Frontier AI development will probably concentrate, leaving the government with less bargaining power.

I don't think that's how that works. Government gets to make laws. Frontier AI companies don't get to make laws. This is only true if you're talking about an AI company that controls an AI so powerful that it can overthrow the government, and if that's what you're talking about then I believe that would require thinking about things in a very different way than how this article presents them.

And: would adopting AI (i.e. paying frontier companies so government employees can use their products) reduce the concentration of power? Wouldn't it do the opposite?

It’s natural to focus on the broad question of whether we should speed up or slow down government AI adoption. But this framing is both oversimplified and impractical

Up to this point, the article was primarily talking about how we should speed up government AI adoption. But now it's saying that's not a good framing? So why did the article use that framing? I get the sense that you didn't intend to use that framing, but it comes across as if you're using it.

Hire and retain technical talent, including by raising salaries

I would like to see more justification for why this is a good idea. The obvious upside is that people who better understand AI can write more useful regulations. On the other hand, empirically, it seems that people with more technical expertise (like ML engineers) are on average less in favor of regulations and more in favor of accelerating AI development (shortening timelines, although they usually don't think "timelines" are a thing). So arguably we should have fewer such people in positions of government power. I can see the argument either way, I'm not saying you're wrong, I'm just saying you can't take your position as a given.

And like I said before, I think by far the bigger bottleneck to useful AI regulations is willingness, not expertise.

Explore legal or other ways to avoid extreme concentration in the frontier AI market

(this isn't a disagreement, just a comment:)

You don't say anything about how to do that but it seems to me the obvious answer is antitrust law.

(this is a disagreement:)

The linked article attached to this quote says "It’s very unclear whether centralizing would be good or bad", but you're citing it as if it definitively finds centralization to be bad.

If the US government never ramps up AI adoption, it may be unable to properly respond to existential challenges.

What does AI adoption have to do with the ability to respond to existential challenges? It seems to me that once AI is powerful enough to pose an existential threat, then it doesn't really matter whether the US government is using AI internally.

Map out scenarios in which AI safety regulation is ineffective and explore potential strategies

I don't think any mapping is necessary. Right now AI safety regulation is ineffective in every scenario, because there are no AI safety regulations (by safety I mean notkilleveryoneism). Trivially, regulations that don't exist are ineffective. Which is one reason why IMO the emphasis of this article is somewhat missing the mark—right now the priority should be to get any sort of safety regulations at all.

Build emergency AI capacity outside of the government

I am moderately bullish on this idea (I've spoken favorably about Sentinel before) although I don't actually have a good sense of when it would be useful. I'd like to see more projection of under exactly what sort of scenarios "emergency capacity" would be able to prevent catastrophes. Not that that's within the scope of this article, I just wanted to mention it.

[^1] Making government more effective in general doesn't seem to me to qualify as an EA cause area, although perhaps a case could be made. The thing that matters on EA grounds (with respect to AI) is making the government specifically more effective at, or more inclined to, regulate the development of powerful AI.

 Thanks for this comment! I don’t view it as “overly critical.”

Quickly responding (just my POV, not Forethought’s!) to some of what you brought up ---

(This ended up very long, sorry! TLDR: I agree with some of what you wrote, disagree with some of the other stuff / think maybe we're talking past each other. No need to respond to everything here!)

A. Motivation behind writing the piece / target audience/ vibe / etc.

Re:

…it might help me if you explained more about the motivation [behind writing the article] [...] the article reads like you decided the conclusion and then wrote a series of justifications

 I’m personally glad I posted this piece, but not very satisfied with it for a bunch of reasons, one of which is that I don’t think I ever really figured out what the scope/target audience should be (who I was writing for/what the piece was trying to do). 

So I agree it might help to quickly write out the rough ~history of the piece:

  • I’d started looking into stuff related to “differential AI development” (DAID), and generally exploring how the timing of different [AI things] relative to each other could matter.
  • My main focus quickly became exploring ~safety-increasing AI applications/tools — Owen and I recently posted about this (see the link).
  • But I also kept coming back to a frame of “oh crap, who is using AI how much/how significantly is gonna matter an increasing amount as time goes on. I expect adoption will be quite uneven — e.g. AI companies will be leading the way — and some groups (whose actions/ability to make reasonable decisions we care about a lot) will be left behind.”
    • At the time I was thinking about this in terms of “differential AI development and diffusion
  • IIRC I soon started thinking about governments here; I had the sense that government decision-makers were generally slow on tech use, and I was also using “which types of AI applications will not be properly incentivized by the market” as a way to think about which AI applications might be easier to speed up. (I think we mentioned this here.)
  • This ended up taking me on a mini deep dive on government adoption of AI, which in turn increasingly left me with the impression that (e.g.) the US federal government would either (1) become increasingly overtaken from within by an unusually AI-capable group (or e.g. the DOD), (2) be rendered increasingly irrelevant, leaving (US) AI companies to regulate themselves and likely worsening its ability to deal with other issues, or (3) somehow in fact adopt AI, but likely in a chaotic way that would be especially dangerous (because things would go slowly until a crisis forced a ~war-like undertaking).
  • I ended up poking around in this for a while, mostly as an aside to my main DAID work, feeling like I should probably scope this out and move on. (The ~original DAID memos I’d shared with people discussed government AI adoption.)
  • After a couple of rounds of drafts+feedback I got into a “I should really publish some version of this that I believe and seems useful and then get back to other stuff; I don’t think I’m the right person to work a lot more on this but I’m hoping other people in the space will pick up whatever is correct here and push it forward” mode - and ended up sharing this piece. 

In particular I don’t expect (and wasn’t expecting) that ~policymakers will read this, but hope it’s useful for people at relevant think tanks or similar who have more government experience/knowledge but might not be paying attention to one “side” of this issue or the other. (For instance, I think a decent fraction of people worried about existential risks from advanced AI don’t really think about how using AI might be important for navigating those risks, partly because all of AI kinda gets lumped together).

Quick responses to some other things in your comment that seem kinda related to what I'm responding to in this “motivation/vibe/…” cluster:

 I also found it odd that the report did not talk about extinction risk. In its list of potential catastrophic outcomes, the final item on the list was "Human disempowerment by advanced AI", which IMO is an overly euphemistic way of saying "AI will kill everyone".

We might have notably different worldviews here (to be clear mine is pretty fuzzy!). For one thing, in my view many of the scary “AI disempowerment” outcomes might not in fact look immediately like “AI kills everyone” (although to be clear that is in fact an outcome I’m very worried about), and unpacking what I mean by "disempowerment" in the piece (or trying to find the ideal way to say it) didn't seem productive -- IIRC I wrote something and moved on. I also want to be clear that rogue AI [disempowering] humans is not the only danger I’m worried about, i.e. it doesn’t dominate everything else for me -- the list you're quoting from wasn't an attempt to mask AI takeover, but rather a sketch of the kind of thing I'm thinking about. (Note: I do remember moving that item down the list at some point when I was working on a draft, but IIRC this was because I wanted to start with something narrower to communicate the main point, not because I wanted to de-emphasize ~AI takeover.)

 

By my reading, this article is meant to be the sort of Very Serious Report That Serious People Take Seriously, which is why it avoids talking about x-risk.

I might be failing to notice my bias, but I basically disagree here --- although I do feel a different version of what you're maybe pointing to here (see next para). I was expecting that basically anyone who reads the piece will already have engaged at least a bit with "AI might kill all humans", and likely most of the relevant audience will have thought very deeply about this and in fact has this as a major concern. I also don't personally feel shy about saying that I think this might happen — although again I definitely don't want to imply that I think this is overwhelmingly likely to happen or the only thing that matters, because that's just not what I believe.

However I did occasionally feel like I was ~LARPing research writing when I was trying to articulate my thoughts, and suspect some of that never got resolved! (And I think I floundered a bit on where to go with the piece when getting contradicting feedback from different people - although ultimately the feedback was very useful.) In my view this mostly shows up in other ways, though. (Related - I really appreciated Joe Carlsmith's recent post on fake thinking and real thinking when trying to untangle myself here.)


 

B. Downside risks of the proposed changes

  1. Making policymakers “more reluctant to place restrictions on AI development...”
    1. I did try to discuss this a bit in the "Government adoption of AI will need to manage important risks" section (and sort of in the "3. Defusing the time bomb of rushed automation" section), and indeed it's a thing I'm worried about.
    2. I think ultimately my view is that without use of AI in government settings, stuff like AI governance will just be ineffective or fall to private actors anyway, and also that the willingness-to-regulate /undue influence dynamics will be much worse if the government has no in-house capacity or is working with only one AI company as a provider.
  2. Shortening timelines by increasing AI company revenue
    1. I think this isn't a major factor here - the govt is a big customer in some areas, but the private sector dominates (as does investment in the form of grants, IIRC)
  3. "The government does not always work in the interests of the people (in fact it frequently works against them!) so making the government more effective/powerful is not pure upside."
     
    1. I agree with this, and somewhat worry about it. IIRC I have a footnote on this somewhere -—I decided to scope this out. Ultimately my view right now is that the alternative (~no governance at least in the US, etc.) is worse. Sort of relatedly, I find the "narrow corridor" a useful frame here -- see e.g. here.)

C. Is gov competence actually a bottleneck?

 I don't think government competence is what's holding us back from having good AI regulations, it's government willingness. I don't see how integrating AI into government workflow will improve AI safety regulations (which is ultimately the point, right?[^1]), and my guess is on balance it would make AI regulations less likely to happen because policy-makers will become more attached to their AI systems and won't want to restrict them.

My view is that you need both, we're not on track for competence, and we should be pretty uncertain about what happens on the willingness side.

D. Michael’s line item responses

1. 

> invest in AI and technical talent

What does that mean exactly? I can't think of how you could do that without shortening timelines so I don't know what you have in mind here.

I’m realizing this can be read as “invest in AI and in technical talent” — I meant “invest in AI talent and (broader) technical talent (in govt).” I’m not sure if this just addresses the comment; my guess is that doing this might have a tiny shortening effect on timelines (but is somewhat unclear, partly because in some cases e.g. raising salaries for AI roles in govt might draw people away from frontier AI companies), but this is unlikely to be the decisive factor. (Maybe related: my view is that generally this kind of thing should be weighed instead of treated as a reason to entirely discard certain kinds of interventions.)

2. 

> Streamline procurement processes for AI products and related tech

I also don't understand this. Procurement by whom, for what purpose? And again, how does this not shorten timelines? (Broadly speaking, more widespread use of AI shortens timelines at least a little bit by increasing demand.)

I was specifically talking about agencies’ procurement of AI products — e.g. say the DOE wants a system that makes forecasting demand easier or whatever; making it easier for them to actually get such a system faster. I think the effect on timelines will likely be fairly small here (but am not sure), and currently think it would be outweighed by the benefits.

3. 

> Gradual adoption is significantly safer than a rapid scale-up.

This sounds plausible but I am not convinced that it's true, and the article presents no evidence, only speculation. I would like to see more rigorous arguments for and against this position instead of taking it for granted.

I’d be excited to see more analysis on this, but it’s one of the points I personally am more confident about (and I will probably not dive in right now). 

4. 

> And in a crisis — e.g. after a conspicuous failure, or a jump in the salience of AI adoption for the administration in power — agencies might cut corners and have less time for security measures, testing, in-house development, etc.

This line seems confused. Why would a conspicuous failure make government agencies want to suddenly start using the AI system that just conspicuously failed? Seems like this line is more talking about regulating AI than adopting AI, whereas the rest of the article is talking about adopting AI.

Sorry, again my writing here was probably unclear; the scenarios I was picturing were more like: 

  • There’s a serious breach - US govt systems get hacked (again) by [foreign nation, maybe using AI] - revealing that they’re even weaker than is currently understood, or publicly embarrassing the admin. The admin pushes for fast modernization on this front.
  • A flashy project isn’t proceeding as desired (especially as things are ramping up), the admin in power is ~upset with the lack of progress, pushes
  • There’s a successful violent attack (e.g. terrorism); turns out [agency] was acting too slowly...
  • Etc.

Not sure if that answers the question/confusion?

5. 

> Frontier AI development will probably concentrate, leaving the government with less bargaining power.

I don't think that's how that works. Government gets to make laws. Frontier AI companies don't get to make laws. This is only true if you're talking about an AI company that controls an AI so powerful that it can overthrow the government, and if that's what you're talking about then I believe that would require thinking about things in a very different way than how this article presents them.

This section is trying to argue that AI adoption will be riskier later on, so the “bargaining power” I was talking about here is the bargaining power of the US federal govt (or of federal agencies) as a customer; the companies it’s buying from will have more leverage if they’re effectively monopolies. My understanding is that there are already situations where the US govt has limited negotiation power and maybe even makes policy concessions to specific companies specifically because of its relationship to those companies — e.g. in defense (Lockheed Martin, etc., although this is also kinda complicated) and again maybe Microsoft.

And: would adopting AI (i.e. paying frontier companies so government employees can use their products) reduce the concentration of power? Wouldn't it do the opposite?

Again, the section was specifically trying to argue that later adoption is scarier than earlier adoption (in this case because there are (still) several frontier AI companies). But I do think that building up internal AI capacity, especially talent, would reduce the leverage any specific AI company has over the US federal government. 

6. 

> It’s natural to focus on the broad question of whether we should speed up or slow down government AI adoption. But this framing is both oversimplified and impractical

Up to this point, the article was primarily talking about how we should speed up government AI adoption. But now it's saying that's not a good framing? So why did the article use that framing? I get the sense that you didn't intend to use that framing, but it comes across as if you're using it.

Yeah, I don't think I navigated this well! (And I think I was partly talking ti myself here.) But  maybe my “motivation” notes above give some context? 
In terms of the specific “position” I in practice leaned into: Part of why I led with the benefits of AI adoption was the sense that the ~existential risk community (which is most of my audience) generally focuses on risks of AI adoption/use/products, and that's where my view diverges more. There's also been more discussion, from an existential risk POV, of the risks of adoption than there has been of the benefits, so I didn't feel that elaborating too much on the risks would be as useful.

7. 

> Hire and retain technical talent, including by raising salaries

I would like to see more justification for why this is a good idea. The obvious upside is that people who better understand AI can write more useful regulations. On the other hand, empirically, it seems that people with more technical expertise (like ML engineers) are on average less in favor of regulations and more in favor of accelerating AI development (shortening timelines, although they usually don't think "timelines" are a thing). So arguably we should have fewer such people in positions of government power.

The TLDR of my view here is something like "without more internal AI/technical talent (most of) the government will be slower on using AI to improve its work & stay relevant, which I think is bad, and also it will be increasingly reliant on external people/groups/capacity for technical expertise --- e.g. relying on external evals, or trusting external advice on what policy options make sense, etc. and this is bad."

8. 

> Explore legal or other ways to avoid extreme concentration in the frontier AI market

[...]

The linked article attached to this quote says "It’s very unclear whether centralizing would be good or bad", but you're citing it as if it definitively finds centralization to be bad.

(The linked article is this one: https://www.forethought.org/research/should-there-be-just-one-western-agi-project )

I was linking to this to point to relevant discussion, not as a justification for a strong claim like “centralization is definitively bad” - sorry for being unclear!

9. 

> If the US government never ramps up AI adoption, it may be unable to properly respond to existential challenges.

What does AI adoption have to do with the ability to respond to existential challenges? It seems to me that once AI is powerful enough to pose an existential threat, then it doesn't really matter whether the US government is using AI internally.

I suspect we may have fairly different underlying worldviews here, but maybe a core underlying belief on my end is that there are things that it's helpful for the government to do before we get to ~ASI, and also there will be AI tools pre ~ASI that are very helpful for doing those things. (Or an alt framing: the world will get ~/fast/complicated/weird due to AI before there’s nothing the US gov could in theory do to make things go better.)

10. 

> Map out scenarios in which AI safety regulation is ineffective and explore potential strategies

I don't think any mapping is necessary. Right now AI safety regulation is ineffective in every scenario, because there are no AI safety regulations (by safety I mean notkilleveryoneism). Trivially, regulations that don't exist are ineffective. Which is one reason why IMO the emphasis of this article is somewhat missing the mark—right now the priority should be to get any sort of safety regulations at all.

I fairly strongly disagree here (with "the priority should be to get any sort of safety regulations at all") but don't have time to get into it, really sorry!

---
 

Finally, thanks a bunch for saying that you enjoyed some of my earlier writing & I changed your thinking on slow vs quick mistakes! That kind of thing is always lovely to hear.

(Posted on my phone— sorry for typos and similar!)

Thanks, this comment gives me a much better sense of where you're coming from. I agree and disagree with various specific points, but I won't get into that since I don't think we will resolve any disagreements without an extended discussion.

What I will say is that I found this comment to be much more enlightening than your original post. And whereas I said before that the original article didn't feel like the output of a reasoning process, this comment did feel like that. At least for me personally, I think whatever mental process you used to write this comment is what you should use to write these sorts of articles, because whatever process you used to write this comment, it worked.

I don't know what's going on inside your head, but if I were to guess, perhaps you didn't want to write an article in the style of this comment because it's too informal or personal or un-authoritative. Those qualities do make it harder to (say) get a paper published in an academic journal, but I prefer to read articles that have those qualities. If your audience is the EA Forum or similar, then I think you should lean into them.

However I did occasionally feel like I was ~LARPing research writing when I was trying to articulate my thoughts, and suspect some of that never got resolved!

I don't think you were LARPing research, your comment shows a real thought process behind it. After reading all your line item responses, I feel like I understand what you were trying to say. Like on the few parts I quoted as seeming contradictory, I can now see why they weren't actually contradictory and they were part of a coherent stance.

I think you (Michael Dickens) are probably one of my favorite authors on your side of this, and I'm happy to see this discussion - though I myself am more on the other side.

Some quick responses
> I don't think government competence is what's holding us back from having good AI regulations, it's government willingness.

I assume it can clearly be a mix of both. Right now we're in a situation where many people barely trust the US government to do anything. A major argument for why the US government shouldn't regulate AI is that they often mess up things they try to regulate. This is a massive deal in a lot of the back-and-forth I've seen on the issue on Twitter.

I'd expect that if the US government were far more competent, people would trust it to take care of many more things, including high-touch AI oversight. 

> Increasing government dependency on AI systems could make policy-makers more reluctant to place restrictions on AI development because they would be hurting themselves by doing so. This is a very bad incentive.

This doesn't seem like a major deal to me. Like, the US government uses software a lot, but I don't see them "funding/helping software development", even though I really think they should. If I were them, I would have invested far more in open-source systems, for instance.

My quick impression is that a competent oversight and guiding of AI systems, carefully working through the risks and benefits, would be incredibly challenging, and I'd expect any human-lead government to make gigantic errors in it. Even attempts to "slow down AI" could easily backfire if not done well. For example, I think that Democratic attempts to increase migration in the last few years might have massively backfired. 

I agree with a good portion of your comment but I still don't think increasing government competence (on AI) is worth prioritizing:

  • SB-1047 was adequately competently written (AFAICT). If we get more regulations at a similar level of competence, that would be reasonable.
  • Good AI regulations will make things harder on AI companies. AI leaders / tech accelerationists will be unhappy about regulations regardless of how competently written they are. On the other hand, the general population mostly supports AI regulations (according to AIPI polls). Getting regulators on board with what people want seems to me to be the best path to getting regulations in place.

Like, the US government uses software a lot, but I don't see them "funding/helping software development"

Suppose it turned out Microsoft Office was dangerous. Surely the fact that Office is so embedded in government procedures would make it less likely to get banned?

IIRC you see similar phenomena (although I can't recall any examples off hand) where some government-mandated software has massive security flaws but nobody does anything about it because the software is too entrenched.

Thanks for the responses!

SB-1047 was adequately competently written (AFAICT). If we get more regulations at a similar level of competence, that would be reasonable.

Agreed

Getting regulators on board with what people want seems to me to be the best path to getting regulations in place.

I don't see it as either/or. I agree that pushing for regulations is a bigger priority than AI in government. Right now the former is getting dramatically more EA resources and I'd expect that to continue. But I think the latter are getting almost none, and that doesn't seem right to me. 
 

Suppose it turned out Microsoft Office was dangerous. Surely the fact that Office is so embedded in government procedures would make it less likely to get banned?

I worry we're getting into a distant hypothetical. I'd equate this to, "Given the Government is using Microsoft Office, are they likely to try to make sure that future versions of Microsoft Office are better? Especially, in a reckless way?" 

Naively I'd expect a government that uses Microsoft Office to be one with a better understanding of the upsides and downsides of Microsoft Office.

I'd expect that most AI systems the Government would use would be fairly harmless (in terms of the main risks we care about). Like, things a few years old (and thus tested a lot in industry), with less computing power than would be ideal, etc. 

Related, I think that the US military has done good work to make high-reliability software, due to their need for it. (Though this is a complex discussion, as they obviously do a mix of things.)

IIRC you see similar phenomena (although I can't recall any examples off hand) where some government-mandated software has massive security flaws but nobody does anything about it because the software is too entrenched.


Tyler Technologies.

But this is local government not federal.
 

I'd expect that if the US government were far more competent, people would trust it to take care of many more things, including high-touch AI oversight.

 

This is probably true, but improving competence throughout the government would be a massive undertaking, would take a long time and also have a long lag before public opinion would update. Seems like an extremely circuitous route to impact.

I mainly agree.

I previously was addressing Michael's more limited point, "I don't think government competence is what's holding us back from having good AI regulations, it's government willingness."

All that said, separately, I think that "increasing government competence" is often a good bet, as it just comes with a long list of benefits.

But if one believes that AI will happen soon, and that a major bottleneck is "getting the broad public to trust the US government more, with the purpose of then encouraging AI reform", that seems like a dubious strategy. 

I overall agree we should prefer USG to be better AI-integrated. I think this isn't a particularly controversial or surprising conclusion though, so I think the main question is how high a priority this is, and I am somewhat skeptical it is on the ITN pareto frontier. E.g. I would assume plenty of people care about government efficiency and state capacity generally, and a lot of these interventions are generally about making USG more capable rather than too targeted towards longtermist priorities.

So this felt like neither the sort of piece targeted to mainstream US policy folks, nor that convincing for why this should be an EA/longtermist focus area. Still, I hadn't thought much about this before, and so doing this level of medium-depth investigation feels potentially valuable, but I'm unconvinced that e.g. OP should spin up a grantmaker focused on this (not that you were necessarily recommending this).

Also, a few reasons govts may have a better time adopting AI come to mind:

  • Access to large amounts of internal private data
  • Large institutions can better afford one-time upfront costs to train or finetune specialised models, compared to small businesses

But I agree the opposing reasons you give are probably stronger.

we should do what we normally do when juggling different priorities: evaluate the merits and costs of specific interventions, looking for "win-win" opportunities

If only this were how USG juggled its priorities!

the main question is how high a priority this is, and I am somewhat skeptical it is on the ITN pareto frontier. E.g. I would assume plenty of people care about government efficiency and state capacity generally, and a lot of these interventions are generally about making USG more capable rather than too targeted towards longtermist priorities.

Agree that "how high-priority should this be" is a key question, and I'm definitely not sure it's on the ITN pareto frontier! (Nice phrase, btw.) 

Quick notes on some things that raise the importance for me, though:

  1. I agree lots of people care about government efficiency/ state capacity — but I suspect few of them are seriously considering the possibility of transformative AI in the near future, and I think what you do to ~boost capacity looks pretty different in that world
  2. Also/relatedly, my worldview means I have extra reasons to care about state capacity, and given my worldview is unusual that means I should expect the world is underinvesting in state capacity (just like most people would love to see a world with fewer respiratory infections, but tracking the possibility of a bioengineered pandemic means I see stuff like far-UVC/better PPE/etc. as higher value)
    1. More generally I like the "how much more do I care about X" frame — see this piece from 2014
    2. (It could also be a kind of public good.)
  3. In particular, it seems like a *lot* of the theory of change of AI governance+ relies on competent/skillful action/functioning by the US federal government, in periods where AI is starting to radically transform the world (e.g. to mandate testing and be able to tell if that mandate isn't being followed!), and my sense is that this assumption is fragile/the govt may very well not actually be sufficiently competent — so we better be working on getting there, or investing more in plans that don't rely on this assumption

And I'm pretty worried that a decent amount of work aimed at mitigating the risks of AI could end up net-negative (for its own goals) by not tracking this issue and thus not focusing enough on the interventions that are actually worth pursuing --- further harming government AI adoption & competence / capacity in the process (e.g. I think some of the OMB/EO guidance from last year looked positive to me before I dug into this, and now looks negative). So I'd like to nudge some people who work on issues related to existential risk (and government) away from a view like: "all AI is scary/bad, anything that is 'pro-AI' increases existential risk, if this bundle of policies/barriers inhibits a bunch of different AI things then that's probably great even if I think only a tiny fraction is truly (existentially) risky", etc. 

--

this felt like neither the sort of piece targeted to mainstream US policy folks, nor that convincing for why this should be an EA/longtermist focus area. 

Totally reasonable reaction IMO. To a large extent I see this as a straightforward flaw of the piece & how I approached it (partly due to lack of time - see my reply to Michael above), although I'll flag that my main hope was to surface this to people who are in fact kind of in between -- e.g. folks at think tanks that do research on existential security and have government experience/expertise.

--

I'm unconvinced that e.g. OP should spin up a grantmaker focused on this (not that you were necessarily recommending this).

I am in fact not recommending this! (There could be specific interventions in the area that I'd see as worth funding, though, and it's also related to other clusters where something like the above is reasonable IMO.)

--

Also, a few reasons govts may have a better time adopting AI come to mind:

  • Access to large amounts of internal private data
  • Large institutions can better afford one-time upfront costs to train or finetune specialised models, compared to small businesses

But I agree the opposing reasons you give are probably stronger.

The data has to be accessible, though, and this is a pretty big problem. See e.g. footnote 17. 

I agree that a major advantage could be that the federal government can in fact move a lot of money when ~it wants to, and could make some (cross-agency/...) investments into secure models or similar, although my sense is that right now that kind of thing is the exception/aspiration, not the rule/standard practice. (Another advantage is that companies do want to maintain good relationships with the government/admin, and might thus invest more in being useful. Also there are probably a lot of skilled people who are willing to help with this kind of work, for less personal gain.)

--

If only this were how USG juggled its priorities!

🙃 (some decision-makers do, though!)

A bit tangential, but I can't help sharing a data point I came across recently on how prepared the US government currently is for advanced AI: our secretary of education apparently thinks it stands for "A1", like the steak sauce (h/t). (On the bright side, of course, this is a department the administration is looking to shut down.)

I've been thinking a lot about this broad topic and am very sympathetic. Happy to see it getting more discussion.

I think this post correctly flags how difficult it is to get the government to change. 

At the same time, I imagine there might be some very clever strategies to get a lot of the benefits of AI without many of the normal costs of integration.

For example:

  1. The federal government makes heavy use of private contractors. These contractors are faster to adopt innovations like AI.
  2. There are clearly some subsets of the government that matter far more than others. And there are some that are much easier to improve than others.
  3. If AI strategy/intelligence is cheap enough, most of the critical work can be paid for by donors. For example, we have a situation where there's a think tank that uses AI to figure out the best strategies/plans for much of the government, and government officials can choose to pay attention to this.

Basically, I think some level of optimism is warranted, and would suggest more research into that area.

(This is all very similar to previous thinking on how forecasting can be useful to the government.)

I imagine there might be some very clever strategies to get a lot of the benefits of AI without many of the normal costs of integration.

For example:

  1. The federal government makes heavy use of private contractors. These contractors are faster to adopt innovations like AI.
  2. There are clearly some subsets of the government that matter far more than others. And there are some that are much easier to improve than others.
  3. If AI strategy/intelligence is cheap enough, most of the critical work can be paid for by donors. For example, we have a situation where there's a think tank that uses AI to figure out the best strategies/plans for much of the government, and government officials can choose to pay attention to this.

I'd be excited to see more work in this direction! 

Quick notes: I think (1) is maybe the default way I expect things to go fine (although I have some worries about worlds where almost all US federal govt AI capacity is via private contractors). (2) seems right, and I'd want someone who has (or can develop) a deeper understanding of this area than me to explore this. Stuff like (3) seems quite useful, although I'm worried about things like ensuring access to the right kind of data and decision-makers (but partnerships / a mix of (2) and (3) could help). 

(A lot of this probably falls loosely under "build capacity outside the US federal government" in my framework, but I think the lines are very blurry / a lot of the same interventions help with appropriate use/adoption of AI in the government and external capacity. )

all very similar to previous thinking on how forecasting can be useful to the government

I hadn't thought about this — makes sense, and a useful flag, thank you! (I might dig into this a bit more.)

This post is focused on what the government can do but I'm curious if you have thoughts about what the private sector can do to meet the government where it is.

I imagine that palantir is making a killing off of adapting generative AI to work for government requirements, but I assume there are still gaps in the marketplace? Do you have a sense for what these gaps are? is there some large segment of the government which would use generative AI if only it was compliant with standard X?

I think this is a good question, and it's something I sort of wanted to look into and then didn't get to! (If you're interested, I might be able to connect you with someone/some folks who might know more, though.)

Quick general takes on what private companies might be able to do to make their tools more useful on this front (please note that I'm pretty out of my depth here, so take this with a decent amount of salt -- and also this isn't meant to be prioritized or exhaustive): 

  • Some of the vetting/authorization processes (e.g. FedRAMP) are burdensome, and sometimes companies do just give up/don't bother (see footnote 12), which narrows the options for agencies; going through this anyway could be very useful
  • Generally lowering costs for tech products can make a real difference for whether agencies will adopt them -- also maybe open source products are likelier to be used(?) (and there's probably stuff like which chips are available, which systems staff are used to, what tradeoffs on speed vs cost or similar make sense...)
  • Security is useful/important, e.g. the tool/system can be run locally, can be fine-tuned with sensitive data, etc. (Also I expect things like "we can basically prove that the training / behavior of this model satisfies [certain conditions]" will increasingly matter — with conditions like "not trained on X data" or "could not have been corrupted by any small group of people" — but my understanding/thinking here is very vague!)
  • Relatedly I think various properties of the systems' scaffolding will matter; e.g. "how well the tool fits with existing and future systems" — so modularity and interoperability[1] (in general and with common systems) are very useful — and "can this tool be set up ~once but allow for different forms of access/data" (e.g. handle differences in who can see different kinds of data, what info should be logged, etc.)

(Note also there's a pretty huge set of consultancies that focus on helping companies sell to the government, but the frame is quite different.)

And then in terms of ~market gaps, I'm again very unsure, but expect that (unsurprisingly) lower-budget agencies will be especially undersupplied — in particular the DOD has a lot more funding and capacity for this kind of thing — so building things for e.g. NIST could make sense. (Although it might be hard to figure out what would be particularly useful for agencies like NIST without actually being at NIST. I haven't really thought about this!)

  1. ^

    I haven't looked into this at all, but given the prevalence of Microsoft systems (Azure etc.) in the US federal government (which afaik is greater than what we see in the UK), I wonder if Microsoft's relationship with OpenAI explains why we have ChatGPT Gov in the US, while Anthropic is collaborating with the UK government https://www.anthropic.com/news/mou-uk-government 

I assume there are still gaps in the marketplace? Do you have a sense for what these gaps are? is there some large segment of the government which would use generative AI if only it was compliant with standard X?

Government Procurement itself. 

The process is incredibly burdensome and time consuming for the procurement officer. Templates exist but if you have never seen one before you would swear it was designed explicitly to prevent the successful procurement of anything. In truth, they are drowned in legalease in order to help them weather protests from unsuccessful bidders. But this makes is so that sometimes it takes 10 to 15 minutes of reading to even figure out what industry a bid opportunity is in and what the agency wants to buy. Then you have to go find out what the minimum qualifications are so you know if your company is even allowed to bid. 

Some poor procurement officer had to add all that gobbledygook to the template, at least one person has to approve it, and then it has to be added to at least one online procurement system. The Feds have many (though GSA is the largest grants.gov is another and several agencies have their own stand-alone procurement sites like Dept of Education and Health and Human Services). States have a ridiculous amount of them because the state level will have one or two, counties might use the state procurement system or have their own, and cities virtually always use aggregator sites rather than having their own. And that's before you get into the cooperative bidding tools/communities like NASPO, Managed Service Providers (MSPs), etc. which support groups of agencies or groups of states. Also, there are some 20-ish different types of bid requests that all have different rules and forms, and not every region uses the acronyms the same way.

And the biggest pain in the *** "statute X" is 2 CFR 200 period of performance. This is the infamous "use it or loose it" clause in federal procedures. The very definition of this statute in the regulations effectively says "even if we say you can use these funds over the next three years, we might change our mind and take anything that you have left back on September 30 of any given year.":

Period of performance means the time interval between the start and end date of a Federal award, which may include one or more budget periods. Identification of the period of performance in the Federal award consistent with § 200.211(b)(5) does not commit the Federal agency to fund the award beyond the currently approved budget period.

Source: eCFR :: 2 CFR Part 200 -- Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards

 

Therefore, a bit of extremely low hanging fruit for someone with the skills:

  • An AI agent that can turn a bullet list of specs into a compliant procurement document aka RFP (Request for Proposals) or RFQ (Request for Quotes) or ITB (Invitation to Bid) or Tender or etc.
  • Generate an appropriate list of potential evaluators from the agency address book such that they have: time on calendar to invest in the review, appropriate and active procurement authority, relevant background to understand the procurement
  • Draft and route request for participation in the committee to potential reviewers, including (when necessary) permission from their supervisor if an evaluator is in a different silo'd team from the procuring team
  • Repeat as needed until a full team is assembled
  • Create a scoring rubric for the evaluation committee to use
  • Review the prior bids for "same, kind, or like" procurements and generate an estimate of cost, as well as allowability and allocability of fund sources (not all government funds can be used for all activities, an activity may be allowable for an agency to perform but they may not have any fund sources to which the cost can be allocated because it is not allowable for those fund sources to pay for)
  • Route the resulting bid request and funding proposal to the appropriate people for approvals via a digital approval process (vs some agencies still getting ink signoffs)
  • Post the approved procurement document on the correct procurement board(s)
  • Send courtesy notifications to vendors via that same tool that the bid is up
  • During the Q&A period, field questions from potential bidders and generate sample answers for approval by the Procurement Officer, who can then just approve or edit before the AI posts the answer to the procurement board(s) as an addendum, either individually or in batches
  • Confirm receipt of bids from bidders so they know if they actually used the board correctly and whether any critical document was missing from their bid
  • On the closing date, archive all bids both complete and incomplete and notify the procurement officer, and create redacted versions ready to be sent out in response to FOIA requests
  • Route complete and timely bids to the evaluation committee with the scoring rubric
  • Route completed scores to the procurement officer for final decision
  • Draft and Post award notice to procurement board(s)
  • As needed draft protest responses for the procurement officer and route them for approval/editing before sending to the bidders
  • As needed draft debriefing notes for the procurement officer to review with bidders who want to know why they lost and how they can do better next time
  • As needed send redacted responses and draft letters in response to FOIA requests
  • Archive complete procurement file in appropriate secure repository, and make FOIA-ready version of the whole thing

This is a reasonably comprehensive-but-generic procurement lifecycle (not counting the contract and implementation portions) that should work for any federal, state, or local entity.

Except for the interfacing with existing software, all of this is doable with ChatGPT prompts currently AFAIK. Though when trying I've had limited success getting it to understand the templates. AFAIK no agency is using AI to help in any step of this process in a meaningful way. I'd love to be proven wrong.

 

There's many other areas that could also provide huge gains for the government, but this one is so completely within the capacity of current AI that I'm wondering whether the government will start using AI to speed up the task or Microsoft will make it entirely possible within MS Word/Copilot first.

I think it would be great to have some materials adapted for policy audiences if it isn't too far out of your team's scope. There is a lot of demand for this kind of practical, implementation-focused work. Just this week, there were multiple US congressional hearings and private events on the future of AI in the US, with a specific focus on adapting to a world with advanced artificial intelligence. 

As an example, the Special Competitive Studies Project hosted the AI+ summit series in DC and launched a free course on "Integrating Artificial Intelligence into Public Sector Missions". These have been very well-received and attended by stakeholders across agencies and non-governmental entities. While SCSP has done more to prepare government stakeholders to adapt than any other nonprofit I am aware of, there is still plenty room for other expert takes.

More from Lizka
Curated and popular this week
Relevant opportunities