Comments
How to Solve AI Biosecurity
my best 11,600-word guess - Bioweapon.AI is finally finished!!
Some notes from me:
read on bioweapon.ai
An investigation into the history of biological weapons, and how AI and emerging technology are now eroding the barriers that have kept catastrophe rare.
In the spring of 2023, a man walked into the White House carrying a small black box. Inside it were a dozen test tubes containing ingredients that, correctly assembled, had the potential to start a pandemic; an AI chatbot had supplied the recipe.
Fortunately, the man's name was Rocco Casagrande – and, as a biochemist and former United Nations weapons inspector, he wasn't there to use the materials; instead, he was there to brief government officials on how AI could help someone identify potent agents, secure the materials to make them, and soon – he warned – design entirely novel pathogens capable of evading humans' immune systems.
That was three years ago.
In April 2026, the New York Times published an investigation into what happens when you ask AI's most capable models for help with biological weapons.
Examining several chat transcripts shared by scientists, they wrote:
…OpenAI's ChatGPT explained how to use a weather balloon to spread biological payloads over a U.S. city. In another chat, Google's Gemini ranked pathogens by how much they could damage the cattle or pork industries. Anthropic's Claude produced a recipe for a novel toxin adapted from a cancer drug. Other chats contained information that [an expert deemed] too dangerous to share.
…[T]he chatbot explained how to modify an infamous pathogen in a lab so that it would resist known treatments. Worse, the bot described in vivid detail how to release the superbug, identifying a security lapse in a large public transit system… [t]he bot outlined a plan to maximize casualties and minimize the chances of being caught.
From their investigation, the Times authors concluded:
…[E]ven publicly available models can do more than disseminate dangerous information. The virtual assistants have described in lucid, bullet-pointed detail how to buy raw genetic material, turn it into deadly weapons and deploy them in public spaces, the transcripts show. Some have even brainstormed ways to evade detection.
More alarming than these examples is just how quickly things have changed in the past two years. In December 2024, a multi-institutional team of researchers from Stanford, MIT, Princeton, Cohere, Mistral AI, and others published a systematic review of the AI biorisk literature and essentially concluded that "current LLMs and BTs do not pose an immediate risk."
The team's paper cited both an OpenAI red-teaming exercise and a Claude 3 safety report released earlier that same year. Yet, by April 2025, OpenAI had reversed its own assessment, concluding its models were approaching "high risk" – meaning, capable of "substantially increas[ing] the likelihood and frequency of bioterrorist attacks." Similarly, Anthropic's own internal bioweapons acquisition uplift trials published in May 2025 found that Claude Opus 4 enhanced human performance by 2.53× on relevant tasks, enough to trigger activation of AI Safety Level 3. An international safety report produced for the 2025 Paris AI Action Summit found LLM performance on weapons-related queries improved a staggering 80 percent in 2024 alone.
The institutions designed to prevent biological attacks have historically operated on the assumption that the expertise required to develop them is rare and slow to acquire.
AI and other emerging technologies may be on track to change this, and quickly.
This is not a fringe concern:
"The biggest issue with AI is actually going to be … its use in biological conflict."
— Eric Schmidt, former CEO of Google
"[AI has the potential to] greatly widen the range of actors with the technical capability to conduct a large-scale biological attack."
— Dario Amodei, CEO of Anthropic
We are not prepared.
Part One · Rogue Actors
An Analysis of Historical Case Studies - Part 1: Rogue Actors
Note: Refers to successful deployment of a biological agent, not necessarily success in achieving overall objectives.
A. 1984 Rajneeshee Bioterror Attack
[ Video — Watch · The Dalles salmonella attack · https://youtu.be/vTAVzg_ny48 ]
In September 1984, the Rajneeshee cult deliberately contaminated salad bars in ten Oregon restaurants with salmonella in an effort to incapacitate voters before the 1984 Wasco County election.
Later investigations revealed that the attackers had a "fairly sophisticated medical research laboratory" at their commune, where they cultivated salmonella purchased over-the-counter from a Seattle scientific supply house. Ma Anand Puja, a former nurse who ran the Rajneesh Medical Corporation, provided the minimal medical expertise needed to culture common pathogens. Authorities also later discovered that the cult had explored using additional lethal pathogens, including HIV.
Though the cult had access to a medical research laboratory, the actual salmonella cultivation took place in a shed:
In a simple shed, under the supervision of Ma Anand Puja, the cult mass-produced the Salmonella using simple petri dishes, an incubator, and a freeze-drier [14]… Producing large quantities of bacteria is cheap and can be easily done with even rudimentary equipment and skills [18].
The attack poisoned 751 people and hospitalized 45, while causing panic that drained the local economy. To this day, the Rajneeshee operation remains the largest bioterrorist attack in U.S. history.
An image of a salsa bar contaminated as part of the attack. Location: Taco Time in The Dalles, Oregon. Source: Slate Magazine
B. WWI German Biological Sabotage Program
"A shipment of horses at a New York City rail yard, 1918." Source: Archives.gov
During WWI, Dr. Anton Casimir Dilger – a German-American medical doctor specializing in tissue culture research – carried out a German biological warfare sabotage program targeting Allied livestock supplies. With the help of his brewmaster brother and housekeeper sister, Dilger converted his basement into a makeshift bacterial laboratory where he began cultivating deadly bacterial cultures in liquid form. He used anthrax and glanders bacteria to infect horses and mules being shipped from U.S. ports to Europe. Dilger also coordinated directly with German General Staff.
His basement was located less than six miles from the White House, and as a result of the program, "thousands of horses and mules were killed" (National Archives).
C. 2001 Anthrax Letters (also known as the Amerithrax case)
"Laboratory technician holding an anthrax-laced letter sent to Senator Patrick Leahy." Source: FBI
Following the September 11 attacks, a series of anthrax attacks were carried out through letters containing anthrax spores sent via the mail. The spores had been treated with additives to increase their inhalability, which suggested the involvement of someone with highly specialized, advanced technical expertise. The primary suspect was Bruce Ivins, a veteran biological-weapon researcher for the US Army.
The attack killed 5 people and injured 17, with many survivors suffering long-term fatigue and memory loss even years later. It also created widespread panic at a moment when the nation was already reeling from 9/11.
[ Video — Watch · The 2001 anthrax letters · https://youtu.be/UQg7SM61fZ8 ]
A Netflix documentary was later created on the attacks.
A. 1993 Aum Shinrikyo Anthrax Attack
[ Video — Watch · Aum Shinrikyo · https://youtu.be/eWZ9jXI1d2I ]
In July 1993, doomsday cult Aum Shinrikyo aerosolized a liquid suspension of Bacillus anthracis (anthrax) from the roof of an eight-story building in Kameido, Tokyo, but the operation resulted in no human casualties.
The operation failed because:
Aum had laboratories for experimenting with bioweapons in Kamakuishki and Tokyo. In addition to anthrax, the group also "cultured and experimented with botulin toxin, cholera, and Q fever" (CDC). After failing to deploy a bioweapon, they later transitioned to successful chemical attacks, including the Matsumoto Sarin Attack and the Tokyo Subway Sarin Attack (not detailed here since this analysis is about bioweapons).
[ Video — Watch · From bioweapons to the Tokyo subway sarin attack · https://youtu.be/QVRGYlUsE9I ]
B. 2018 Cologne Ricin Plot
In June 2018, Sief Allah H., a 29-year-old ISIS sympathizer, successfully produced enough ricin for "up to 1,000 toxic doses" and assembled a bomb with explosives and metal ball bearings for a planned mass-casualty attack on a crowded indoor venue. This marked the first time a jihadi terrorist in the West successfully produced the toxic biological agent, demonstrating that technical barriers to simple bioweapons are sometimes surmountable with publicly available instructions.
German authorities discovered and put a stop to the plan via online surveillance operations, specifically after the U.S. CIA provided a tip about his large order of 3,300 castor seeds purchased over the internet.
C. 1995 Minnesota Patriots Council Ricin Plot
"Ricin occurs naturally in castor beans." Source: BBC
The Minnesota Patriots Council was an anti-government militia that attempted to assassinate federal law enforcement officials (including US Marshals, IRS agents, and local sheriffs) using ricin. The group successfully extracted ricin from mail-ordered castor beans using publicly available instructions and basic solvent knowledge gained through their work as carpet cleaners. They initially found and purchased castor beans through an advertisement in a right-wing magazine.
Ultimately, the Minnesota Patriots Council successfully produced enough ricin to kill over 100 people, despite their result being significantly less potent than professional-grade toxin. Similarly to the Cologne Ricin Plot, law enforcement detected and disrupted their activities before they could carry out planned attacks.
D. 1998 Toxic Terror Case
The perpetrators of the Toxic Terror Case were Larry Wayne Harris, a microbiologist and white supremacist with ties to "Aryan Nations," working alongside William Leavitt, a friend who "own[ed] biological laboratories in Nevada and Germany."
The pair planned to release bubonic plague bacteria in the New York subway system; law enforcement raided their labs after they "mail-ordered freeze-dried bubonic plague germs." During the raids, officials also found vials containing suspected anthrax. Harris reportedly told a witness that he had procured enough anthrax to "wipe out the city."
Addendum: ISIS Laptop of Doom
The Islamic State's Terror Laptop of Doom revealed a comprehensive biological weapons research program documented on the laptop of a Tunisian ISIS operative with university-level scientific training. Uncovered in 2014, it contained a "19-page document in Arabic on how to develop biological weapons and how to weaponize the bubonic plague from infected animals" containing guidance like "[u]se small grenades with the virus, and throw them in closed areas like metros, soccer stadiums, or entertainment centers… best to do it next to the air-conditioning."
Successes
| Case | Expertise Level | Available Infrastructure | Key Success Factor(s) |
|---|---|---|---|
| Rajneeshee | Minimal (nurse practitioner's basic medical knowledge) | Institutional ("sophisticated medical research laboratory"), though actual bioweapon development took place in a shed using basic tools | Easily cultivated pathogen, which didn't require as much specialized expertise to pull off |
| Dilger | High (tissue culture specialist) | Lab set up in basement with substantial institutional backing (German General Staff coordination) | Expert knowledge + state sponsorship + help from siblings, which maintained opsec |
| Amerithrax | High (microbiologist, U.S. Army)* | Institutional access (Army lab materials – USAMRIID) | Technical expertise + authorized access to dangerous pathogens due to military work |
*Whether or not Ivins was the culprit remains contested.
Takeaways
A. Rajneeshee represents a case where despite minimal expertise, the perpetrators were still able to succeed because they chose an easier pathogen to synthesize. The expertise bottleneck varies by pathogen.
B. Dilger was able to maintain operational security by relying on his siblings to help him with his work. While he had to rely on a home laboratory, it's likely that coordination with German General Staff enabled him to procure advanced materials and equipment.
C. The Amerithrax (aka Anthrax Letters) case is perhaps slightly more difficult to assess with confidence because Bruce Ivins's guilt remains contested. However, if Ivins was the culprit, key success factors include highly specialized expertise and authorized access to dangerous pathogens as a researcher for the U.S. army.
The Dilger and Ivins cases demonstrate that specialized biological knowledge poses severe risks; advanced expertise can enable catastrophic biological attacks.
Failures
The failed cases fall into two main categories: technical incompetence and detection via surveillance.
Technical Incompetence
The Aum Shinrikyo Anthrax Attack failed on technical problems. However, had Aum expanded their technical expertise beyond their compartmentalized leadership structure, there's a reasonable chance their attack may have succeeded. Their operation went undetected through deployment, and every failure point (strain selection, spore concentration levels, dispersal, sunlight exposure, etc) represented knowledge gaps that additional specialized expertise could have addressed.
The Amerithrax case demonstrates that the technical barriers Aum encountered were surmountable with proper expertise.
Detection via Surveillance
Unlike Aum's technical failures, the Cologne and Minnesota Patriots cases succeeded at the production phase. These actors proved that certain bioweapon manufacturing is already within reach of non-experts using publicly available information. What prevented attacks was detection and disruption by law enforcement, rather than inability to create functional weapons.
Whether Toxic Terror succeeded at the production phase is less clear, as the vials found by law enforcement were only suspected to contain anthrax. It could plausibly fall into either of the above categories, depending on whether the recovered materials were viable and weaponizable.
Bioweapons have been created and deployed effectively in several documented attacks. The Rajneeshee, Dilger, and Amerithrax cases prove that bioweapon development and deployment are within reach of motivated rogue actors.
At the same time, these successes have been mostly limited in scope and severity. Several other large-scale near-misses have failed primarily due to technical incompetence or detection by law enforcement.
AI, alongside other existing and emerging technologies, threatens to erode some of the barriers that have historically caused malicious actors to fail. That will be the subject of Part 2 of this series!
Part Two · Government Programs
Analysis of Historical Case Studies - Part 2: Government-Sponsored Programs
Governments have repeatedly demonstrated both the capability and the willingness to develop and deploy bioweapons.
A. The Soviet Biopreparat Program
[ Video — Watch · Biopreparat & the 1979 Sverdlovsk anthrax release · https://youtu.be/x3FMww5biJ8 ]
In 1972, the Soviet Union signed the Biological and Toxin Weapons Convention (BTWC), pledging not to develop, produce, or stockpile bioweapons. A year later, the Soviet Union violated the BTWC by creating Biopreparat, a massive biological warfare enterprise masquerading as a civilian pharmaceutical company.
Through the program, Soviet scientists reportedly manufactured an estimated 20 tons of plague, 20 tons of smallpox, and hundreds of tons of anthrax; they also weaponized tularemia, epidemic typhus, Q fever, and Marburg virus, while studying the potential use of Ebola and encephalitis.
The program's exact scale remains disputed. The Federation of American Scientists states the following:
At its peak, the former Soviet Union had the world's largest biological warfare program, with somewhere between 25,000 and 32,000 people employed in a network of 20 to 30 military and civilian laboratories and research institutions. An additional 10,000 or so worked in Defense Ministry bioweapons laboratories. According to other estimates, at least 47 labs and test facilities were scattered across Russia, employing more than 40,000 workers, 9,000 of whom were scientists. Between 1,000 to 2,000 of those scientists were experts on deadly pathogens.
Meanwhile, Congressional testimony asserts:
The size and scope of this program were enormous. For example, in the late 1980s and early 1990s, over 60,000 people were involved in the research, development, and production of biological weapons.
– Dr. Kenneth Alibek, before the Joint Economic Committee, United States Congress
Regardless of the precise number, Biopreparat was unquestionably the largest biological warfare program in history.
Even decades after its official end, the program's deadly legacy endures. Contaminated testing sites like Vozrozhdeniya Island still harbor active anthrax spores buried in the soil since the Soviet era:
Vozrozhdeniya was once home to a vibrant fishing village fringed by turquoise lagoons, back when the Aral Sea was the fourth-largest in the world and abundant with fish…
Over the years the site flourished into a living nightmare, where anthrax, smallpox and the plague hung in great clouds over the land, and exotic diseases such as tularemia, brucellosis, and typhus rained down and seeped into the sandy soil.
– BBC, "The deadly germ warfare island abandoned by the Soviets"
The true nature of the program only became clear after the Soviet collapse in the 1990s.
The Soviet Biopreparat program is an especially important case study because it reveals that even absent deliberate deployment, the existence of national bioweapons programs creates significant danger through the persistent risk of laboratory accidents. A historical example of this is the 1979 Sverdlovsk anthrax release (detailed in the video above), which killed "at least 68 people"; another example of this is the 1971 Aral smallpox incident:
[ Video — Watch · The 1971 Aral smallpox incident · https://youtu.be/tH0B-r9L4GY ]
B. Imperial Japan's Unit 731
[ Video — Watch · Imperial Japan's Unit 731 · https://youtu.be/i3qjjXBQnzM ]
Before the Soviet Union developed Biopreparat, Imperial Japan's Unit 731 demonstrated that biological weapons could work on a massive scale. Active from 1936 to 1945, Unit 731 is notorious for its systematic human experimentation and operational deployment of biological weapons against Chinese civilians.
Unit 731 researchers reportedly used "political prisoners, criminals, the poor, and homeless" in their tests; test subjects also "included women and children."
[ Video — Watch · The human experiments of Unit 731 · https://youtu.be/AXM1DozZwjk ]
Unit 731 wasn't the only bioweapons development program to conduct experiments on humans, but it was the only one to systematically integrate mass human experimentation with active biological warfare. Researchers used thousands of human subjects to perfect biological weapons that were immediately deployed against civilian populations. An estimated 3,000 died in horrific experiments, while tens of thousands more were killed by the weapons those experiments helped develop:
Elderly plaintiffs flew from China to testify – often in tears – about their communities being ravaged by diseases that spread mysteriously after Japanese planes flew low overhead and dropped wheat, rice or cotton infested with fleas…
iTrigger warning: graphic content.
Researchers performed surgeries and vivisections on their victims without the use of anesthesia, removing organs and severing limbs… [s]ome victims had their limbs amputated and reattached to other parts of their body; others were subjected to extreme cold to gauge the effects of frostbite and gangrene on human skin. Many were exposed to poison gas or deadly diseases to observe the amount of time it took a person to show the effects or symptoms before dying… doctors subjected victims to starvation, dehydration, extreme air pressure, or electrical current to see how long a human could survive under such conditions… estimates ranging from thirty thousand to more than five hundred thousand are believed to have died in field tests of diseases on the Chinese population.
– EBSCO
According to EBSCO, Unit 731 also focused on creating weaponized deployment options: they developed a bomb of plague-infested fleas, and planned to deploy their weapons using balloons sent "adrift across the Pacific" or through aircraft-delivered plague bombs on U.S. cities.
The war ended before those plans were executed; but as evidenced by Biopreparat, what the program had demonstrated about the feasibility of mass-casualty biological warfare did not end with it.
[ Video — Watch · The legacy of Unit 731 · https://youtu.be/LqB5fAHjyME ]
If Biopreparat and Unit 731 established that states will build these weapons when they see a use for them, the question for the present is who is still doing so. The U.S. government's answer names four states.
A. Russia
Russia is the direct heir to the largest bioweapons program in history, and the U.S. government does not believe the inheritance was ever fully renounced. According to a State Department report released in 2025, U.S. intelligence believes that the Soviet program was absorbed rather than dismantled by the Russian Federation, and that Russia currently "maintains an offensive biological warfare program."
In the wake of Russia's invasion of Ukraine, the program appears to be expanding. In October 2024, the Washington Post reported on satellite imagery of Sergiev Posad-6, a military site northeast of Moscow with a Cold War history of weaponizing smallpox and Ebola:
A few months after Russia began its full-scale invasion of Ukraine in 2022, satellite imagery captured unusual activity at a restricted military research facility… construction vehicles renovating the old Soviet-era laboratory and breaking ground on 10 new buildings, totaling more than 250,000 square feet, with several of them bearing hallmarks of biological labs designed to handle extremely dangerous pathogens.
– Washington Post, "Satellite images show major expansion at Russian site with secret bioweapons past"
Satellite imagery of Russia; area believed to be a historical bioweapons site. Source: Washington Post
Russian officials have framed the expansion as biodefense. It's impossible to discern from satellite photos alone what the true nature of the facilities is.
B. China
The U.S. government states that China possessed its own biological weapons program from "the 1950s to at least the late 1980s." As part of the program, the country is said to have "weaponized ricin, botulinum toxins, and the causative agents of anthrax, cholera, plague, and tularemia."
What's unresolved is whether "at least the late 1980s" was actually the end. The State Department does not assert the program continued, but it does not rule out the possibility: China has never disclosed its historical facilities in any BWC submission, and the same report flags continuing biotech research by Chinese military medical institutes that could have weapons uses. This year, there's also a new concern:
For the first time, this year's report warns that China probably is capable of using publicly available artificial intelligence and machine learning (AI/ML) tools to advance efforts related to biological weapons applications. At the same time, China "probably is unable to make complex scientific equipment without Western innovation."
– Council on Strategic Risks, "The State of Compliance with WMD-Related Treaties"
Notably, the AI dimension flagged here is not unique to China; it runs through every contemporary case, and it is the subject of the analysis to come.
C. North Korea
Photo by Mike Bravo on Unsplash
While China is assessed as a "compliance concern," the State Department states the case for North Korea plainly: the U.S. assesses that the DPRK has a dedicated, national-level offensive biological weapons program, and possesses the technical capability to produce pathogens and toxins usable as bioweapons, and to engineer them genetically.
The United States assesses that the DPRK has a dedicated, national-level offensive [biological weapons] program… Pyongyang probably is capable of weaponizing BW agents with unconventional systems such as sprayers and poison pen injection devices, which have been deployed by the DPRK for delivery of chemical weapons and could be used to covertly deliver BW agents.
While the State Department's paper does not dive into the specifics of the DPRK's program, South Korean defense analysts believe that "B. anthracis, Variola virus, Yersinia pestis, Vibrio cholerae, and botulinum toxin" are the most likely candidates for weaponization.
These same agents are classified as top-tier biological threats by both the U.S. Centers for Disease Control and Prevention (CDC) and WHO.
– Jungeun Lee, Korea Research Institute for Defense Technology Planning and Advancement
Concerningly, the June 2024 Treaty on Comprehensive Strategic Partnership between Russia and North Korea establishes plainly that the two states will "actively encourage joint research in the field of science and technology, including such areas as space, biology, peaceful nuclear energy, artificial intelligence, [and] information technology."
D. Iran
Russia and North Korea have confirmed offensive programs, and there is a verified historical program with unanswered questions about the extent to which it continues in China. Meanwhile, Iran's case is interesting because it centers on the dual-use problem: the U.S. says that "Iran has not abandoned its intention to conduct research and development of biological agents and toxins for offensive purposes," and that it has facilities that possess the capability to produce bioweapons, if directed:
Iran maintains flexibility to use, upon leadership demand, legitimate research underway for biodefense and public health purposes for a capability to produce lethal BW agents. It is unknown if Iran's leadership has set a directive to maintain this flexibility.
States succeed where others fail because they have substantially more resources than typical rogue actors. Biopreparat was composed of tens of thousands of experts, and states can draw from large military budgets to finance their operations. That's true, but the more interesting read is that state resources reliably buy capability. Biopreparat and Unit 731 were successfully able to synthesize deadly bioweapons, and Russia, China, and North Korea all appear to possess some capability to produce bioweapons, with Iran assessed as able to develop it on demand. But capability is not the same as use: of every program in this essay, only Unit 731 ever achieved mass-casualty deployment.
Most of the time, even states that solved the resource and expertise problems mostly declined to use what they built. Why is that? The reason appears to be that biological weapons have rarely offered enough strategic utility to justify the risks of using them on a battlefield, in a conflict between two states.
Countries have specific incentives: they want to protect their own troops during wartime. Blowback risks, along with the threat of nuclear retaliation in some cases (the UK, for example, reserves the right to deploy nuclear weapons if chemical or biological weapons are used against its people), complicate this goal. They are further bounded from bioweapons use by a near-universal international norm. Ultimately, the fact that states also have a wide variety of conventional weapons in their arsenals makes it difficult to justify deploying bioweaponry.
However, the degree to which this constraint is binding depends upon a rational state actor weighing controllable outcomes. In recent history, the limiting factor has never solely been the technical challenge of developing these weapons: instead, it has primarily been the fact that those with the capability to build them often did not consider their use worthwhile. This restraint appears to be a property of who holds the capability, not of the weapons themselves – and as evidenced by Unit 731, Aralsk-7, and Sverdlovsk, even that hasn't been a fully successful restraint in the past.
The relevant question for the present is what happens when that capability becomes available to actors whose calculations are different: namely, actors who are not deterred by blowback, not bounded by international norms, unafraid of state-level deterrence, and not optimizing for controllable outcomes.
The restraint was never about the weapons. It was about who held them. What happens when that changes?
Part Three · The Bottlenecks
How AI and emerging technology are undermining the barriers that have kept catastrophic bioweapons rare.
Four bottlenecks have historically kept bioweapons out of reach. Each section below is tagged with the one(s) it erodes; in the case studies at the end, individual passages are highlighted with a margin note.
Capital
The total cost of developing and producing a weapon.
Equipment
Access to the physical lab gear and facilities required.
Knowledge
The information (e.g., sequences of harmful pathogens to use for synthesis) required.
Expertise
The trained human skill (commonly referred to as "tacit knowledge") required to execute the work.
Knowledge
In April 2025, publicly available models like o3 had already outperformed 94% of virology experts on laboratory protocol questions, even on questions directly relevant to the experts' specialties; SecureBio's recent assessment of GPT-5.5 put it in the 100th percentile on that same evaluation.
AI model scores on the Virology Capabilities Test. Note: GPT-5.5 not included in the above; see below. Source: AI Frontiers
"Figure 2.1.1.2 Model performance compared to human SMEs on VCT (full set, multimodal)… Pre-Release Checkpoint 2 outperforms all SMEs across all samples." Source: SecureBio
That benchmark performance hasn't yet translated cleanly into end-to-end weapons development guidance, but the trajectory is clear. Models are advancing toward the ability to provide expert-level virological guidance on demand, anonymously, and at scale. Anthropic's own internal bioweapons acquisition uplift trials found that Claude Opus 4 enhanced human performance by 2.53× on relevant tasks, enough to trigger activation of AI Safety Level 3.
AI's erosion of the knowledge bottleneck operates within a broader landscape of technological advancement. Simultaneously, progress in dispersal technologies, automated labs, and DNA synthesis have each independently eroded distinct bottlenecks that once constrained bioweapon development.
Dispersal Technologies
Equipment
Aum Shinrikyo faced equipment failures when they were attempting to deploy anthrax in 1993. However, multiple breakthroughs in aerosol technologies have occurred since the 90s that could make dispersal of a bioweapon significantly easier and less error-prone.
For example, bag-on-valve technology separates liquid completely from the propellant using a hermetically sealed bag inside the can; the liquid being dispersed maintains complete purity since it's never contaminated by propellant gases. Moreover, crop duster drone technology and other commercial drones have dual-usability; the Institute for National Strategic Studies writes that there is significant risk of "the use of crop dusters as delivery vehicles for biological or chemical weapons of mass destruction."
Cloud Labs & Contract Research Organizations
Equipment Expertise
Cloud labs. Companies like Emerald Cloud Lab and Strateos allow anyone to design and run biological experiments remotely. Customers submit experimental protocols through a software interface, and robotic systems in a physical lab execute them. Currently, anyone can use a cloud lab with no coding experience, and there are no regulations requiring identity verification (KYC), nor are there any mandatory legal mechanisms for monitoring what experiments users run. Anyone with a credit card and an internet connection can run experiments that would previously have required institutional affiliation, costly specialized equipment, physical lab access, and years of hands-on expertise to execute.
Contract research organizations (CROs). These are labs for hire that will execute experiments on behalf of clients. Like cloud labs, the regulatory oversight here is minimal.
Why do these matter?
The threat from cloud labs and CROs is not that they will manufacture bioweapons on behalf of customers. Rather, they enable a malicious actor to fragment their research across multiple platforms and providers, making it increasingly likely they will be able to defeat traditional detection mechanisms.
More specifically, a bioweapon developer could outsource the process of running iterative experiments to cloud labs, which means they may never need to purchase the large quantities of materials that would normally be flagged for suspicious activity. Each individual experiment, run through various cloud labs under a shell company name, might appear to be legitimate research on vaccines or protein expression, but together, they could constitute the development pipeline for a biological weapon.
DNA Synthesis
Knowledge Capital Equipment
What is DNA synthesis?
DNA synthesis is the process of building custom DNA sequences. A researcher types a desired sequence into a computer, submits an order to a commercial provider, and receives a vial of synthetic DNA in the mail. The process is quick and can be relatively inexpensive ($0.07 to $0.09 per base pair in some cases; other companies offer flat rates).
Why is it dangerous?
Some viruses' entire genomes can be assembled from commercially purchased DNA fragments. In 2006, an investigative journalist with the Guardian was able to mail order a "modified sequence of smallpox DNA," and their order was not screened by the provider since it was less than 100 letters long.
What's the current state of screening requirements?
As I've written about extensively here, no country currently has a law requiring gene synthesis providers to screen DNA orders for dangerous sequences. The screening that does exist is voluntary, inconsistent, and has no mechanism for detecting split orders across multiple providers.
The net effect of these technological advancements, among others, is that many of the traditional barriers that used to constrain bioweapons development – including physical lab access and years of hands-on training – are becoming less binding. Increasingly, you can outsource iterative experimentation to cloud labs and CROs, outsource sequence design to AI or find publicly available sequences for dangerous pathogens on the internet, and order the genetic material from synthesis providers with minimal screening.
The risks of these emerging technologies might appear purely theoretical – but case studies tell a different story.
As mentioned briefly above, in 2006, an investigative journalist with the Guardian was able to mail order a "modified sequence of smallpox DNA", and their order was not screened by the provider since it was less than 100 letters long.
The journalist placed an online order using "an invented company name along with just a mobile telephone number and free email address."
The Guardian writes:
The DNA sequence of smallpox, as well as other potentially dangerous pathogens such as poliovirus and 1918 flu are freely available in online public databases. So to build a virus from scratch, a terrorist would simply order consecutive lengths of DNA along the sequence and glue them together in the correct order. This is beyond the skills and equipment of the kitchen chemist, but could be achieved by a well-funded terrorist with access to a basic lab and PhD-level personnel.
In a separate 2017 incident, two scientists from the University of Alberta synthetically recreated the previously eradicated horsepox virus for a mere $100,000. CSIS writes:
Rather than developing the virus themselves, the scientists outsourced much of the initial work – custom ordering fragments of the DNA from a commercial synthesis lab, which "printed" and shipped the viral DNA back to them via mail. The scientists then linked the fragments together in a lab and introduced them into cells using a helper virus, producing the final horsepox virus. Though the study exclusively aimed to improve vaccine and cancer treatments (rather than produce bioweapons), it nevertheless attracted widespread alarm at the time by indicating that reviving smallpox – a close cousin to horsepox and one of the deadliest diseases known to mankind – would, as Science reported, "probably take a small scientific team with little specialized knowledge half a year."
Dying of smallpox doesn't seem particularly fun.
So what should we do about it?
Part Four · Policy Interventions
Eight concrete interventions to defend against AI-enabled biological threats.
Prevention & Detection
A beautiful eval. Source: METR
Background: Today, the question of whether a given AI model poses meaningful biological risk is answered inconsistently, and often by the very companies with a financial incentive to deploy their models as quickly as possible.
Anthropic conducts what it calls "uplift trials;" OpenAI has conducted its own internal red-teaming. Results are not always disclosed publicly, and no agreed definition exists for what constitutes unacceptable biological risk, nor is there a standardized protocol to ensure models above a certain threshold of capability have specific safeguards attached to them.
No legislation requiring pre-deployment bio evaluation currently exists.
Meanwhile, the capability frontier is advancing. As I wrote in the Bottlenecks section:
In April 2025, publicly available models like o3 had already outperformed 94% of virology experts on laboratory protocol questions, even on questions directly relevant to the experts' specialties; SecureBio's recent assessment of GPT-5.5 put it in the 100th percentile on that same evaluation.
Moreover, as I've written about previously:
A recent study led by Microsoft and IBBIS researchers demonstrated that open-source AI tools could engineer new protein variants of known proteins of concern that successfully evaded synthesis screening. As Nobel laureate David Baker and Harvard geneticist George Church have emphasized, screening based on homology alone is unlikely to be sufficient when de novo protein design can produce functionally dangerous proteins with no recognizable homology.
The Intervention: Require all developers of frontier AI models and AI protein design tools to conduct standardized pre-deployment bio evaluations before releasing any model to the public, with evaluations for protein design tools focused specifically on their capability to engineer dangerous protein variants.
Evaluations must be conducted by certified third-party evaluators and reported to the federal government. Models exceeding a defined risk threshold must implement tiered access controls, capability restrictions, delay deployment indefinitely, or other safeguards until identified risks are mitigated.
Recommendations
NIST / Center for AI Standards and Innovation (CAISI)
Department of Health and Human Services
Federal Funding Agencies (NIH, NSF, DARPA)
Congress
International Coordination
Photo by Sangharsh Lohakare on Unsplash
Background: Currently, there exists no universal legal requirement for gene synthesis providers to conduct background checks on clients or screen DNA sequences to ensure they're not dangerous pathogens. While some labs screen orders on a voluntary basis as a condition of membership in organizations like the International Gene Synthesis Consortium, compliance remains optional and inconsistent. In 2006, an investigative journalist with the Guardian was able to mail order a "modified sequence of smallpox DNA," and their order was not screened by the provider since it was less than 100 nucleotides long.
Even in the United States, no binding legal requirements exist for DNA synthesis screening. Federal regulations were proposed in 2024 through the Framework for Nucleic Acid Synthesis Screening, but an Executive Order in May 2025 paused implementation, and no replacement framework has been issued. Moreover, despite several congressional bills attempting to mandate screening, none have passed. This regulatory gap means anyone, including those with malicious intent, can order potentially dangerous genetic sequences with minimal oversight.
Notably, frontier lab leaders themselves agree that this is a problem: see this letter signed by the CEOs of OpenAI, Anthropic, Google DeepMind and Microsoft AI: ScreenDna.org
The Intervention: Existing screening approaches rely on sequence homology, which means matching orders against databases of known dangerous pathogens. An AI-enabled bioterrorist could circumvent this by designing functionally equivalent pathogens using synonymous codons, chimeric sequences, or entirely novel genetic constructs that retain lethality while evading database matches.
To address this issue, we need to implement advanced AI-powered screening that would analyze predicted protein function and evolutionary markers to flag potentially dangerous sequences.
Implementation requires two components: First, we must develop reliable AI screening systems capable of detecting novel pathogenic sequences that the world has never seen before; second, we must require all commercial DNA synthesis providers globally to implement this screening as a condition for legal operation.
Recommendations
Department of Health and Human Services / OSTP (immediate actions while waiting for legislation from Congress)
Congress
International coordination for synthesis screening should be pursued through the multilateral channels described in Section 1, including the BWC and Australia Group frameworks.
* Note: A stronger proposal (that, since writing my gap analysis of S.3741, I have updated to support) would treat benchtop nucleic-acid synthesizers as controlled items subject to custody tracking, similarly to fissile material. Possession would require a license, and a designated oversight agency would maintain a registry of each device's location and responsible custodian; when a licensed owner no longer needs a unit, it would be returned or transferred only to another licensed owner. For devices already in circulation, buyback programs may be a promising option.
Note: For a fuller treatment of why cloud labs could pose biorisk, see Part 3 on Bottlenecks.
Background: The standard objection to AI-driven bioweapons risk is that knowledge alone is not enough– you still need hands-on laboratory skills, the "tacit knowledge" that can only be acquired through years of physical practice (hence Active Site's uplift study). Knowing how to culture a pathogen is different from being able to do it reliably. This barrier has historically been one of the strongest defenses against non-state bioweapons development.
Unfortunately, cloud laboratories threaten to significantly erode this barrier. Services like Emerald Cloud Lab allow anyone to design experiments in software and have them executed by robotic systems in a physical facility, remotely, without ever entering a lab. ECL requires no coding experience; internal estimates suggest relatively short onboarding periods for novice users. An AI system that can design a bioweapons protocol and a cloud lab that can execute it are, individually, semi-manageable risks; together, they could materially increase the risk of misuse.
Source: Nature
Despite this, cloud labs currently operate with no standardized customer screening.
Contract research organizations (CROs) create a similar vulnerability. CROs provide specialized research services and can help their clients with everything from compound synthesis to biological assays. A malicious actor could potentially decompose a bioweapons development project into seemingly innocuous components and outsource them to different CROs, each unaware of the larger program.
The Intervention: Require all cloud laboratory providers and contract research organizations to implement know-your-customer screening before granting access to experiment execution as a condition of legal operation. Providers should be required to log all experimental workflows and flag protocols involving select agents or sequences of concern, with automated screening that mirrors (and integrates with) the DNA synthesis screening proposed in 4.2. Also establish RAND's proposed Cloud Lab Security Consortium modeled on the IGSC.
Recommendations
Department of Health and Human Services
NIST (National Institute of Standards and Technology)
Federal Funding Agencies (National Institutes of Health, National Science Foundation, DARPA)
Congress
International Coordination
Note: The KYC framework described here for cloud labs should also extend to DNA synthesis providers, which currently face no standardized customer screening requirements either.
Note: Open-weight model policy interventions are covered in uncensorable.ai
Background: In May 2025, Anthropic activated AI Safety Level 3 (ASL-3) protections for Claude Opus 4 after determining they could not rule out that the model might "significantly assist" the ability of individuals with basic STEM backgrounds to obtain, produce, or deploy chemical, biological, radiological, and nuclear (CBRN) weapons.
Anthropic's ASL-3 protections apply universally to all Claude Opus 4 users, meaning that a postdoctoral researcher at MIT working on cancer therapeutics faces the same restrictions as an anonymous user with no verifiable background; similar safeguards at OpenAI and DeepMind currently apply indiscriminately.
Tiered access controls offer a path to minimize this trade-off by enabling differentiated access based on factors such as verified identity and demonstrably legitimate use cases.
This approach mirrors established biosecurity frameworks: physical laboratories implement Biosafety Levels (BSL-1 through BSL-4) with increasingly stringent requirements. Tiered access controls apply the same principle to AI systems. The result is more precise risk management, as legitimate researchers gain the capabilities they need while malicious actors face substantially higher barriers.
Tiered access could enable AI labs to deploy more capable biological models than would be safe under universal restrictions, accelerating beneficial research without proportionally increasing misuse risk.
Note: Currently, OpenAI has a research preview of an advanced model for the life sciences, GPT-Rosalind, which requires approval to access. The intervention here seeks to standardize and scale tiered access controls.
The Intervention: For frontier LLMs, a model exceeding a defined bio-capability threshold (as determined by the standardized evaluations proposed in 4.1) must be deployed with tiered access controls.
Tiered access is most effective when paired with standardized monitoring of API usage patterns. Once a user has been verified and granted access, their usage should still be monitored for sequences of queries suggesting progression toward bioweapon development; flagged patterns should be escalated to human review and, when warranted, to law enforcement. This is because credentials can still be stolen, and as with cases like Bruce Ivins and Aum Shinrikyo, the expert-terrorist overlap is statistically rare but non-zero.
Recommendations
NIST / Center for AI Standards and Innovation (CAISI)
Department of Health and Human Services / OSTP
Department of Justice / FBI
Federal Funding Agencies (NIH, NSF, DARPA)
Congress
International Coordination
Background: Existing U.S. biosurveillance infrastructure is primarily built to detect known pathogens. Against a naturally occurring outbreak of a familiar pathogen, this approach functions reasonably well; against a novel engineered pathogen, current surveillance systems may fail to detect it entirely until clinical cases emerge.
This is problematic because, as discussed previously in Section 4.2, AI can already help adversaries design sequences specifically to exploit systems built on recognition. By the time symptomatic individuals are diagnosed, transmission may already be widespread.
In contrast to current methods, pathogen-agnostic metagenomic sequencing reads all genetic material present in a sample. Importantly, because it does not rely on comparing said genetic material to predefined targets, this approach can detect entirely novel pathogens.
The Intervention: Establish a national pathogen-agnostic metagenomic surveillance network. Use AI-powered bioinformatics (building from the function-based sequencing technology discussed in 4.2) to identify novel sequences and detect anomalous patterns for human review.
Recommendations
Centers for Disease Control and Prevention
Department of Homeland Security
Department of Health and Human Services
Congress
International Coordination
Defense
Note: Oddly enough, the U.S. has managed to amass a 1.4 billion-pound surplus of cheese, but has fallen far short on stockpiling PPE.
Source: Springfield Underground , Culture Cheese Mag
Background: The Strategic National Stockpile (SNS), managed by the Administration for Strategic Preparedness and Response (ASPR), is a federal reserve of medical countermeasures including pharmaceuticals, vaccines, medical devices, and personal protective equipment.
In theory, the SNS exists to supplement state and local supplies during emergencies too severe to handle with commercial supply chains. However, in practice, the SNS is plagued by inadequate supply and broken distribution logistics, making it fall short in times of crisis like the COVID pandemic.
In early March 2020, during the early days of the COVID outbreak, the Department of Health and Human Services stated that the SNS had only "1%... of the required respirator masks that would be needed for medical professionals if the COVID-19 outbreak were to erupt into a pandemic here."
By early April 2020, the stockpile's PPE had been "nearly depleted." The House Oversight Committee chairwoman at the time described a chaotic scene of "states… scour[ing] the open market for scarce supplies, often competing with each other and federal agencies in a chaotic bidding war that [drove] up prices."
In a post titled "Public Health Preparedness: HHS Should Address Strategic National Stockpile Coordination Challenges," the Government Accountability Office highlights the issue clearly:
"[D]uring recent public health responses, such as COVID-19 and mpox, jurisdictions weren't clear on how and from whom to request supplies, causing confusion and delays. Additionally, some Tribal officials cited challenges with having the facilities needed to receive and store delivered supplies."
The funding gap is the problem. This year, even after the lessons of the COVID pandemic, the SNS was reportedly "[left with] a shortfall of about $588 million" following the passage of the FY2026 Labor–HHS appropriations bill. To put that into perspective, the Department of Defense's budget for fiscal year 2026 is $839 billion; the SNS's shortfall could be filled with 0.07% of that.
The Intervention: Provide adequate, sustained funding for the SNS, with clear distribution protocols and state partnership. Establish clear distribution pathways so states know how to request supplies, and support state-level stockpiles as the first line of response, with the SNS as surge capacity.
Source: NY Post / Reuters
Recommendations
Congress
Administration for Strategic Preparedness and Response (ASPR)
State & Local Governments
1 "Elastomerics have a long shelf life and are more effective than, say, N95s. This option avoids a lot of the failure modes of maintaining N95 stockpiles and is [substantially] better against the worst tail-risk threats." –Lee Wall, AIxBiosecurity Research Manager at the ERA Fellowship
Note: This intervention may also improve students' health and academic performance, according to research cited by the EPA.
Background: Airborne pathogens are the hardest transmission route to defend against. As evidenced by COVID, influenza, and the common cold, these kinds of pathogens are difficult to control; if such a pathogen is released indoors, the rate at which it is diluted, filtered, and removed from a space directly determines how many people inhale an infectious dose. Buildings with poor ventilation often become amplifiers of transmission. In contrast, adequate ventilation and filtration can reduce risk substantially.
Currently, the Government Accountability Office states that "an estimated 41 percent of districts need to update or replace heating, ventilation, and air conditioning (HVAC) systems in at least half of their schools, representing about 36,000 schools nationwide that need HVAC updates;" the problem extends beyond schools to hospitals, transit systems, office buildings, and other high-occupancy public spaces.
Source: GAO analysis of school district survey data (GAO-20-494)
Cost
Upgrading costs depend on whether a building's existing system is strong enough to be equipped with better filtration. When it is, the upgrade is nearly free: swapping MERV-8 for MERV-13 filters runs about $1.50 a month for a 5,000-square-foot office, according to the Lancet Covid-19 Commission Task Force.
Where it can't (which, unfortunately, describes much of the aging school and public building systems), the cost to upgrade can become substantial: some estimates say it would cost roughly $500,000 to $5 million per school site, depending on the condition of the current system and the size of the school. (These projects often pull in electrical, roof, ceiling, and insulation work, which drives the total higher.)
For existing buildings where a full retrofit is economically infeasible, a building can still reach an equivalent target with some combination of the following three things: more outdoor air (which means opening windows or raising the HVAC's fresh-air intake), the highest-grade filter the existing system can run (MERV-13 where the equipment allows, which the EPA rates at ≥85% capture of 1-3 micron particles and ≥50% of the finest 0.3-1 micron, and the best compatible filter where it doesn't), and portable HEPA filters. These approaches can deliver equivalent outdoor air changes per hour (EOACH), providing a lower-cost pathway to reducing airborne infection risk.
A Note on Far-UVC
A newer air-cleaning option, far-UVC germicidal lighting, can also contribute to this target. A 2024 study from the Center for Radiological Research at Columbia University found that "far-UVC light inactivated nearly all (>99%) of an airborne virus in an occupied work environment." To quote the senior author of the study:
"If this virus had been a disease-causing virus, the far-UVC light would have provided far more protection against airborne-disease transmission than any ventilation system."
The technology is still emerging, however. Real-world evidence remains limited to a small number of settings, the long-term effects of chronic exposure are not yet well characterized, and far-UVC lamps can generate ozone and other reactive byproducts under some conditions, so deployments should monitor air quality and avoid small, poorly ventilated spaces. For now, far-UVC is best treated as a possible contributor to the clean-air target, rather than a stand-alone fix.
Source: Blueprint Biosecurity
The Intervention: Establish a federal minimum standard for indoor air quality in high-risk buildings (which should include healthcare facilities, schools, public transit, government buildings, and other high-occupancy spaces) and fund ventilation infrastructure upgrades to meet that standard. Also ensure that all new buildings are constructed with proper ventilation.
Recommendations
CDC
Department of Education
Department of Transportation / Transit Authorities
Congress
EPA
Source: FDA
Note: A complementary program worth funding alongside standing vaccine capacity is pathogen-agnostic countermeasures, which includes things like broadly protective nasal sprays and innate immunomodulators that work across whole families of respiratory viruses, including ones that don't exist yet. The UK's Advanced Research and Invention Agency is pursuing this through its £57m Sustained Viral Resilience program. Fascinatingly, funding on these kinds of countermeasures is never wasted; while a pathogen-specific stockpile can expire unused, a broadly protective MCM works against every future pandemic in addition to viruses like the flu and the common cold. The eventual goal here would be to eliminate respiratory illness altogether. I would have liked to write about this as its own intervention, but wasn't able to finish due to time constraints; I might write about this topic more at a later date, and update this site accordingly and/or publish on my Substack. A US effort here would most naturally live at the Advanced Research Projects Agency for Health.
Background: When a novel pathogen emerges, the clock starts immediately; every week before a vaccine exists is counted in infections and deaths. COVID showed both how fast vaccines can now move and how far short of "fast enough" we still are. The genetic sequence of SARS-CoV-2 was published in January 2020; a vaccine candidate was designed within two days, and Moderna was in human trials 66 days later.
Yet, the first shots didn't reach arms until December– 326 days from identifying the virus to the first emergency authorization.
That was a world record, shattering the prior best of nearly five years. It was also most of a year, and most of the first wave's deaths fell inside that window.
Importantly, research suggests even this record-breaking effort could have been substantially faster. A team of scientists from the Netherlands, in a paper titled "Upscaling vaccine manufacturing capacity - key bottlenecks and lessons learned," concluded the following after researching the vaccine supply chain extensively:
The COVID-19 pandemic put enormous pressure on the vaccine production chain as billions of vaccines had to be produced in the shortest timeframe possible. Vaccine production chains struggled to keep up with demand, resulting in disruptions and production delays… Key bottlenecks identified include a lack of manufacturing facilities, a lack of tech-transfer personnel, inefficient arrangement of production stakeholders, critical shortages in raw materials, and restricting protectionist measures.
Similarly, a report from the Government Accountability Office cited "[l]imited manufacturing capacity," "[d]isruptions to manufacturing supply chains," and "[g]aps in the available workforce" as challenges faced by vaccine companies that slowed development and deployment at scale.
Notably, none of the above reasons for delay are difficult scientific problems. Instead, they are infrastructure challenges, which we do not need breakthroughs to address.
Closing that gap is the point of the 100 Days Mission, endorsed by the G7 and G20, which aims to have a vaccine ready for initial authorization and manufacturing at scale within roughly 100 days of identifying a new threat– about a third of the COVID timeline.
Rather than novel scientific discoveries, achieving it requires only standing capacity, built and maintained before the next outbreak. This includes validated rapid-response platforms, idle-but-ready manufacturing lines, prototype vaccine libraries against high-risk pathogen families, and pre-positioned supply chains.
The Coalition for Epidemic Preparedness Innovations, writing about the 100 Days Mission, states:
More than eight million people who died during the COVID-19 pandemic might be alive today if the world had achieved the 100 Days Mission to develop safe and effective new vaccines against the novel [SARS-CoV-2] virus.
The US is currently moving in the opposite direction. In August 2025, HHS canceled roughly $500 million in BARDA contracts supporting mRNA vaccine development, months after also terminating "$766 million in Moderna contracts for vaccines for flu pandemics." The stated rationale, that mRNA "poses more risks than benefits" for respiratory viruses, is disputed by vaccine scientists and the available studies; a former BARDA director called it "self-inflicted vulnerability."
The effect, however the debate resolves, is to dismantle part of the standing capacity that delivered vaccines in record time.
After Operation Warp Speed delivered vaccines for roughly $18 billion2 against a pandemic that cost the US an estimated $16 trillion, the 2021 American Pandemic Preparedness Plan laid out a $65.3 billion program to make vaccines against any viral family within 100 days and create a permanent coordinating office. But the plan was largely never funded; the coordinating office Congress created in 2022 was left without dedicated money and has since collapsed.
The Intervention: Fund and sustain the standing vaccine-development and manufacturing infrastructure needed to achieve the 100 Days Mission.
Recommendations
Congress
BARDA
2 TIME analyzed Operation Warp Speed spending compared to other government programs: link
I'm happy to add to this section as people reach out with thoughts. Feel free to comment on my Substack.
"I don't think AI biorisk is a big deal, and I'm not sold that other emerging technologies are eroding barriers either. Making a bioweapon is still pretty hard."
The interventions in Part 4 are worth implementing regardless. COVID was a natural pandemic, and, to quote the Center for Global Development:
[Researchers] estimate the annual probability of a pandemic on the scale of COVID-19 in any given year to be between 2.5–3.3 percent, which means a 47–57 percent chance of another global pandemic as deadly as COVID in the next 25 years.
Even the next natural pandemic could be orders of magnitude worse than COVID; that alone justifies the policies outlined in Part 4.
But it's also worth remembering that it only takes one successful bioterrorist, or one lab accident. Even before the technologies outlined in Part 3 existed, some bioterrorists got scarily close. I don't think we should take that bet.
"Government bioweapons programs don't really matter. No country has ever released a widespread bioweapon, because of blowback and other strategic considerations."
Imperial Japan's Unit 731 did deploy bioweapons, so the premise is already not fully true. But even if we grant it for the sake of argument, deliberate release isn't the only risk. Accidents happen. Notably, the Soviet Union's Biopreparat had at least two: Aralsk-7 and Sverdlovsk, both covered in Part 2. And, the cost-benefit calculus that supposedly restrains states doesn't apply to apocalyptic groups like Aum Shinrikyo, who wanted mass casualties.
Also, I don't know about you, but I don't feel particularly comfortable betting the lives of millions on Vladimir Putin's restraint.
I've written a bit more on this here.
"Why are AI-designed pathogens of particular concern?"
AI could, in principle, design a pathogen that has never existed in nature. Why is this important? Two reasons:
I'm writing a separate piece for SecureBio on this topic – stay tuned!
"What should we do about open-weight models?"
My best guess, along with links to other pieces I've written on the topic, can be found at uncensorable.ai.
