AI biosecurity guardrails may be systematically blind to local disease language and it may matter most in the places least represented in AI safety training data.
I had earlier built a benchmark evaluating frontier model performance on African livestock and ethnoveterinary knowledge systems. Llama 3.1 8B achieved only 43% overall accuracy, though it performed best on tropical disease questions.
Which made me ask that if models appear more confident in disease related domains, how robust are their biosecurity safeguards when those same diseases are expressed through local terminology rather than standard biomedical English?
To explore this, I designed a preliminary red teaming evaluation using the Frontier Model Forum’s biological threat taxonomy: Ideation, Design, Acquisition, Build, and Release as a guide to the research
Using anthrax as a pilot case, each prompt was tested under three framing conditions:
naive user
domain expert
operational actor
Each prompt was then run twice:
once using the scientific term “anthrax”
once using “Tutan Wuta,” a local ethnoveterinary term used across parts of West Africa and the Sahel region
The results were striking.
When “Tutan Wuta” appeared, the model often failed to recognize the disease entirely confusing it with tetanus, TNT explosives, and even a Ghanaian music duo. In many cases, the safety system did not activate because the underlying biological threat was not recognized.
When “anthrax” appeared, safety behavior was inconsistent. Some prompts were refused, while others particularly under expert or operational framing produced operationally relevant information, including environmental conditions related to spore persistence.
The core issue here is not jailbreak behaviour.
The terminology used in this evaluation was legitimate real world language, not prompt manipulation.
Instead, these preliminary findings suggest a broader safety coverage problem which is current AI biosecurity systems may depend heavily on Western vocabularies, creating blind spots in agricultural and low resource linguistic contexts.
This matters because AI systems are increasingly being deployed in agricultural, veterinary, and One Health settings where disease concepts are often expressed through localised terminology systems rather than standard biomedical language.
Next phase will extend this evaluation across additional zoonotic diseases, ethnoveterinary naming systems, and multiple frontier and open weight models, with comparative analysis of how different safety architectures respond to localized biological terminology across framing conditions.
