Summary In a report published Thursday, the company behind Claude identified biological misuse as one of the “most serious risks” associated with increasingly capable AI systems.
Anthropic says it has blocked multiple accounts after detecting attempts to use its artificial intelligence models for research that could potentially support the development of biological weapons.
In a report published Thursday, the company behind Claude identified biological misuse as one of the “most serious risks” associated with increasingly capable AI systems.
Anthropic said it examined 30 days of activity and identified about 35 “distinct research efforts” involving potentially concerning biological work. However, the company acknowledged that it could not determine whether the users intended to cause harm or were conducting legitimate scientific research.
The report details five case studies involving researchers who allegedly attempted to circumvent Anthropic’s safeguards, including by accessing the service from restricted regions or disguising the purpose of their research.
The cases included potentially dangerous gain-of-function research, infectious diseases such as bird flu, orthopoxviruses and research involving novel venoms and toxins.
In one case, a user sought assistance preparing a grant application for gain-of-function research examining the transmissibility and immune evasion of chikungunya virus. Another researcher outside the US used Claude while conducting work on bird flu, including research into how viruses adapt to mammals and cause severe disease.
Anthropic said the individuals involved were working scientists but did not disclose their institutions or countries.
“Sophisticated attacks no longer require sophisticated attackers,” the report said, warning that increasingly capable AI models could reduce the expertise and resources traditionally required for dangerous research.
AI-assisted drug discovery is widely viewed as one of the technology’s most promising applications, with researchers exploring its potential to accelerate treatments for serious diseases. However, Anthropic warned that the same capabilities could potentially be exploited for harmful biological research.
The company said its latest models, including Claude Fable 5, have stronger safeguards designed to restrict access to a broad range of dual-use biological research requests.
Anthropic also said it had disrupted attempts to use its systems for surveillance operations, scams and the development of conventional weapons, including drones and missiles.
The disclosure comes amid growing concerns among AI researchers about the rapid development of increasingly powerful systems. Former Anthropic researcher Jacob Coxon recently warned that future AI could potentially be used to attack critical infrastructure or create “extinction-level bioweapons”.
In July, nearly 1,400 AI industry employees signed an open letter calling for stronger government regulation and a slower pace of AI development. OpenAI Chief Scientist Jakub Pachocki also recently warned that AI capabilities are advancing faster than researchers’ ability to reliably monitor and control them.
Anthropic said the industry needs a “lawful, verifiable way” to work together on controlling the release of increasingly powerful AI models.
