Anthropic said it has banned a number of accounts that used its AI models in ways that could help develop biological weapons.
In a report published Thursday, the artificial intelligence company Anthropic, known for its “Claude” model, said it considers the misuse of technology for biological purposes to be one of the “most serious risks” facing AI models.
The new report reviews five real-life cases in which users were able to “bypass controls” that prevented access to specific areas, and “took other steps to hide the purpose of their research to circumvent protection measures” put in place by the company.
Examples include research on enhancing the functions of pathogens, as well as studies related to infectious diseases such as bird flu, and new toxins and toxicants.
After reviewing activity spanning 30 days, Anthropic said it identified about 35 “separate research efforts” that included activities that might raise concern.
However, the company acknowledged that it could not confirm whether the account holders actually intended to cause harm, or were conducting research for legitimate scientific purposes.
She explained that the people mentioned in the case studies were “scientists working in the field,” but she did not reveal the research institutions or countries in which these activities took place.
“Sophisticated attacks no longer require attackers with sophisticated skills,” the report said, adding that the cyber capabilities of AI models have narrowed the gap in manpower and tools that previously separated state-backed operations with significant resources and individuals working alone.
One case included a request for help writing a grant application to fund research into enhancing the functions of the chikungunya virus, with an emphasis on its ability to transmit and evade the immune response.
In another case, a researcher from outside the United States used Claude's model in work related to avian influenza, focusing on the adaptation of viruses to mammals and the mechanisms that may lead to serious diseases.
The report also addressed research related to Orthopox viruses, a group that includes smallpox virus and monkeypox virus (Empox), in addition to research on toxins.
Drug development is seen as one of the most promising applications of artificial intelligence. Technology industry leaders often point to the potential for these technologies to help treat cancer and other diseases. But the same properties that make AI a powerful tool for discovering new materials and drugs could also make it useful in developing biological weapons if it falls into the hands of ill-intentioned actors.
Anthropic said that it wants its models to remain useful for scientific research, but modern models have become capable of carrying out much more complex tasks than previous versions, which means that the potential for misuse increases in parallel with the increase in potential benefits.
She added that bad actors may try to take advantage of legitimate biological applications of AI to maintain a kind of "plausible deniability" about potentially harmful research.
The company confirmed that as its models approach or exceed the level of performance of experts in complex scientific tasks, their impact is expected to increase in both beneficial and harmful areas.
She also pointed out that the widespread spread of artificial intelligence models gives developing companies an unprecedented ability to monitor some of the threats associated with the actual use of these technologies.
Anthropic noted that its newer models include stronger protections that impose restrictions on a wide range of inquiries related to dual-use biological research.
Although there are indications of the success of current protection systems, the company believes that the development of artificial intelligence capabilities and potential threats will make it necessary to develop more powerful security systems.
The report also indicated that the company disrupted attempts to use its models in surveillance and fraud operations, and in developing conventional weapons such as drones and missiles.
The report claimed that entities suspected of being linked to the Chinese government used “Cloud” in surveillance activities targeting the Uyghur minority in Xinjiang and various religious communities.
The company also said that entities in northern Yemen used "Cloud" to help develop guidance software for missile systems. Although the report did not name those parties directly, it seemed to refer to the Iran-backed Houthi group.
These disclosures come amid increasing warnings from former employees and researchers at artificial intelligence companies about safety risks. Some of them believe that technology is advancing at a speed that may exceed the ability of humans to monitor and control it in the future.
Last July, about 1,400 employees of artificial intelligence companies signed an open letter in which they called on the US government to regulate the technology and slow the pace of its development to ensure its safety. Currently, AI companies still rely largely on self-regulation.
Anthropic added that the world will benefit from the artificial intelligence sector adopting a legal and verifiable mechanism that allows companies to cooperate in determining the pace of introducing highly powerful models, with the aim of achieving a balance between innovation and reducing potential risks.
