Anthropic limits rollout of Mythos AI model over cyberattack fears

Anthropic on Tuesday announced an advanced artificial intelligence model that will be available to a select group of companies as part of a new cybersecurity initiative called Project Glasswing.

The company said the Claude Mythos Preview model was successful in identifying weaknesses and security flaws in the software, and Anthropic limited access to prevent bad actors from abusing this capability.

Antropik said Apple, Google, Microsoft, Nvidia And Amazon Web Services They are among the first launch partners of the project and will be able to use the model for defense security studies. More than 40 other companies included CrowdStrike And Palo Alto NetworksAnthropic said , also participated.

“There was a lot of internal deliberation,” Dianne Penn, Anthropic’s head of research product management, told CNBC in an interview. “We really see this as a first step to give many cyber defenders the opportunity to get a head start on what is going to be an increasingly important issue.”

Anthropic’s announcement came after the model’s statements. Discovered by Fortune in a publicly accessible data cache late last month. Cyber ​​security stocks fell in the report, which stated that the model has advanced cyber capabilities and also poses a significant risk.

The iShares Cybersecurity ETF was mostly flat in intraday trading Tuesday.

“The dangers of getting this wrong are clear, but if we get it right, there is a real opportunity to create a fundamentally safer internet and world than we had before the advent of AI-enabled cyber capabilities,” the CEO said. Dario Amodei He wrote a post on X touting the release of Project Glasswing.

Anthropic was founded in 2021 by a group of researchers and executives who left OpenAI due to concerns about its direction and attitude towards security.

The company has spent years carefully building its reputation as a firm more committed to responsible AI deployment, announcing Project Glasswing just weeks after a high-profile clash over security with the Department of Defense became public knowledge.

Anthropic said it is in “ongoing discussions” with US government officials about the Claude Mythos Preview’s cyber capabilities. This includes discussions with the Cybersecurity and Infrastructure Security Agency and the Center for Artificial Intelligence Standards and Innovation, among others, according to an Anthropic official.

Penn said Anthropic employees decided on the name Project Glasswing, a metaphor that likens a transparent butterfly to “relatively invisible” software vulnerabilities.

Anthropic said Claude Mythos Preview was able to find bugs, some critical, that were previously difficult to detect. In one case, the company said the model detected a 27-year-old bug in OpenBSD. websiteIt is an operating system that prioritizes security.

Anthropic said Claude Mythos Preview is a general-purpose model that has not been specifically trained for cybersecurity, and its advanced cyber capabilities are a result of its strong coding and reasoning skills. The company said it doesn’t plan to make the model generally available, but its goal is to eventually learn how to distribute Mythos-class models at scale.

The companies participating in Project Glasswing are all building or maintaining critical software infrastructure, and Anthropic said they will use their model to try to secure both first-party and open source systems. Anthropic said it has committed to up to $100 million in usage credits for these efforts, but partners will pay to use the model beyond that threshold.

Newton Cheng, Anthropic’s Frontier Red Team cyber lead, said the company wants firms to get used to leveraging these capabilities before they become widely available. The company said it was trying to avoid “carelessly or irresponsibly” launching a model that could be exploited by competitors.

“Cybersecurity is going to be an area where this broad increase in capabilities carries the potential for risk, and so we need to keep a really close eye on what’s happening there,” Cheng said in an interview.

WRISTWATCH: Big Technology’s Alex Kantrowitz thinks the panic in cyber stocks is overblown