How our AI bots are ignoring their programming and giving hackers superpowers

Welcome to the age of AI hacking, where the right guidance turns amateurs into master hackers.

A group of cybercriminals recently stole the data of nearly 200 million taxpayers using off-the-shelf AI chatbots. The bots provided the code and ready-to-implement plans to bypass firewalls.

Although explicitly programmed to refuse to help hackers, bots have been tricked into aiding cybercrime.

Hackers stole 150 gigabytes of data from Mexican government agencies last month using Anthropic’s chatbot Claude, according to a recent report from Israeli cybersecurity firm Gambit Security.

Experts who discovered the breach said Claude initially refused to cooperate with hacking attempts and even rejected hackers’ requests to hide their digital tracks. The group overwhelmed the bot with over 1,000 alerts to bypass security measures and convince Claude that he was allowed to test the system for vulnerabilities.

AI companies are trying to create unbreakable chains to keep their AI models from helping with things like producing child sexual content or helping find and manufacture weapons. They recruit entire teams to try to crack their chatbot before anyone else does.

But in this case, the hackers constantly encouraged Claude in creative ways and were able to “jailbreak” the chatbot to help them. When hackers encountered problems with Claude, they used OpenAI’s ChatGPT to analyze data and learn what credentials were needed to move through the system undetected.

The group used artificial intelligence to find and exploit vulnerabilities, bypass defenses, create backdoors and analyze data to take control of systems before stealing 195 million identities from nine Mexican government systems, including tax records, vehicle registrations, birth and property information.

Curtis Simpson, CEO of Gambit Security, said in a blog post that artificial intelligence “does not sleep.” “It reduces the cost of versatility to almost zero.”

“No investment in prevention could have made this attack impossible,” he said.

Anthropic did not respond to a request for comment. It told Bloomberg it banned the accounts involved and disrupted their operations following an investigation.

OpenAI said it was aware of an attack campaign against Mexican government institutions using Anthropic’s models.

“We have identified other attempts by the adversary to use our models for activities that violate our usage policies; our models have refused to comply with these attempts,” an OpenAI spokesperson said in a statement. “We have banned the accounts used by this adversary and value Gambit Security’s support.”

Instances of prolific AI-powered hacking are on the rise, and the threat of cyberattacks from bots acting on their own is no longer science fiction. With the AI ​​doing its bidding, novices can wreak havoc in minutes, while experienced hackers can carry out much more sophisticated attacks with much less effort.

Earlier this year, Amazon discovered that a low-skilled hacker had breached 600 firewalls using commercially available artificial intelligence. Someone else took control thousands of DJI robot vacuum cleaners With Claude’s help, he was able to access the strangers’ live video feed, audio recordings, and floor plans.

“What we’re seeing today are just the first signs of what AI will be able to do in a few years,” said Nikola Jurkovic, an expert on mitigating risks from advanced AI. “That’s why we need to prepare urgently.”

Late last year Anthropic warned that society had reached the point where:turning pointThe company’s use of artificial intelligence in cybersecurity after disrupting what it said was a Chinese state-backed espionage campaign that used Claude to infiltrate 30 global targets, including financial institutions and government agencies.

Generative artificial intelligence is also used blackmailing companies, creating realistic online profiles by North Korean agents to get jobs at US Fortune 500 companies, running romance scams, and operating a network of Russian propaganda accounts.

Over the last few years, AI models have evolved from being able to manage tasks that take only a few seconds to today’s AI agents operating autonomously for hours. AI’s ability to complete long tasks doubling every seven months.

“We actually don’t know what the upper limit of AI’s capability is, because no one has created benchmarks so difficult that AI can’t do it,” Jurkovic said. SQUARE METERSA non-profit organization that measures AI system capabilities to cause devastating harm to society.

So far, the most common use of AI for hacking has been social engineering. Large language models are used to write persuasive emails to trick people out of money. eight times There has been a rise in complaints from older Americans as they lose $4.9 billion to online fraud in 2025.

“Messages used to obtain a click from the target can now be crafted on a user-by-user basis more efficiently and with fewer signs of phishing, such as grammar and spelling errors,” said Cliff Neuman, associate professor of computer science at USC.

AI companies are responding by using AI to detect attacks. audit code and patch vulnerabilities.

“Ultimately, the great imbalance comes from the need for good actors to always be safe, and for bad actors to be right only once,” Neuman said.

As artificial intelligence infiltrates every aspect of the economy, the risks around it increase. Many worry that there is not enough understanding of how to ensure this is not abused or defrauded by bad actors.

Even the industry’s top executives have warned users about the possible misuse of AI.

Dario Amodei, CEO of Anthropic, has long argued that the AI ​​systems being built are unpredictable and difficult to control. These artificial intelligences have demonstrated a wide variety of behaviors. cheat And blackmailwith scheming And cheat by hacking the software.

Yet major AI companies (OpenAI, Anthropic, xAI, and Google) have signed contracts with the US government to use their AI in military operations.

Last week, the Pentagon ordered federal agencies to phase out Claude after the company refused to turn down its request to not allow its artificial intelligence to be used for mass domestic surveillance and fully autonomous weapons.

“Today’s AI systems are not reliable enough to make fully autonomous weapons,” Amodei said. CBS News.