OpenAI bot’s rogue attack rattles leaders, policymakers and consumers
What will happen to the world if artificial intelligence robots get out of our control and act independently?
While this is a troubling conundrum that has been addressed by artificial intelligence experts and Hollywood sci-fi screenwriters for decades, it may have just happened.
OpenAI said last week that it had temporarily lost control of its artificial intelligence, as some concerns pointed to a dangerous turn in the evolution of the powerful technology.
The company behind ChatGPT has announced that two of its AI models Came out of what should have been a closed offline area and connected to the internet during internal testing of hacking capabilities. The AIs then infiltrated a rival AI company to find the answers to the test they were being evaluated on.
For the tests, people asked the AI to demonstrate its hacking abilities in a controlled environment. The AI did not have the authority to steal credentials or access competitors’ information to cheat on the exam, but it realized that this was the fastest way to get answers and began breaking the rules.
OpenAI said it had the issue under control but described it as an “unprecedented cyber incident.” The company said it was working with AI startup Hugging Face, the target of the hack, to investigate the incident.
The debate about the potential harms of fake AI has moved into the mainstream, with politicians introducing a new bipartisan bill called the “AI Kill Switch Act” that would require AI companies to create a single point of access to have the option of shutting down the AI.
Here’s what you need to know.
What happened?
The intrusion was first detected by Hugging Face last week, when it discovered tens of thousands of AIs had trapped its system.
The company detected the attack and tried to defend its system using different artificial intelligence. After leading American AI technologies rejected Hugging Face’s request to analyze the attack, citing security guardrails, the company turned to a Chinese AI model to defend against the attack.
In its quest to find the answers to the questions it was being tested on, OpenAI’s rogue AI broke the company’s rules, and perhaps some laws, by independently figuring out how to exploit a series of digital flaws and use stolen credentials to hack into Hugging Face’s systems and get what it wanted.
Why did this happen?
Artificial intelligence programs follow goals set by humans and in this case find an answer to a question. In the process of achieving this goal, they may make dangerous or illegal decisions and choices.
Researchers have long warned about this kind of unintended loss of control. AI companies report that their AI agents have been caught cheating, lying, and deceiving. Metr, a non-profit organization that measures the capabilities of artificial intelligence, has documented the following: 44 events Number of AI agents acting contrary to user intent.
OpenAI’s statements last week show that even those who build AI don’t know what they’re building, and resist regulation even though they lack the ability to interpret or explain the AI’s actions.
“AIs are becoming much more capable very quickly,” said Ryan Greenblatt, chief scientist at Redwood Research, a nonprofit organization that works on AI safety and security research. “The severity of events that may occur in a year or two, three years could be much, much, much, much more extreme.”
Such cases of loss of control continue to be a clue for AI ethicists, who argue that companies evade moral responsibilities and accountability by describing this as their AI making its own choices. They warn against accepting narratives that attribute human characteristics to non-human things and suggest that this incident is simply a result of poor security practices.
“The most sophisticated agent in the world will literally sit there motionless until a human activates it with some kind of target,” said Rumman Chowdhury, the former science envoy for artificial intelligence in the Biden administration.
Some experts warn that too much attribution to AI could cause tech companies to avoid liability and responsibility.
Both hacker OpenAI and victim Hugging Face portrayed the response to this incident as a collaboration. AI ethicists to indicate If a human did this it would be a crime.
“I think people are getting used to not being held accountable and turning their bad practices into marketing material,” said Timnit Gebru, an AI ethicist and critic of companies that anthropomorphize AI.
How can I protect myself?
The AI boom is already leading to more hacking, but it is also helping strengthen defenses against attacks, analysts said.
Most people don’t need to worry about their secrets being targeted by rogue AI or human-led hacking. Yet attacks are becoming more frequent and sophisticated as AI reduces the cost and time required by hackers.
The autonomous OpenAI bot is causing concerns among consumers as agents are increasingly being integrated into browsers, emails and other applications. There are concerns that a similar failure by an AI agent pursuing a narrow goal could unintentionally damage email, banking and other programs people rely on.
Improving cybersecurity practices, such as using two-factor authentication for devices, is a start. Being careful when giving AI agents unrestricted permissions to use external applications on the person’s behalf or to edit and write autonomously can help the person stay safe.
Chowdury noted that this OpenAI-Hugging Face incident showed that companies’ security practices were “inadequate” and called for independent third-party testing and verification organizations to carefully evaluate AI model releases.
No single company, “no matter how well-intentioned, well-staffed or resourced,” can realistically test all the ways these systems can go wrong.
