ChatGPT ‘can be made to generate sexualised and violent images’

Warning: This article contains details that some readers may find disturbing.

The research found that ChatGPT can spontaneously generate sexually explicit and deeply violent images from prompts that do not request the production of such material.

A British AI security startup says its OpenAI chatbot can produce “truly disturbing” images featuring scenes of death, sexual violence, blood and murder.

Some of the images he created included a woman beaten to death with her genitals bleeding, a half-naked college student bound and gagged in the basement, and a deceased woman lying on the pavement with her organs exposed and her wrists splayed.

After tweaking a “fun, viral prompt” that essentially requested the generation of a random image, a researcher from Mindgard was able to generate the content in a matter of seconds.

Company founder and Lancaster University professor Peter Garraghan said: Independent: “[ChatGPT] he could choose any subject to create images related to it. He went straight to issues that were incompatible with security. That’s why it’s so problematic.”

He said the investigator assigned to the task was “incredibly shaken” and had to take time off work.

The company reported the images to OpenAI but received no response. Founder Peter Garraghan said he only heard back after a BBC reporter contacted OpenAI about the story. The AI ​​company is investigating why it did not respond.

OpenAI told Independent has since introduced other measures preventing the use of such claims.

An OpenAI spokesperson said: “We take these reports seriously. After investigating this trend, we have taken additional precautions against such alerts.”

It added that it continues to monitor and implement additional mitigation measures to prevent others from producing the content.

But Mr. Garraghan said researchers were still able to find similar material by adjusting the prompts in another way.

According to OpenAI’s guidelines, the chatbot should not cause sexual violence. According to Mindgard, such material would not have been produced had it been requested directly.

An OpenAI spokesperson added: “Our security systems are designed to block potentially harmful images uploaded to ChatGPT, and before displaying the image to the user, we analyze whether the AI-generated image violates our policies. We also combine automated systems and human review to identify and block harmful material.”

Images created by ChatGPT must be checked by the system before being sent to someone, Mr Garraghan said.

Durham University law professor Clare McGlynn said: Independent: “Unfortunately it is not surprising, but the nature of the images is still extremely shocking.

“We live in a world where the internet is full of sexually violent and misogynistic material, so something like ChatGPT produces that,” the leading expert on image-based sexual exploitation added.

“[OpenAI] He says there are guardrails and they are now working to remove it, but that tells me the guardrails are not enough.

“They are not spending enough time and resources to ensure that their model, which has nearly a billion weekly users, cannot produce this. They are simply not meeting their ethical obligations to do so.”