The AI avatars rallying Australians to their cause
He looks like the city’s professional in every way; a sharp coat, a confident tone of voice, a bright background.
But the woman who called for anti-immigration marches and rallies to end the “invasion” of Australia this weekend has barely stepped out of the CBD meeting room.
He is a digital creation of a few words fed into an AI video creator.
AAP FactCheck research finds similar AI-generated avatar videos have spread online, making claims of a “migrant invasion” and election rigging, sometimes with violent overtones.
In one video, a gunman stands in front of a cheering crowd and warns of a “civil uprising” coming from “hard-working Australians”.
AAP’s FactCheckers analyzed content on two main accounts: LoveMyAu primarily features young, professional-looking female AI avatars on TikTok, X and Facebook; SaltyHypsi, on the other hand, showcases young Anglo-Australian men on TikTok, often dressed in military fatigues.
Many of LoveMyAu’s videos promote the anti-immigration March for Australia events, the most recent of which was on Sunday.
In one of the videos, a woman in a blue dress looks at the camera and says, “We must resist, or we will perish.”
“Invasions were never welcome,” says another.
“Invaders cannot move in and take over unchallenged countries.”
Another video, viewed more than 70,000 times, shows a line of smartly dressed women strolling along the boardwalk and chanting remarks reminiscent of a political broadcast.
“I believe the government is using mass immigration to manipulate the election results,” says one of them.
Other videos from the same account use similar avatars, accusing the government of throwing “our own people” into the streets to host immigrants and urging Australians to “fight, fight, fight” or lose everything.
Although the avatars are realistic, they bear the familiar characteristics of artificial intelligence; unnaturally smooth skin, inconsistent lighting and shadows, and slightly off-kilter lip sync. Some clips still show the Veo watermark that identifies Google’s AI video creator.
The SaltyHypsi account features avatars of soldiers, men speaking at rallies, and figures encouraging viewers to sign petitions against tobacco and alcohol taxes with a digital ID.
In a video viewed more than 60,000 times, a man wearing a bulletproof vest is seen holding a gun in front of crowds waving Australian flags.
“Our Prime Minister refuses to take Australian citizens seriously,” he says.
“A civil rebellion is what comes from hard-working Australians.”
In another photo, a man standing in the Australian bush says: “Australia, the time to march is over and taking action may be our last chance to protect Australia and our children’s future.”
AAP FactCheck asked both accounts a series of questions but received no response.
Such videos can be created in seconds by typing a prompt into one of numerous text-to-video AI generators.
Dr D., a digital sociologist at the ANU School of Cybernetics. Jessamy Perriam says the emergence of avatars is not a coincidence and that they are probably designed to be an improved version of the people they want to impress.
“They create avatars to look like desirable, acceptable Australians for others who don’t fit that description,” says Dr Perriam.
AI also makes users’ messages delivered by the “six o’clock TV news face” while maintaining the anonymity of those behind it, making it appear more socially acceptable, he adds.
“Would there be as much engagement if these videos were just a video of the person who created the video sitting in front of a webcam and saying the same words?”
Bill Browne, director of the Australia Institute specializing in digital technology and political advertising, says comments under many videos show some viewers are confused about whether the people depicted are real.
“It’s disturbing how easy it is to make convincing depictions of something that never happened and someone who never existed,” he says.
“I don’t think politicians, regulators, or the public yet fully appreciate the implications of AI or large language models.”
University of Sydney academic Fiona Martin researched the use of artificial intelligence and deepfake and said that the use of such avatars will become widespread rapidly in the coming years.
He sees the purpose of such content as an attempt to heighten emotional responses, confirm prejudices, and encourage people to include others in the cause.
“The problem is that even if we regulate against AI political figures in Australia, you still see hostile state actors conducting operations in countries that would never regulate that activity,” he says.
