The fabricated images of the globally renowned pop star made the rounds on Wednesday, amassing tens of millions of views before being taken down, according to CNN reports.
Similar to most social media platforms, X has policies prohibiting the dissemination of "synthetic, manipulated, or out-of-context media that may deceive or confuse people and lead to harm."
Without explicitly mentioning Taylor Swift, X released a statement: "Our teams are actively removing all identified images and taking appropriate actions against the accounts responsible for posting them." A report from 404 Media suggested that the images may have originated in a Telegram group, where users share explicit AI-generated images, often created with Microsoft Designer. The group's users allegedly joked about the images of Swift going viral on X.
During this time, the term "Taylor Swift AI" trended on the platform, further promoting the images and drawing more attention. Swift's fans countered by flooding the platform with positive messages about Swift, using related keywords. The sentence "Protect Taylor Swift" also trended. While Swifties expressed frustration at X's slow response, the incident sparked a broader conversation about the proliferation of non-consensual, computer-generated images of real people.
Oren Etzioni, a computer science professor at the University of Washington specializing in deepfake detection, stated, "It's always been a dark undercurrent of the internet, nonconsensual pornography of various sorts. Now it's a new strain of it that's particularly noxious." He predicts a surge in AI-generated explicit images, considering the success perceived by those who created the images.
Carrie Goldberg, a lawyer representing victims of deepfakes, emphasized that social media platform rules are insufficient, urging companies to prevent such content from being posted in the first place. She suggested that AI on these platforms could identify and remove such images, emphasizing that technology can be part of the solution.
However, X faces challenges in detecting fake and damaging imagery and misinformation. Elon Musk's decisions in 2022, including loosening content rules and reducing the moderation team, have been criticized for allowing problematic content to flourish.
Ben Decker of Memetica, a digital investigations agency, commented on the situation, saying that targeting figures like Taylor Swift could bring the conversation about AI deepfakes to the forefront. He suggested that the involvement of prominent figures might prompt action from legislators and tech companies.
The issue extends beyond celebrities, as everyday people also fall victim to this form of insidious misinformation. In December, Canada's cybersecurity watchdog warned about AI-generated images and videos undermining faith in democracy during elections.
In a new report, the Communications Security Establishment (CSE) highlighted the threat of political deepfakes, making it harder for people to trust online information about politicians or elections. CSE chief Caroline Xavier stated, "Despite the potential creative benefits of generative AI, its ability to pollute the information ecosystem with disinformation threatens democratic processes worldwide." The agency warned that cyber threats during elections are more likely to happen in the future.
