TikTok has banned 20 accounts after the BBC highlighted the use of AI-generated black female influencers to drive users to sites promoting sexually explicit content.
They are part of a growing trend on Instagram and TikTok that has been criticised as racist, exploitative and misleading because of racial tropes and language used.
The BBC and researchers from the independent artificial intelligence (AI) publication Riddance found dozens of accounts on the two platforms featuring highly sexualised black women digital characters or avatars.
The images and videos were generated by AI but not labelled as such, in apparent breach of the platforms’ guidelines.
Nearly all the accounts were on Instagram and about a third also had versions on TikTok. Instagram’s parent company, Meta, told the BBC it was investigating, but did not say it had taken any action.
The avatars are often shown dressed in skimpy swimwear or other revealing clothing and portrayed with exaggerated body shapes.
Some have exceptionally dark skin tones that have been digitally manipulated, giving them an artificial appearance.
Account names include terms such as “black”, “noir”, “dark” and “ebony”.
Several include comments about white men in their posts, such as “loves white men” and “why I need a white guy in my life”. Many of the accounts follow or like each other.
The BBC, working in collaboration with analysts Jeremy Carrasco and Angel Nulani from Riddance, has identified 60 such accounts, mainly on Instagram, that have carried links, or chains of links, to paid-for sexually explicit content on third-party sites.
The sites labelled the imagery as AI-generated, but the Instagram accounts did not.
The research also identified many more accounts on both Instagram and TikTok with similar AI-generated avatars that did not link to paid content.
‘I WAS ANGRY’
One of the accounts shut down by TikTok – though still operating on Instagram at the time of publication – has caused further anger by stealing videos from real people.
It is presented as the account of a striking AI-generated character and had amassed three million followers within a few weeks of its creation in December.
But it has modified and posted videos from a genuine content creator, Riya Ulan, a model based in Malaysia.
The avatar’s face, which has an exceptionally dark, artificially created skin tone, is overlaid on to Ulan’s body, movements, clothing and backdrop are replicated.
“I was angry,” Ulan told the BBC. “Of course my videos are all out there… It doesn’t mean that you can just take it and steal it and post it as your own.”
One of the manipulated videos reached over 35 millions views on TikTok and 173 million on Instagram, about 47 times the views on Ulan’s original post.
While the three videos that clearly match Riya’s content are not sexual, other videos on the AI account using the same digitally created character show it in revealing clothing or performing provocative actions.
A chain of links from the account leads to paid-for adult content.
“I’m not sure if I’m more concerned about them taking my video to promote their explicit content or [that] people actually believe in that,” she says.
It is becoming harder for users to tell whether content is real and “people keep on falling for these AI models”, she adds.
Many viewers appear to treat the avatars as real, despite their unrealistic features.
In posts or Instagram stories, some of the accounts deny using AI, including the one that took Ulan’s content.
Ulan says she reported the account to both platforms multiple times but, at the time, the content was not removed. TikTok banned it after the BBC approached it for comment.
‘UNREALISTIC DEPICTIONS’
“I believe these accounts are racist because their existence perpetuates a long history of the exploitation of black people,” says Nulani.
“Their use of caricatures, race-play terminology and unrealistic depictions of black women prove they’re not concerned with our safety or well-being, but our ability to be capitalised as part of the online porn machine,” she adds.
Carrasco, who critiques AI trends and techniques on his social media accounts, says “the new thing is the quantity of shameless, racist depictions of extremely black people”.
While “that fetish” may have existed in the past, AI “gives it new purchase”, he says.
He explains that AI makes it easier to remove undertones in images and videos in order to create dark skin tones that are not natural, and to create effects that would previously have required animation or skin painting.
Also, he says, there are no social consequences for an avatar: “There’s no shame… that’s something AI uniquely exploits.”
Houda Fonone, a Moroccan model and content creator who advocates for more authentic representations of black women, says the trend is about “erasure”.
“Silky hair, extremely thin bodies and impossibly flawless skin… it’s as if black beauty can only be accepted when ‘refined’.”
She says this risks reinforcing stereotypes while “our stories and real-life experiences are replaced by an artificial image”.
The BBC sent Meta and TikTok examples of the accounts we identified, and asked for their responses.
Two days later, a TikTok spokesperson told the BBC the company had “removed content and banned accounts which breach our rules”.
Within a few days, 20 accounts had been labelled “banned” on the app.
“TikTok prohibits AI generated content of individuals used without their permission, we have zero tolerance for content which promotes off-platform sexual services,” the spokesperson said.
The company says it prohibits and removes AI-generated content that is harmful or misleading, and requires users to label realistic AI-generated content. It says it had also applied correct labelling to a number of videos.
Meta says it was investigating the content brought to its attention.
It says it wants users to know when they are looking at posts that have been made with AI, and that it has policies in place for the labelling of AI-generated content. Nine Instagram accounts the BBC has tracked appear to no longer exist.
Neither platform gave details about its response to Ulan’s initial reports.
– BBC
In an age of information overload, Sunrise is The Namibian’s morning briefing, delivered at 6h00 from Monday to Friday. It offers a curated rundown of the most important stories from the past 24 hours – occasionally with a light, witty touch. It’s an essential way to stay informed. Subscribe and join our newsletter community.
The Namibian uses AI tools to assist with improved quality, accuracy and efficiency, while maintaining editorial oversight and journalistic integrity.
Stay informed with The Namibian – your source for credible journalism. Get in-depth reporting and opinions for
only N$85 a month. Invest in journalism, invest in democracy –
Subscribe Now!
