Meta is facing new criticism after researchers found thousands of AI-powered “nudify” advertisements running across Facebook and Instagram despite the company’s policies banning sexually exploitative content.
According to a report from the Tech Transparency Project (TTP), a Chinese advertising partner called GatherOne placed more than 7,500 ads promoting services that digitally remove clothing from images of women. Researchers said one promoted app had previously been linked to child sexual abuse material, yet users could still access it through Meta’s platforms.
The findings add to growing scrutiny over whether Meta’s AI moderation and advertising systems can keep pace with increasingly sophisticated generative AI abuse.
“While Meta has removed a large number of the network’s ads, it appears to have made an intentional choice to continue to do business with GatherOne,” said the TTP in a blog post outlining its findings. “This may reflect the company’s desire to keep tapping into the massive Chinese advertising market, regardless of the impact on users.”
Although exact figures are impossible to determine, TTP reported that GatherOne may be spending upwards of $70 million per year on Meta ads, making it a prominent advertising partner.
TTP has focused several of its recent campaigns on the failure of major advertising networks to tackle sexual and illegal ads on their platforms. Apple and Google have also been found to have failed significantly in policing their own networks, with thousands of ads slipping through automated testing systems and remaining live even after reports and user feedback. Advertising partners that routinely flout the rules also appear to avoid bans, particularly when they operate larger accounts.
GatherOne told TTP that it had halted the creation of new advertising accounts for services relating to nudify and face-swap tools, although it did not say it would stop campaigns that were already running.
Meanwhile, in a statement to Bloomberg News, Meta said the company said it takes “aggressive steps” against nudify apps. Meta added that advertisers who violate its policies may face ad removals, financial penalties, and the suspension or termination of their advertising accounts.
Meta already in hot water over Muse Image
Meta has suffered several PR nightmares relating to AI, both through its own internal models and the activities of its advertising partners. It was forced to pull a feature from Muse Image just days after launch that allowed users to reference public figures and place their faces on other bodies or into prompted scenarios.
The Screen Actors Guild of America and other organisations pushed back against the feature being automatically enabled for public Instagram and Facebook accounts. Meta dropped the feature shortly afterwards, potentially in response to mounting legal pressure.
It was not the first time Meta had created AI systems featuring celebrities without permission. Last year, its chatbot allowed users to chat with AI characters resembling celebrities such as Taylor Swift, Anne Hathaway, and Scarlett Johansson. These chatbots could be given flirtatious personalities, and could create scantily clad images of the celebrities they were imitating.
Must-read security coverage
- UK Police Convicts Pair in £5.5 Billion Bitcoin Launder Case
- Blackpoint Cyber vs. Arctic Wolf: Which MDR Solution is Right for You?
- How GitHub Is Securing the Software Supply Chain
- 8 Best Enterprise Password Managers
May lead to stricter bans in the future
It seems unlikely that Meta will stop testing the limits of AI involving famous people and measuring how users respond. It just launched an AI mode that pulls from public posts, public groups, and Reels. The company is ultimately competing with several major rivals for AI adoption, and Meta and CEO Mark Zuckerberg have staked heavily on AI becoming the next major computing platform.
However, its apparent lack of care around AI deployment and social media more broadly may push regulators and other officials towards further restrictions or outright bans for selected groups of users.
Meta has already faced several lawsuits over allegedly addictive systems targeting children, including one in New York and another that was fortunately for Meta dropped a week before trial. The European Union is reportedly considering a similar investigation.
Australia, Malaysia, and France have already introduced social media bans for children, with France setting the threshold at under 15 and the other two countries at under 16. Turkey, Spain, and the United Kingdom have passed legislation, while five other countries are currently preparing similar measures. The EU may also introduce its own social media ban for children, potentially removing a significant number of Instagram and Facebook accounts.
Whether Meta changes its advertising enforcement remains to be seen. However, the TTP findings are likely to intensify questions from regulators and safety advocates over how effectively the company detects AI-enabled abuse and whether repeat advertisers face meaningful consequences for violating its policies.
Related News: Meta recently faced backlash over its AI tools after pulling a Muse Image feature that allowed users to place public figures’ faces into AI-generated images.