Meta Oversight Board Urges Stronger Protections Against Sexualized Deepfakes

Meta's Oversight Board has called on the social media company to strengthen its protection for ordinary people targeted by sexualized deepfakes. The Board recommends the addition of AI-generated impersonations in Meta's Adult Sexual Exploitation policy, arguing that those images and videos are non-consensual by default. It also wants Meta to allow users to designate "connected accounts," such as trusted friends and family, who can report potential violations like non-consensual intimate imagery on their behalf.

Finally, the Board recommends making AI-generated sexual impersonation a separate category from harassment and nudity in the company's content reporting and appeal forms. At the moment, only the residents of Texas and Florida have access to a specialized form that lists deepfake intimate imagery as a reason for the report. The Board wants all Meta users to have access to it, because "AI-generated non-consensual intimate abuse, including sexualized impersonations, is a global issue."

The Meta Oversight Board came up with these recommendations after investigating an incident wherein the company kept ignoring a user's report about the sexualized impersonation of a friend on Instagram. It launched the investigation after it received an appeal from the user who reported an AI-generated video on Instagram showing a woman adjusting her dress, with her underwear visible in a few frames. According to the Board's report, the reporter said they were a friend of the person being impersonated in the video without content. The person who was depicted in the AI video had already closed her account.

Two users originally reported the video to Meta, but the company didn't remove the deepfake. The user who appealed to the Board had submitted an appeal to Meta first, but the company still didn't remove the video from Instagram. After the Board itself raised the issue with Meta, the company merely made the post adults-only but concluded that it did not merit removal under its community standards.

Meta told the Board that at the time the post was originally reported, it had no indications that the individual in the AI deepfake was a real person. If the depicted individual herself had reported the video, it would have violated its Adult Sexual Exploitation policy. The self-report would have served as a clear sign of non-consent. Other credible indicators of non-consent in Meta's eyes are reports from law enforcement, media or trusted partners. Captions or page titles suggesting that images or videos are shared in a "vengeful or sensationalist manner" will work, as well.

The Board says Meta's responses to its investigation indicates that the only viable way for non-public figures to establish non-consent is to self-report. It wouldn't be easy for ordinary people to get law enforcement or the media involved, after all. Those avenues are mostly accessible to public figures. Meta is required to respond to these recommendations, but it's not obligated to implement them. If it does choose to adopt them, the Board will monitor its implementations. For the particular case that started this investigation, the Board has overturned Meta's decision to leave the video up and has required the company to remove the post.

"It is clear that the scale, speed and sophistication of AI tools have resulted in a proliferation of AI-generated sexualized non-consensual content globally. The spread of sexualized deepfake videos leads to reputational and psychological harm, which disproportionately impacts women and girls, and has a chilling effect on participation in social and political life," the Board wrote in its report of the investigation.