New App Store Rules Target AI Deepfakes and Nudify Apps

Apple's expanded moderation rules introduce a clearer expectation for app developers regarding AI-generated content. While sexually explicit material was already prohibited, the new guidelines specifically mandate developers to actively identify and eliminate AI deepfakes from their platforms. This shifts some of the responsibility for content moderation to the app creators themselves, under threat of removal from the App Store.

The emphasis on proactive prevention and removal, particularly for AI content creation apps, highlights a growing concern over the misuse of generative AI technologies. The mention of xAI's Grok chatbot, which faced a warning from Apple over nude deepfakes, indicates that these rules are a direct response to reported issues and aim to formalize previous ad-hoc interventions.

Furthermore, the new guideline 2.5.19 addresses a "bait-and-switch" tactic, where apps might misrepresent their purpose during the review process and later introduce harmful functionalities. This rule aims to curb the distribution of apps designed to create nonconsensual intimate imagery, such as "nudification" apps, by closing a loophole that allowed developers to circumvent initial content screening.

These updates reflect an ongoing effort by Apple to maintain control over the content distributed through its ecosystem, extending its existing bans on illicit content like human trafficking and child exploitation to alternative app distribution methods. The company's firm stance suggests a low tolerance for non-compliance, aiming to protect users from the evolving risks associated with AI-generated content.