San Francisco wants AI 'nudify' apps thrown out of Apple and Google's stores
San Francisco's City Attorney sent cease-and-desist letters to Apple and Google demanding the removal of 13 AI face-swap and nudify apps used to create nonconsensual sexual images.
San Francisco has decided to attack one of generative AI’s most toxic uses through the door that hurts: the app stores. The City Attorney sent cease-and-desist letters to Apple and Google demanding the removal of 13 face-swap and “nudify” apps — tools used overwhelmingly to create fake sexual images of real people without consent, mostly targeting women and girls.
What are nudify apps and why are they in the crosshairs?
They’re applications that take an ordinary photograph and generate an undressed or sexualised version of the person in it, in a few taps and with zero technical skill required. The city argues Apple and Google profit from these apps — which break the stores’ own rules against harmful and sexually exploitative content — and, beyond removal, it demands information on how they were reviewed, ranked and monetised.
The move is smarter than it first looks: instead of chasing app developers one by one, in a cat-and-mouse game the internet knows all too well, the City Attorney is aiming at the distribution layer. The two companies control the gateways of mobile software — close the gate, and these tools’ reach shrinks from millions to a trickle.
Will Apple and Google actually remove the apps?
That’s the open question. Both companies have pulled apps like these before, but the claim that the problem is systemic — from weak review to rankings that promote them — implicates the store model itself, not just a dozen bad actors. The case could set a legal precedent on platform responsibility for distributing abuse tools, a debate Europe is watching closely while tightening its own regulatory squeeze on AI. The official documents are published on the San Francisco City Attorney’s site.
We’ve seen platforms move when the topic is imagery and AI — Meta, for one, walked back an Instagram AI feature after user backlash. The difference is that this time it isn’t the market asking: it’s a legal demand, with a deadline and letterhead.
By Oliver Grant
Image: Bernard Spragg. NZ from Christchurch, New Zealand / Wikimedia Commons (CC0)
