Your Kid's Face Is a Dataset Now

There’s a warning going around this week telling parents not to share photos of their children online because of AI abuse risks. Deepfakes, training data, image manipulation. The usual horror show.

My daughter is sixteen. She’s been on the internet long enough to manage her own privacy better than most adults I know. But I think about the kids who weren’t afforded that choice, whose entire childhood exists in searchable, scrapable, publicly accessible form because their parents wanted to share a birthday photo in 2011 and didn’t think too hard about it. I don’t blame those parents. The platforms were designed to make sharing feel natural and harmless. It wasn’t, but the consequences weren’t obvious yet.

They’re obvious now.

The thing that gets me about this conversation is the hypocrisy sitting right at the centre of it. We’re being warned, quite rightly, about the risks of sharing images online. Meanwhile, governments are rolling out mandatory age verification schemes that require you to upload a photo ID and a selfie just to access parts of the internet. Spain’s data protection authority just fined a biometric verification company over a million euros for mishandling exactly that kind of data. The same technology we’re being warned to avoid feeding is being made compulsory by the people issuing the warning. That tension doesn’t resolve neatly. I don’t think it’s meant to.

Someone in the discussion I was reading made the point that the messaging is genuinely incoherent: sharing a photo is so dangerous we need public health-style warnings, but normalising biometric submission to random third-party companies is apparently fine. They’re not wrong. The UK’s online safety regime is particularly egregious here, building a surveillance architecture under the banner of child protection while the actual children’s data gets handled by contractors whose security practices range from adequate to, apparently, fine-worthy.

I’m not a tinfoil hat person. I use cloud services. My photos back up automatically to Apple’s servers and I’ve made a rough peace with that because the alternative is a level of friction I haven’t been willing to accept. But I’m aware that’s a choice made from convenience, not from any illusion that it’s safe. The difference between me and a parent posting their kid’s face to a public Facebook account tagged with the school name and suburb is a difference of degree, not kind.

The image poisoning tools that came up in discussion, Glaze, Nightshade, Foxglove, are interesting in theory. The honest assessment seems to be that they work against training pipelines but do essentially nothing against inference-time image manipulation, which is what most of the abuse cases actually involve. Someone wanting to generate a deepfake doesn’t need to train a model on your child’s photos. They just need a few reference images and a model that already exists. The attack surface is different from what those tools were designed to address. Useful to know before you invest time in them believing it solves the problem.

What would actually help is less exciting: private accounts, limiting who can see what, not posting identifiable images of kids publicly in the first place, and having direct conversations with grandparents about why forwarding photos through Facebook Messenger to seventeen relatives is not a controlled distribution. That last one is genuinely hard. The technology literacy gap in extended families is where a lot of this falls apart.

I don’t know what the right answer looks like at a policy level. Stronger data protection law, actual enforcement, liability for platforms that allow non-consensual intimate imagery, better tools for takedown. All of that seems obvious and all of it moves slowly. In the meantime the data centres keep getting built, the models keep getting trained, and somewhere there’s a sixteen-year-old whose entire childhood is already out there, in full resolution, waiting.

That’s not a comfortable place to leave it. But I think it’s an honest one.