[1] Specifically, "...synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated", see https://artificialintelligenceact.eu/article/50/
I think that AI service providers should have safeguards and encoded attribution. This solution helps when people lazily share things with friends or on social media I suppose, rather than stopping motivated bad actors.
The only way to actually implement this I think would be to ban all local models, and to have the service providers store perceptual hashes all generated images and video. It feels like the cat's out of the bag already though (for images at least).
We need to be super careful with how legislation around this is passed and implemented. As it currently stands, I can totally see this as a backdoor to surveillance and government overreach.
If social media platforms are required by law to categorize content as AI generated, this means they need to check with the public "AI generation" providers. And since there is no agreed upon (public) standard for imperceptible watermarks hashing that means the content (image, video, audio) in its entirety needs to be uploaded to the various providers to check if it's AI generated.
Yes, it sounds crazy, but that's the plan; imagine every image you post on Facebook/X/Reddit/Whatsapp/whatever gets uploaded to Google / Microsoft / OpenAI / UnnamedGovernmentEntity / etc. to "check if it's AI". That's what the current law in Korea and the upcoming laws in California and EU (for August 2026) require :(
Remove/Bypass Google's SynthID AI Watermark - https://news.ycombinator.com/item?id=46692023 - Jan 2026 (1 comment)
SynthID – A tool to watermark and identify content generated through AI - https://news.ycombinator.com/item?id=45071677 - Aug 2025 (83 comments)
SynthID Detector – a new portal to help identify AI-generated content - https://news.ycombinator.com/item?id=44045946 - May 2025 (1 comment)
Also, if it's essentially a sort of metadata, can't the output generated image be replicated (e.g. screenshot) and thus stripped of any such data?
In a sense, the identifier company can be an arbiter of the truth. Powerful.
Training people on a half-solution like this might do more harm than good.
But I suppose it ads friction so better than nothing.
Watermarking text without affecting it is an interesting seemingly weird idea. Does it work any better than (with knowledge of the model used to produce said text), just observing the perplexity is low because its "on policy" generated text.
>The watermark doesn’t change the image or video quality. It’s added the moment content is created, and designed to stand up to modifications like cropping, adding filters, changing frame rates, or lossy compression.
But does it survive if you use another generative image model to replicate the image?
I'm thinking of historical images, where there aren't a huge number of existing images and no more will ever be created.
If I see something labeled "Street scene in Paris, 1905". I want to know if it is legit.
...But it can be hard to tell the difference between content that’s been
AI-generated, and content created without AI.
Pro-Tip: Something like that Sherbet colored dog is always AI generatedWhat incentive do open models have to adopt this?
Some previous discussion:
"Generate a pure white image." "Generate a pure black image." Channel diff, extract steganographic signature for analysis.