Maybe its just me who is paranoid because I happen to spend a fair bit of time in the advertising world, but the first thing I did when memory was launched on Claude/Chatgpt - was to switch them off. And it helps that they are not even useful, and would actually downgrade your experience by polluting the context of irrelevant details. I go one step ahead - if there is a personal discussion you want to have - maybe use another account like provided by the likes of companies like openrouter etc.

I would argue that we should have regulation that should prohibit the storage of user profile information by AI companies, and any such memories feature should exclusively reside on the users servers. Infact, maybe go one step ahead, that 'memory' firms cannot be owned by AI firms and vice versa.

Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.

But turns out I was playing 4D cybersecurity chess

Might be the first time I see someone complain about their website being protected from a scraper, instead of the other way around.

But after seeing this, I think I might switch to a weekly VM reset rather than monthly.

BTW, if anyone is interested in a decent setup for an AI agent jail, the scripts at https://jai.scs.stanford.edu/arch-vm.html are what I used, plus adding a few more packages to the pacstrap command such as dotnet-sdk. I then made the guest root directory a BTRFS subvolume, so that I can snapshot it. Then spinning up a new VM is a `sudo btrfs subvol snap template-root newvm` command (basically instant) followed by running the `qemu-system-x86_64` command (takes a couple of seconds). It's easy, but I retain complete control over the contents of the VM. It's been great so far.

Either we stunt the models to the point where they are not useful, or we allow things like this to seep in and create one of the most insecure concepts the internet (and maybe tech as a whole) has ever seen: a robot that can be tricked.

A close friend prompted it about some troubleshooting of a pet smart feeder and it responded with instructions but using my pet’s name to my friend.

I found that extremely strange for it to be a coincidence. My pet's name is not that generic for it to be in training data, and the connection to my friend makes it more strange to me.

That made me wonder if there’s cache pollution or some session data leakage in it exposing stuff. (My friend has been in our wifi for example)

Has anybody else noticed something like this?

It’s not a terrible idea really, but I wish it would’ve asked me first.

Nice write up of your findings. Enjoyed reading an article written by a real human.

Now I believe the biggest threat to personal information exfiltration are the people themselves and there's quite literally nothing you can do about it.

I’m not sure I get the pushback on the robots file. Shouldn’t the robot prevention be ON by default?

That's a hard one for Cloudflare, no? They got to where they are by being (if you want to be cynical, playing the role of) the benevolent, neutral guardians of the internet, a one-stop shop that makes most of the bad nonsense go away without much effort on the part of the developer. Continuing that stance probably does mean some basic AI crawler blocking by default, unfortunately. At least they document it [1].

[1] https://developers.cloudflare.com/bots/additional-configurat...

I made a prototype where AI automatically fills in the checkout basket for an amusement park. I found that ChatGPT tells how many adults, how many kids, what date suits you.

There are quite some security concerns, but fully banning AI from filling in query parameters with relevant user data is not the solution. This is also why I think Claude didnt give the bounty. Their solution would likely be a combination of trusted domain allow list and better security model that protects user agent.

Tangentially, I was experimenting indirect prompt injections in Claude Code (also using the user-agent trick) with Fable-5 [0]. Eventually, it executed untrusted code just by asking "Summarize this repo". Interesting times ahead...

[0] https://veganmosfet.codeberg.page/posts/2026-07-15-quest_rce

> "no bounty was awarded"

Ridiculous. Anthropic engineers are not just stupid to allow such a vuln in the first place, but they also try to hide such vulns from their bosses because a bounty payout would need to be explained to the finance team.

Stockholm syndrome

You have the right to remain silent, the profile your LLM has made about you can and will be used against you in a court of law.

what?