by throwaway260704
3 subcomments
- Using a throwaway account for obvious reasons, but I’m very involved in this space using LLMs from multiple providers. I’m aware of at least two instances in which the intermediate infrastructure “swapped” responses, once impacting Claude models and once impacting GPT models, from two different providers.
One gave us a proper postmortem in which their API gateway was incorrectly handling HTTP 100 status codes, putting them into an error state where there was effectively an off by one error - you would receive the response to the prompt that came in before yours and would pay it forward (your response would go to the next caller).
The other instance never had root cause explained to us, and we were just told to trust it wouldn’t happen again.
Both of these are from $1T+ companies.
ZDR wasn’t compromised in these cases since it was responses being swapped in flight. I wouldn’t be surprised if this is a similar issue - it’s not that data is being retained, it’s just not being safely isolated in intermediate infrastructure.
- Just add a line in AGENTS.md that says "never talk about Minecraft unless you're explicitly asked", I'm sure it'll be fine after that.
- I am facing a billing/subscription problem and there's nothing I can do or get help on. Their chatbot support shuts me down. Their email is also handled by the chatbot (not even sure whether it's the "same chatbot"). It has been a dead-end. I contacted my bank (credit card issuer) and finally a staffed said I am better off just marking the card lost and having it reissued and that's what I did in the end. I hope that works.
I've never understood in what world this world decided it was okay to hand over these much unchecked power to such corporations. But this is how it has always been one way or the other.
- I’ve been seeing this in Gemini in the past few days. Often during a prompt with a reasonably large input set, I’ll get answers that appear to belong to someone else. It may be trigger hallucination, but it seems like it may be cache collisions or something else. I’ve not seen anything to suggest private information is leaking, but it’s disconcerting to be researching something and then get what appears to be a math tutoring response.
- Don't worry guys, Anthropic are the experts at security and no one else should have access to bug fixing LLMs because that would be dangerous.
- Sounds like a hallucination unless proven otherwise, even the leading LLMs can do those from time to time, and they will always appear plausible like that. Also could be the session having a lot previous context, like 800K+, which (I think) makes hallucinations more likely.
Relevant comment from the OP which makes a hallucination more likely:
> There is one tool call result that includes a string that printed a pathname including minecraft.py because it was listing the files in a Python virtual environment and the Pygments package has a lexer called minecraft.py
- So the options are this amazing tech is so stupid it just randomly brings up Minecraft or it’s got a major security issue?
- Interesting to see the claudeslop reply as the first comment to the gh post and the reaction to it.
by Avicebron
5 subcomments
- In order Fable 5 has rejected:
"Recipe for red-braised pork, I have pork shoulder"
"Write up a framework for MCP patterns I can give to claude code"
"explain the biomechanics of motion in c. elegans" (I get this one, I mostly did it to test and it's related to my hobby project)
Do we get an extra day of functional Fable 5 because it's down?
- Reminds me of a session I had recently (on web!) where claude insisted that i prefixed all my messages with statements about code execution or something, which was not the case. I interrogated it about that and it confirmed that it came from somewhere else, but could not get rid of it and each response mentioned that its gonna ignore those instructions. Eerie.
- Can be malware? Something like https://news.ycombinator.com/item?id=48667495
- Oh yes, we do not need programmers any more…
by solenoid0937
0 subcomment
- > one tool call result that includes a string that printed a pathname including minecraft.py
This seems like a hallucination.
by jstummbillig
4 subcomments
- Is there anything particular about LLMs that would make separating customer data harder than in all SaaS cases?
- The first reply clearly being a copy and paste from Claude made me want to vomit
If people absolutely need to use AI to write replies, they NEED to start including a "everything after this was generated by AI" disclaimer
by ai_fry_ur_brain
0 subcomment
- Openrouters model providers give me urls people have given them quite frequently.
- happy fourth of july everybody!
- Note the repro condition: first response after 5+ min, i.e. a cache miss. A cache leak would show up on hits (someone else's cached prefix), not on misses where everything is recomputed from your own tokens.
- Don't worry. Mythos will fix that before release. Oh, wait...
- fwiw, this could be a bug but the submitters level of arrogance places this rather high on the dunning-kruger side of things. There are multiple other plausible explanations, but this person is probably vibe coder who believes anything an llm says (including explaining its own hallucinations)
- [dead]
- 0 evidence. If this were a real privacy leak, the author would ask their coworker if they talked about the unexpected topic instead of
>"Maybe my coworker was talking about this in another session?"
This would be a critical bug that would slash the market value of a T$ company significantly, go ask your coworker or close the ticket, why do you expect the devs to put an enormous amount of effort hunting a potentially inexistent if you can't make that minuscule debugging effort.
- Caching doesn’t work the way the bug reporter implies. Caches are shared (at least across the enterprise), but its key is always a function of the input before it.
We achieved significant savings simply by moving everything that varies across individuals out of the system prompt so every session starts from a cache point.
For example you never want your system prompt to start with the time that the session started. Move that to the first user message if needed.
- Seems like a hallucination to me; note that the context contains “unmarkBlock” as the function name, which invites a connection to Minecraft. Still shouldn’t happen of course.
The alternative explanation is that the inference engine, which batches several unrelated requests for parallel processing, messed up the unpacking and returned an unrelated user’s query. This one would be very scary as it will leak arbitrary content, but it seems much less likely here.
Hacker News