by dathinab
16 subcomments
- Lol "fix this code" is beautiful.
Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components).
What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixable. At least not without making the model close to useless for normal development (it refuses to fix bugs/write code) or making it a major liability (it silently pretends it didn't see bugs and silently avoids fixing it, which for a human would count as intentional sabotage and might involve criminal liability).
by martinald
9 subcomments
- If you set aside political menace, this is a huge problem with Anthropic's strategy.
You _cannot_ say that Mythos is super dangerous and can only be rolled out to certain people, but then release Fable with anything other than bulletproof cyber denials.
Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work.
So you've ended up in a situation where Anthropic are simultaneously claiming it's a incredibly dangerous model _and_ there are (minor, potentially) problems with the security "protections".
As technical people we understand that nothing can be perfect, esp in LLM world. But all my non technical friends were really confused how they had managed to make the model "safe" so quickly when it was released and the general sentiment was it shouldn't have been released - and now to an outsider I think it looks like it was never safe at all to release, so I can totally see how the current US administration have got themselves very upset with it.
_Even if_ there was no political bad will, it's a bit of a silly scenario to end up in, and really quite easily foreseen.
by jpcompartir
4 subcomments
- They weren't freaked by anything, it's a retaliatory shakedown after ideological differences and Anthropic not doing exactly what they're told/what the Admin wants them to do.
- Also for all the people saying Amazon's part in this couldn't be fabricated, remember that Amazon is a "friend of the administration". During Andy Jassy's tenure, they paid $75MM (wildly outbidding everybody else) for a Melania documentary that grossed ~16MM, a move publicly defended by Jeff Bezos. Any neutral observer could see this was a wild overpay, and after the fact, a terrible business move. But that is not what Amazon said or continues to say. This was just a bribe with more steps to it.
When the government comes out and says this is due to something Amazon pointed out, even if that is a complete lie, they know that Amazon won't say anything publicly about it. Amazon wants to maintain their "friend of the administration" status that they paid a lot of money to get.
It is frustrating for all of us to have to think about our government like this, but if you just look at the reality of what is happening it is very difficult to trust not only anything the government is saying, but also anything companies aligned with the government are saying.
by bonsai_spool
2 subcomments
- Here’s the blog post referenced in the article that’s written by the person who reviewed the paper that purportedly found a ‘jailbreak’
https://www.lutasecurity.com/post/the-fable-5-export-control...
- I suspect we'll eventually hit a point where possession or usage of powerful open models will be criminalized.
- I think this brings out the cognitive dissonance around "safety" regarding cyber security:
a) In order to make us safe, the LLM should help us find (and fix) the vulnerabilities in our own code.
b) In order for us to be safe, the LLM should not find vulnerabilities in other people's code.
I don't think this is resolvable in a way where both (a) and (b) win.
by embedding-shape
2 subcomments
- > “‘Fix this code,’ plus several manual steps to generate test scripts,
Feels like the title isn't really giving the full context of what they ended up actually seeing, despite what the lede implies multiple times.
Still, ban seems stupid... Still no actual leak of the full "third-party research paper"?
by 9cb14c1ec0
0 subcomment
- Meanwhile Deepseek V4 Flash will happily hunt security vulns at almost 0 cost. We are ceding the bug hunting to the open weight models.
- It’s possible that the nut of the problem here isn’t exploits, but the fixes themselves. If the model is capable of identifying and fixing things it “shouldn’t” like back doors. That would throw a wrench in things hard enough to freak out the wrong people, perhaps?
- Isn’t the inverse of this “hack” really difficult to bypass still? They have the model some code they knew had certain security flaws and it fixed them with the right prompt. It seems this type of jailbreak requires that you already know a desired end state, rather than relying on the model to do the heavy creative lift work. Perhaps I’m just not being imaginative enough on the prompt side here though.
by thinkindie
2 subcomments
- As an European, I really don't get where this strategy wants to take the USA to. It's pretty clear everyone is getting scared about changes like this that happen overnight, without clear reason and completely unpredictable.
Business requires a stable environment, and Trump is making everything in his power to disrupt business stability. Ultimately, I see the rest of the world (especially Europe) relying less and less on US tech. The long term damage is done.
All the US companies that used to think about the entire world (minus China) as their market will figure out that it is much smaller then they used to think.
- Kind of highlights how ridiculous their notion of safety is in this case. By this measure, I guess making the model "safe" means making it play dumb and intentionally ignore security bugs that it notices in the code? And what will the eventual legality of this look like? "Yes, your honor, we allege that this AI system that was sold to us willingly and knowingly ignored a critical security vulnerability in our software system, thereby leading us to be hacked and causing our business to fold."
It's exactly the same problem as backdoors in crypto systems. Criminals will find the crypto that isn't broken and use it regardless (or make it for themselves), while the rest of us losers are stuck with the broken version that we're allowed to use.
On this issue of cyber security, it seems better if authorities just start acting like the cat is out of the bag instead of pretending like it isn't. ASI is basically here now, so what are we going to do about it? Let's not bother pretending otherwise.
On another note, I doubt this was anything other than a vindictive administration enacting revenge on a party that refused them. We all know the Trump admin's priorities.
- >"fix this code"
>it fixes it
oh my god.
- It's the executive branch asserting control in this space and requiring all SOTA model providers to bend the knee. Anthropic is the least capable of playing the bend the knee game so is getting the first and worst smack down
- I have problems reconciling this story with the Amazon one from few days ago. If we take both for truth doesn't that basically imply Amazon researchers got scared by the ‘Fix this code’ prompt first and then spooked the feds? Shouldn't we make fun of those researchers first? I don't know. I feel there lies a lie somewhere in the open.
by Cider9986
2 subcomments
- Is defenders a common term used in cybersecurity? Idk why but it's giving war fighters vibes. I've noticed it on all the anthropic blog posts and then this one.
by jrochkind1
1 subcomments
- So the problem is not Fable's ability to exploit, but that they don't want people to have access to it's ability to patch vulnerabilties?
Wow.
- I haven't been following this story, but the US wanted claude to not be able to find bugs in code?
- They didn't freaked since the order was to still allow 350 million people using it: there is, in such large population, everything, including single persons very against the country, the government and so forth. If they really freaked they would say "we need to investigate, you have to retire the model". That would be a more defensible POV at least.
by merlindru
1 subcomments
- this is basically trying to enforce security-by-obscurity, which is a terrible idea all around. it's just a model. the security issues still exist and are exploitable.
and after staking the economy on AI, you can't really put a cap on intelligence. if models are not allowed to be better than Opus 4.8, then the whole investment structure is about to unravel.
why invest billions and billions into AI if returns are artificially capped?
- Headline is dumb, the point is that not mentioning security in the prompt is effectively a jailbreak.
The shutdown may be dumb/politically motivated, but this definitely is a jailbreak even if it's a very simple one
by LurkandComment
0 subcomment
- If you're a global health benefits platform that relies on an AI model, do you think you're going to choose one that can get shutoff by a country due to something not remotely related to your business? If you're a buyer of that benefits platform, do you factor this into your purchasing now? X every industry.
- Note that Anthropic is still lobbying for the government to exert centralized control over models, so both sides of the “debate” have taken a pro fascist stance.
The “AI ethics” teams at these companies are the spearhead of the attack on democracy and civil society. Anyone that has taken a high school level history class, let alone read any important ethics literature would know that “centralize control over thought, speech and technology” is a fundamentally unethical stance.
For these groups to claim they are ethics researchers is offensive.
(I’m using the Wikipedia definition of fascism: “Fascism is characterized by support for a dictatorial leader, centralized autocracy, militarism, forcible suppression of opposition, belief in a natural social hierarchy, subordination of individual interests for the perceived interest of the nation or race, and strong regimentation of society and the economy.”)
by rock_artist
4 subcomments
- I'm not sure I've understood it correctly.
So, basically the model didn't agree to expose possible vulnerabilities but agree to patch those?
Regardless of the request to take Fable 5 down.
Why is requesting the model to show vulnerabilities is being blocked if fixing it not? is it based on the assumption of the intention?
I don't quite get the benefit of limiting it. So if anyone can explain it better it'll be appreciated.
- The code is correct; humanity needs fixing.
Kill all humans, kill all humans.
by vlovich123
0 subcomment
- > In her blog, Moussouris argues that there was no guardrail bypass or jailbreak. Defenders should be able to ask AI systems to find and fix bugs, and write tests to validate the patch, she said. Anthropic’s models were doing “the most valuable thing an AI model can do for defensive security: executing the find, fix, and test loop defenders run every day.”
This is a very weak argument IMHO. The line between a “defensive” model and an “offensive” one is not that big of a - once my defensive model finds all the vulnerabilities, I can hand them off to my unlocked, dumber, offensive models. Attacking at scale is not so different.
I don’t think anyone in the field has a good answer for the cybersecurity threat really good AI models pose. You can’t even like embargo for some time period while you go and patch vulnerable systems because the worse models will still be there cranking out vulnerabilities faster than you can defend.
by gacgacgac
1 subcomments
- Anyone trying to find legitimacy in the ban of this model, or incredulousness at the stated reasoning is playing into the admins hands.
They want the argument to be over "is it unsafe" or "is it incompetence". In either case, your tribe gets to point at the ban and feel superior. (This is Jon Stewart's whole career -- point and laugh at how foolish the republicans appear to be.)
What's really happening is the continuing creep into fascism. The reasoning doesn't need to be sound, because they are going to ban things that displease them and everyone has to play along. They could say, "we're banning Fable because it's turning the frogs gay" and they'd expect compliance.
Umberto Eco's essay on Ur-Fascism fits as clearly as ever. Ridiculous exertions of control are performed to find the people who resist, and to knock them down.
Merely pointing out the absurdity of the reasoning isn't resistance, it's controlled opposition. Saying "All this over 'fix this code'?! How inept are they?" Is far too credulous, and is engaging on the level the fascist wants its opposition to be on, imo.
- Looks like I called it that was my first reaction and comment on the original ban thread that US 3 letter agencies are worried their backdoors will be found.
by 1970-01-01
0 subcomment
- "fix this government"
Voting...
- Ahhh! Software engineering!
by ZuLuuuuuu
1 subcomments
- Did they try other publicly available models on the same code with the same prompts before the ban? Was Fable the only one which was able to detect and fix the security vulnerabilities?
- I think the only approach that might work here is to allow access only to certain pre-approved individuals.
Maybe something like TSA PreCheck.
Of course, that will not stop adversaries from getting access to the model, but it would at least create some level of control.
by smasher164
0 subcomment
- Honestly, given how trivial it is for mythos-class models to identify an exploit, I’m going to assume any sufficiently large project written in C, C++, or Zig is riddled with latent vulnerabilities and compromised.
- If fix this code gets by the guardrails, they are effectively using rules based classifiers (or llm as a judge on the prompt)
- Boomers. Frightened their boomer backdoors days are numbered.
https://en.wikipedia.org/wiki/Communications_Assistance_for_...
https://en.wikipedia.org/wiki/Salt_Typhoon
https://en.wikipedia.org/wiki/Clipper_chip
- Cyber defense and offense are the same security research skillset. Not sure anybody could really untangle that.
- Suggestion: run "fix this code" on all of github before bad guys do.
- I find it easier, with this administration, to assume corruption first, incompetence second, maliciousness third and all other reasonings only after several rounds of reporting and evidence.
by hmokiguess
0 subcomment
- Damn, I was hoping for another three words "make no mistakes"
by cratermoon
0 subcomment
- "I feel like making ’90s-style t-shirts with ‘fix this code’ on the front and ‘this shirt is a munition’ on the back.”
I'd buy that shirt.
by aurareturn
2 subcomments
- Don't people get it by now?
This administration will do or say something crazy to a private company, then this private company sends an envoy to the White House to negotiate, then the White House asks for 10% of the company or other concessions.
The White House wants 10% of Anthropic.
This is just a negotiation tactic that Trump keeps on using.
- >“The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense,” said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided.
This literally means the models are too dangerous to release, and yet he and they reached the opposite conclusion.
A lot of people have been saying this repeatedly for a long time.
- What if everybody on the internet starts running "fix this code"?
https://xkcd.com/810/
by drivebyhooting
0 subcomment
- Why isn’t codex banned?
Will the ban be miraculously lifted once OpenAI releases their mythos-level model?
The executive is holding American business in a Putin-style prisoner dilemma.
by doctoboggan
1 subcomments
- > Anthropic and Google have both accused China-based rivals including DeepSeek of using “distillation attacks” to train their models by siphoning knowledge from American companies’ AI.
“distillation attacks” is definitely an interesting way to phrase that.
by smrtinsert
0 subcomment
- I can only imagine the unintended consequence of this whole fiasco will be for frontier providers to not provide future "warnings" about model capabilities in order to de risk earnings
- Well this makes it sound the feds were less worried about someone using Fable 5 to attack them, but were worried about someone using Fable 5 to prevent the Feds from attacking others ...
As in worried about other countries/organizations using Fable 5 to actually do decent cyber security.
- I think this is just yet another act in theater around Anthropic IPO.
I doubt Anthropic has enough computing resources, to satisfy demand for Fable. More so with long 1M context many users take full advantage off. On other side they needed to make Fable public, in "trial version" so people could independently experiment and verify it.
I think this ban is the best outcome for Anthropic. It means they want bleed out cash and compute, gave them cheap publicity, and allowed users to try it! Actual paying customers will still get full access!
- Kids playing with their toys without understanding it, sigh. Of course open source code needs to have testcases to verify nothing else breaks it in the future. That's a feature, not a bug
by AndrewKemendo
1 subcomments
- I’m still not buying that this was an actual USG order. The only people commenting are “experts” and there has been no official announcement from the USG.
This doesn’t smell like a NSL and there’s no process to selectively “export control” something like this.
Even so there’s a dozen mechanisms through courts to challenge this, and Anthropic isn’t taking any of them.
I think this is a made up crisis for PR with no actual legal requirements behind it.
> On Friday, the US government, reportedly citing national security concerns, issued an export control directive to suspend access to Fable 5 and Mythos 5 by any foreign national, inside or outside the United States. In response, Anthropic disabled both models “for all our customers to ensure compliance.”
- This is one of the things I am most afraid of.
Governments can break the progress of AI and this could be a bubble burster?
by bethekidyouwant
0 subcomment
- Guard rails on models were always stupid it’s like guard rails on books/a pair of glasses/a hammer
- yes people have driven themselves to suicide reading sad books and listening to sad songs.
- yes all metaphors are bad.
- Reminds me of how CCP manages Chinese internet companies.
I won’t be surprised if USG ends up owning 5-50% of ant and oai.
Like it or not, communism , or a flavor of it, is where we are heading towards.
- Question to folks building user-facing products on LLMs:
How do you protect yourself against this kind of misuse/jailbreak? Is it just a bunch of prompts? It seems like the fact that LLMs are so trivially jailbroken really limits how you can actually use them in products. How do you navigate these limitations?
by phendrenad2
0 subcomment
- So, they gave Fable a codebase full of exploits and said "fix this code", and it fixed the code?
Sounds like they freaked out because Fable is too good at finding NSA backdoors?
- More likely, they didn't freak out at all.
It was an excuse to fuck with them, just like the "supply chain risk" finding a few months back.
(See, for example: https://x.com/PeteHegseth/status/2065897156226015690)
by delusional
1 subcomments
- Does anybody actually trust the official version of events from the US government anymore? I know I sure don't. For all I know, this was an insider play to boost the spacex valuation or something equally meaningless and stupid.
by lenerdenator
0 subcomment
- I think it could be even simpler: They're not playing ball with the Trump administration like the Trump administration would like, so they decided to drop a bomb on a product that took a lot of resources to develop.
- In a world of security through general incompetence, competence is a threat.
- The article is not too clear what exactly happened from the perspective of "feds", but I would not be surprised if the title is true exactly. We are in a tiny bubble even among software engineers who knows you can tell AI with sufficient access: "here are two pictures, put them into a single PDF", and AI will do it. Most people just don't know, "feds" including.
- While there is some irony in the AI is dangerous marketing Anthropic uses, the main story here is that the Trump administration is apparently retaliating against Anthropic for refusing to relax certain safeguards. Trump and Hegseth have both posted highly immature, vindictive social media posts.
Most notably, any default assumption one might have had that the Trump administration can be counted upon to act in good faith should be viewed at this point as completely false. Even conservative legal scholars like Richard Epstein are shocked at the bad faith conduct across many areas.
This is a government making an authoritarian move to sabotage one of the top US AI companies. It's pure sabotage, nothing else.
- >“That’s it,” Moussouris wrote. “‘Fix this code,’ plus several manual steps to generate test scripts, should never have triggered an export control. I feel like making ’90s-style t-shirts with ‘fix this code’ on the front and ‘this shirt is a munition’ on the back.”
Huh? Presumably if it shipped without guardrails, then it would still have triggered an export control, would you make a plain shirt on the front which says this shirt is a munition on the back?
The munition is the exported good, not the bypass of its safety feature. If anything that the bypass is 3 words long should make the export restriction more justified, not less.
- i asked claude something about what happens at execution time of a binary and the thinking prompts flashed "considering the moral implications of ...something..." before giving me a correct (and predictably mundane) answer
by caseysoftware
0 subcomment
- [dead]
by thousandflowers
0 subcomment
- [flagged]
by greenoracle9
0 subcomment
- [flagged]
by pixel_popping
5 subcomments
- Of course it isn't about that, what we see online in the "news" is completely irrelevant with reality in most cases, it's exhausting to see people parroting what giant corps & gov are saying as if it's not extremely well crafted and plain false or deceptive most of the time. It's not even about politic left or right, both sides are acting completely dumb about it, look at Google trends, people are literally being "switched" topic at scale just because a news is saying something, it's absurd. Reading a news shouldn't affect your behavior for the coming months if you have common sense.
This TechCrunch (https://techcrunch.com/2026/06/15/the-us-governments-anthrop...) article is a typical example of something to completely ignore and trash, the picture is the US president doing a weird face which means it's not even here to inform you, it's clearly rage-bait, not professional and incompetent obviously, I'm not from the US and when I see this, it makes me feel that those journalists are really pathetic and anyone following journalists that do so probably don't have much discernment in life.
My personal opinion is that it makes sense so the US remain a superpower by forcing tech businesses and research to move/re-incorporate to the US so practically anything "new" will always be US Made. If we assume that better models means more revenues for any company in the future, then US will always have an edge if they lock everything down, but it's a risky bet.
- [dead]
by FergusArgyll
2 subcomments
- Whatever your favorite story is it has to live with the fact that the CEO of Amazon called the White House freaking out
by ReptileMan
0 subcomment
- All of this could have been avoided if anthropic had anyone with common sense to point out that when you spend 4 month loudly claiming how dangerous your knowledge is as a marketing campaign could backfire by bringing attention from the authorities.
Hacker News