Why do I need a system prompt at all?

Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time.

Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for.

Why do these models (supposedly) keep getting “better” (on benchmarks) but continue to degrade in output quality while grtting more expensive for actual work?

I use Claude every day but I’m getting disillusioned by the so-called “progress”. If my employer wasn’t paying for my access I would not pay for any of these things. Don’t even get me started on the absolute brainrot inflicted on people that I work with from depending on these things every day, it’s depressing.

I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.

But I also have a hands-on human-in-the-loop working style so I guess maybe for people who just want to say "implement all open features in github issues" and walk away maybe there needs to be more of all this CLAUDE.md stuff

However I suspect there was always some gearhead type attraction to setting up detailed harness configs that may be unnecessary and more like hobbyist tinkering.

Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want. With reasoning traces hidden, I am sometimes not even sure if it used those memories or just independently decided that PCI-DSS subsection-whatever is somehow relevant to this PR that has the word “credit”.

There is no way for me to fully configure memory preferences at a granularity which would be useful, and so I continue to use context files (and other tools, sometimes) to ensure the right memories are stored and surfaced at the right times.

There’s a lot of room for agent memory improvement across the ecosystem, and I don’t think the LLM providers should try to own this vertical slice. This will never happen though, because it makes us “sticky”.

Or maybe I’m holding it wrong.

I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around much more frequently than 4.8.

Most people generate CLAUDE.md with /init at least at first, so it gets filled only with the superficial top level things that Claude already noticed during that first run. By this logic, shouldn't CLAUDE.md contain the exact opposite of what /init currently includes?

> Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start.

This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.

Managing the context that agents have available to them is far too important to leave to the agents themselves. Agents tend to write far too much into their memory, they are terrible at trimming it down, and their choice of what to include is very poor. I have had much more predictable results by disabling auto-memory and actively shaping my CLAUDE.md, skills, and documentation instead.

Maybe one day agents will be able to manage their own context, but that day is not today.

I thought it was only a problem for English communication, for which we have little care, but the same applies to code now. More and more code appears to be “probabilistic code”, homogenized to what is most common in the data LLMs were trained on.

I'm eager to test the new version. If it's 80% shorter now, then saying hello should only cost Fable $0.20. Exciting times.

> When we first rolled out Claude Code, we needed to be sure that Claude avoided worst case scenarios, such as deleting files. This meant we would give particularly strong guidance that might not always be true

Oh, AI Company, you would want us to do that, won't you?

I don't want to trust claude cli, kilo, glm, claude desktop, codex etc.

It's high time Apple and Linux world wakes to the advent of LLMs and local agenting coding tools, so that users get really intuitive (read GUIs, simple CLIs) OS level tools and APIs that these coding tools are forced to adhere to that users can control broadly or in a micro managed way, if they need to. So that they don't have to worry about when the model will go berserk and fool the tool.

I've been skeptical and these guidelines validate this. I continue to document code and write specs as I've always done. If an agent produces poor output or misunderstands, I use that as an opportunity to improve the docs, but in a way that that aims to be accessible for human peers, not the quirks of the current generation of models.

This (among many others) is the reason I use GPT over Claude. It adheres very closely to your instructions and rules. You can build up your own workflows and systems as a result.

Claude just does whatever it wants, regardless of what you tell it. It's a miserable experience.

Claude is what you use when you want to one-shot a simple cookie-cutter product that has no nuance in it. A product that everyone else is going to develop as well.

ChatGPT is what you use when you want bespoke products and steerability over complex codebases with nuanced decision making.

Anthropic are optimizing for the person who has never written a line of code in their life and has no idea what they want. Computer go brrrr. OpenAI are optimizing for software developers who want to take a systems based approach.

Other comments in this thread show that your mileage may vary. But we spend so much money on Claude Code and give it so many responsibilities that we deserve at least some undeniable proof that it’s bringing value.

Where is the evidence that this new type of prompting is better on real life examples?

To summarize- they were embedding the CLAUDE.MD in a system-reminder with this disclaimer at the end: "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task."

This flew under the radar and they never addressed it, but it felt like at least a big part of the "nerfing" story. I no longer have a Claude Code subscription to test it, but I think it's a useful exercise for most people to sniff the traffic at least once to get an idea of what the back and forth with the Claude Code harness entails.

As others have noted, Anthropic seems to be on a path to make coding ever more accessible to non-coders, and in doing so has removed alot of the controls from devs who do want a more guided experience.

To summarize- they were embedding the CLAUDE.MD in a system-reminder with this disclaimer at the end: "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task."

This flew under the radar and they never addressed it, but it felt like a real part of the "nerfing" story. I no longer have a Claude Code subscription to test it, but I think it's a useful exercise for most people to sniff the traffic at least once to get an idea of what the back and forth with the Claude Code harness entails.

As others have noted, Anthropic seems to be on a path to make coding ever more accessible to non-coders, and in doing so has removed alot of the controls from devs who do want a more manual experience.

It would be interesting to see benchmarks, including these changes, of how harness affects model performance - including both model-agnostic harnesses such as OpenCode and Pi as well as increasingly model-specific ones like Claude Code and Codex.

For the model-agnostic/model-inclusive harnesses like OpenCode it would make sense (if they don't already do it) to keep the harness itself generic and then have per-model sets of skills designed to get the best performance out of each specific model.

- we should try to give good non self contradicting guidance

- we should expect the team member to have knowledge of the craft

- we should focus on higher level, taste and preferences

Starting with Fable 5, if it goes off the rails, it is more difficult to correct it, because it is overall wrong, but covers its tracks with plausible sounding arguments, so it is hard to pin point and correct.

Now, this article points out techniques that were useful to rely on models more, but those peaked at 4.6. Now according to benchmarks Opus 5 is on the frontier. But when when it has looser reigns, it ends up gaslighting me even more with abstract word soup than any model before.

> We removed over 80% of Claude Code’s system prompt for models like Claude Opus 5 and Claude Fable 5 with no measurable loss on our coding evaluations.

And no report of how it impacted earlier models. Was it just BS all along?

80% of the prompt was wasted tokens.

Saying that "give Claude judgment" is too vague for agent implementors. Given the lack of specific details, my takeaway is that we need to go and review all context and rework prompts from prompts/descriptions from scratch until they pass the evals again.

Also: if you deploy code written with assistance from Claude, and then shit goes down, and then investigators look into your prompts, this way of working isn’t going to look good for you from a liability standpoint. Not a fan of this manner of working and thinking.

It feels like coding agents get better at a faster rate and are easier to train.

Not Claude Code but I just had a task where it started referring another conversation that was complete nonsense and throwaway. I absolutely don't want things to get added to some memory behind my back.

A big reason I use LLMs is because I can try out wild ideas and then just throw it away. I don't want those to pollute the context.

At my previous company they have a series of smoke tests for models mostly focused on performance and architecture. I'm still on the team chat and results just came in:

- It's abohrrent at c++: it keeps generating code with data races and, more rarely, use-after-free bugs! It doesnt seem to be able to reason about lifetimes. This on a mostly mid/junior team. It's a bug fest.

- architecture in c++ is a verbose and layered mess even for simple things, which paired with the previous bugs I mentioned is scary.

- For rust obviously there's no use-after-free, but has same architecture pitfalls of layers upon layers. It uses copy and clone all over and performance is bad. Trying to unwrap all that is messy and costs lots of time. Once in a while it generates unsafe code for some non obvious reason

The scary stuff is non determinism. You get different depending on who prompts the agent but there's always some flavour of the points mentioned above. Funny that my team was very adamant on AI-first (why I left) and now they writting more and more code by hand after some very serious bugs and, as they say, dead moments where they have to wait, sometimes hours, and start wondering about the value of their skills, for the model to generate the next spaghetti recipe

Just ask that once per week or so.

I am thankful for the kick in the ass for me to switch full-time into my bespoke harness utilizing open weights & GPT 5.6 and discontinue yak-shaving it with Claude Code.

What are "other sources"?

> Now: Let Claude use judgement

No, it should follow my rules exactly. I don't care what code examples it was trained on - it will either write code the way I want, or I'll use another model.

How about no? I am the one who judges.

Hmm, so what happens in greenfield projects ? In any case, at least all the slop will be consistent.

But well knowing these big AI companies that probably is their goal all along. To lock us to themselves