Ok, I can buy this

> It is about the engineering of context and providing the right information and tools, in the right format, at the right time.

when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution?

If the definition of "right" information is "information which results in a sufficiently accurate answer from a language model" then I fail to see how you are doing anything fundamentally differently than prompt engineering. Since these are non-deterministic machines, I fail to see any reliable heuristic that is fundamentally indistinguishable than "trying and seeing" with prompts.

We are entering a new era of gamification of programming, where the power users force their imaginary strategies on innocent people by selling them to the equally clueless and gaming-addicted management.

Drew Breunig has been doing some fantastic writing on this subject - coincidentally at the same time as the "context engineering" buzzword appeared but actually unrelated to that meme.

How Long Contexts Fail - https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-ho... - talks about the various ways in which longer contexts can start causing problems (also known as "context rot")

How to Fix Your Context - https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.... - gives names to a bunch of techniques for working around these problems including Tool Loadout, Context Quarantine, Context Pruning, Context Summarization, and Context Offloading.

I had one view of what these things were and how they work, and a bunch of outcomes attached to that. And then I spent a bunch of time training language models in various ways and doing other related upstream and downstream work, and I had a different set of beliefs and outcomes attached to it. The second set of outcomes is much preferable.

I know people really want there to be some different answer, but it remains the case that mastering a programming tool involves implemtenting such, to one degree or another. I've only done medium sophistication ML programming, and my understand is therefore kinda medium, but like compilers, even doing a medium one is the difference between getting good results from a high complexity one and guessing.

Go train an LLM! How do you think Karpathy figured it out? The answer is on his blog!

The agents cannot change their internal state hence they change the encompassing system.

They do this by injecting information into it in such a way that the reaction that is triggered in them compensates for their immutability.

For this reason I call my agents „Sammy Jenkins“.

Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. It is about the engineering of context and providing the right information and tools, in the right format, at the right time. It’s a cross-functional challenge that involves understanding your business use case, defining your outputs, and structuring all the necessary information so that an LLM can “accomplish the task."

That’s actually also true for humans: the more context (aka right info at the right time) you provide the better for solving tasks.

Yes, if you have an over-eager but inexperienced entity that wants nothing more to please you by writing as much code as possible, as the entity's lead, you have to architect a good space where they have all the information they need but can't get easily distracted by nonessential stuff.

One thing that is missing from this list is: evaluations!

I'm shocked how often I still see large AI projects being run without any regard to evals. Evals are more important for AI projects than test suites are for traditional engineering ones. You don't even need a big eval set, just one that covers your problem surface reasonably well. However without it you're basically just "guessing" rather than iterating on your problem, and you're not even guessing in a way where each guess is an improvement on the last.

edit: To clarify, I ask myself this question. It's frequently the case that we expect LLMs to solve problems without the necessary information for a human to solve them.

RAG wasn’t invented this year.

Proper tooling that wraps esoteric knowledge like using embeddings, vector dba or graph dba becomes more mainstream. Big players improve their tooling so more stuff is available.

LLM DO NOT REASON !

THEY ARE TOKEN PREDICTION MACHINES

Thank you for your attention in this matter!

---

Forget AI "code", every single request will be processed BY AI! People aren't thinking far enough, why bother with programming at all when an AI can just do it?

It's very narrow to think that we will even need these 'programmed' applications in the future. Who needs operating systems and all that when all of it can just be AI.

In the future we don't even need hardware specifications since we can just train the AI to figure it out! Just plug inputs and outputs from a central motherboard to a memory slot.

Actually forget all that, it'll just be a magic box that takes any kind of input and spits out an output that you want!

"Back in the day", we had to be very sparing with context to get great results so we really focused on how to build great context. Indexing and retrieval were pretty much our core focus.

Now, even with the larger windows, I find this still to be true.

The moat for most companies is actually their data, data indexing, and data retrieval[0]. Companies that 1) have the data and 2) know how to use that data are going to win.

My analogy is this:

    > The LLM is just an oven; a fantastical oven.  But for it to produce a good product still depends on picking good ingredients, in the right ratio, and preparing them with care.  You hit the bake button, then you still need to finish it off with presentation and decoration.

I've seen lots of AI demos that prompt "build me a TODO app", pretend that is sufficient context, and then claim that the output matches their needs. Without proper context, you can't tell if the output is correct.

Single prompts can only get you so far (surprisingly far actually, but then they fall over quickly).

This is actually the reason I built my own chat client (~2 years ago), because I wanted to “fork” and “prune” the context easily; using the hosted interfaces was too opaque.

In the age of (working) tool-use, this starts to resemble agents calling sub-agents, partially to better abstract, but mostly to avoid context pollution.

After working on something related for some months now I would like to put it out there based on the considerable attention being put towards "context engineering". I am proposing the *Context Window Architecture (CWA)* – a conceptual reference architecture to bring engineering discipline to LLM prompt construction. Would love for others to participate and provide feedback. A reference implementation where CWA is used in a real-world/pragmatic scenario could be great to tease out more regarding context engineering and if CWA is useful. Additionally I am no expert by far so feedback and collaboration would be awesome.

Blog post: https://mrhillsman.com/posts/context-engineering-realized-co...

Proposal via Google Doc: https://docs.google.com/document/d/1qR9qa00eW8ud0x7yoP2XicH3...

While models were less powerful a couple of years ago, there was nothing stopping you at that time from taking a highly dynamic approach to what you asked of them as a "prompt engineer"; you were just more vulnerable to indeterminism in the contract with the models at each step.

Context windows have grown larger; you can fit more in now, push out the need for fine-tuning, and get more ambitious with what you dump in to help guide the LLM. But I'm not immediately sure what skill requirements fundamentally change here. You just have more resources at your disposal, and can care less about counting tokens.

LLM farts — Stochastic Wind Release.

The latest one is yet another attempt to make prompting sound like some kind of profound skill, when it’s really not that different from just knowing how to use search effectively.

Also, “context” is such an overloaded term at this point that you might as well just call it “doing stuff” — and you’d objectively be more descriptive.

A couple of days ago I fired up o4-mini-high, and I was blown away how long it can remember things, how much context it can keep up with. Yesterday I had a solid 7 hour session with no reloads or anything. The source files were regularly 200-300 LOC, and the project had 15 such files. Granted, I couldn't feed more than 10 files into, but it managed well enough.

My main domain is data science, but this was the first time I truly felt like I could build a workable product in languages I have zero knowledge with (React + Node).

And mind you, this approach was probably at the lowest level of sophistication. I'm sure there are tools that are better suited for this kind of work - but it did the trick for me.

So my assessment of yesterdays sessions is that:

- It can handle much more input.

- It remembers much longer. I could reference things provided hours ago / many many iterations ago, but it still kept focus.

- Providing images as context worked remarkably well. I'd take screenshots, edit in my wishes, and it would provide that.

And who is going to do that? The "context engineer", who doesn't know anything about the subject and runs to the LLM for quick answers without having any ability to evaluate if the answer is solid or not?

We saw the same story with "data scientists". A general understanding of tools with no understanding of the specific application areas is bound to result in crappy products, if not in business disasters.

Obviously we’ve got to tame the version of LLMs we’ve got now, and this kind of thinking is a step in the right direction. What I take issue with is the way this thinking is couched as a revolutionary silver bullet.

If the majority of the code is generated by AI, you'll still need people with technical expertise to make sense of it.

Anyway, seems like most of these algorithms are fairly ad hoc things built into all the various agents themselves these days, and not something that exist in their own right. Seems like an opportunity to make this it's own ecosystem, where context tools can be swapped and used independently of the agents that use them, similar to the LLMs themselves.

To direct attention properly you need the right context for the ML model you're doing inference with.

This inference manipulation -- prompt and/or context engineering -- reminds me of Socrates (as written by Plato) eliciting from a boy seemingly unknown truths [not consciously realised by the boy] by careful construction of the questions.

See Anamnesis, https://en.m.wikipedia.org/wiki/Anamnesis_(philosophy). I'm saying it's like the [Socratic] logical process and _not_ suggesting it's philosophically akin to anamnesis.

And to drag this back to politics - that kind of suggests that when we have political polarisation we just have context that are so different the LLM cannot arrive at similar conclusions

I guess it is obvious but it is also interesting

You are constructing the set of context, policies, directed attention toward some intentional end, same as it ever was. The difference is you need fewer meat bags to do it, even as your projects get larger and larger.

To me this is wholly encouraging.

Some projects will remain outside what models are capable of, and your role as a human will be to stitch many smaller projects together into the whole. As models grow more capable, that stitching will still happen - just as larger levels.

But as long as humans have imagination, there will always be a role for the human in the process: as the orchestrator of will, and ultimate fitness function for his own creations.

I have 3 vim commands:

ZB $n: paste the buffer $n inside backticks along with the file path.

Z: Run the current buffer through our llm and append the output

ZI: Run the yank register through our llm and insert the output at the cursor.

The commands also pass along my AGENTS.md

Basically I'm manually building the context. One thing I really like is that when it outputs something stupid, I can just edit that part. E.g. if I ask for a plan to do something, and I don't like step 5, I can just delete it.

One humorous side effect is that without the clear chat structure, it sometimes has difficulty figuring out the end-of-stream. It can end with a question like "would you like me to ...?", answer itself yes, and keep going.

https://en.wikipedia.org/wiki/Stone_Soup

You need an expert who knows what to do and how to do it to get good results. Looks like coding with extra steps to me

I DO use AI for some tasks. When I know exactly what I want done and how I want it done. The only issue is busy typing, which AI solves.

I get coining a new term and that can be useful in itself but I don't see a big conceptual jump here.

Prompts and context.

Hopes and expectations.

Black holes and revelations.

We learned to write and then someone wrote novels.

Context, now, is for the AI, really, to overcome dogmas recursively and contiguously.

Wasn't that somebody's slogan someday in the past?

Context over Dogma

I'm trying to figure out how to build a "Context Management System" (as compared to a Content Management System) for all of my prompts. I completely agree with the premise of this article, if you aren't managing your context, you are losing all of the context you create every time you create a new conversation. I want to collect all of the reusable blocks from every conversation I have, as well as from my research and reading around the internet. Something like a mashup of Obsidian with some custom Python scripts.

The ideal inner loop I'm envisioning is to create a "Project" document that uses Jinja templating to allow transclusion of a bunch of other context objects like code files, documentation, articles, and then also my own other prompt fragments, and then to compose them in a master document that I can "compile" into a "superprompt" that has the precise context that I want for every prompt.

Since with the chat interfaces they are always already just sending the entire previous conversation message history anyway, I don't even really want to use a chat style interface as much as just "one shotting" the next step in development.

It's almost a turn based game: I'll fiddle with the code and the prompts, and then run "end turn" and now it is the llm's turn. On the llm's turn, it compiles the prompt and runs inference and outputs the changes. With Aider it can actually apply those changes itself. I'll then review the code using diffs and make changes and then that's a full turn of the game of AI-assisted code.

I love that I can just brain dump into speech to text, and llms don't really care that much about grammar and syntax. I can curate fragments of documentation and specifications for features, and then just kind of rant and rave about what I want for a while, and then paste that into the chat and with my current LLM of choice being Claude, it seems to work really quite well.

My Django work feels like it's been supercharged with just this workflow, and my context management engine isn't even really that polished.

If you aren't getting high quality output from llms, definitely consider how you are supplying context.

For programming I don't use any prompts. I give a problem solved already, as a context or example, and I ask it to implement something similar. One sentence or two, and that's it.

Other kind of tasks, like writing, I use prompts, but even then, context and examples are still the driving factor.

In my opinion, we are in an interesting point in history, in which now individuals will need their own personal database. Like companies the last 50 years, which had their own database records of customers, products, prices and so on, now an individual will operate using personal contextual information, saved over a long period of time in wikis or Sqlite rows.

I don't want to delete all thoughts right away as it makes it easier for the AI to continue but I also don't want to weed trhough endless superfluous comments

Sounds like good managers and leaders now have an edge. Per Patty McCord of Netflix fame used to say: All that a manager does is setting the context.

I think good context engineering will be one of the most important pieces of the tooling that will turn “raw model power” into incredible outcomes.

Model power is one thing, model power plus the tools to use it will be quite another.

I know context engineering is critical for agents, but I wonder if it's also useful for shaping personality and improving overall relatability? I'm curious if anyone else has thought about that.

I almost always rewrite AI written functions in my code a few weeks later. Doesn't matter they have more context or better context, they still fail to write code easily understandable by humans.

Also, for anyone working with LLMs right now, this is a pretty obvious concept and I'm surprised it's on top of HN.

"Actually, you need to engineer the prompt to be very precise about what you want to AI to do."

"Actually, you also need to add in a bunch of "context" so it can disambiguate your intent."

"Actually English isn't a good way to express intent and requirements, so we have introduced protocols to structure your prompt, and various keywords to bring attention to specific phrases."

"Actually, these meta languages could use some more features and syntax so that we can better express intent and requirements without ambiguity."

"Actually... wait we just reinvented the idea of a programming language."

The concept of prompting - asking an Oracle a question - was always a bit limited since it means you're really leaning on the LLM itself - the trained weights - to provide all the context you didn't explicitly mention in the prompt, and relying on the LLM to be able to generate coherently based on the sliced and blended mix of StackOverflow and Reddit/etc it was trained on. If you are using an LLM for code generation then obviously you can expect a better result if you feed it the API docs you want it to use, your code base, your project documents, etc, etc (i.e "context engineering").

Another term that has recently been added to the LLM lexicon is "context rot", which is quite a useful concept. When you use the LLM to generate, it's output is of course appended to the initial input, and over extended bouts of attempted reasoning, with backtracking etc, the clarity of the context is going to suffer ("rot") and eventually the LLM will start to fail in GIGO fashion (garbage-in => garbage-out). Your best recourse at this point is to clear the context and start over.

- compile scripts that can grep / compile list of your relevant files as files of interest

- make temp symlinks in relevant repos to each other for documentation generation, pass each documentation collected from respective repos to to enable cross-repo ops to be performed atomically

- build scripts to copy schemas, db ddls, dtos, example records, api specs, contracts (still works better than MCP in most cases)

I found these steps not only help better output but also reduces cost greatly avoiding some "reasoning" hops. I'm sure practice can extend beyond coding.

Personally, my goalpost still hasn’t moved: I’ll invest in using AI when we are past this grand debate about its usefulness. The utility of a calculator is self-evident. The utility of an LLM requires 30k words of explanation and nuanced caveats. I just can’t even be bothered to read the sales pitch anymore.

ie. the new skill in AI is complex software development

pg said a few months ago on twitter that ai coding is just proof we need better abstract interfaces, perhaps, not necessarily that ai coding is the future. The "conversation is shifting from blah blah to bloo bloo" makes me suspicious that people are trying just to salvage things. The provided examples are neither convincing nor enlightening to me at all. If anything, it just provides more evidence for "just doing it yourself is easier."

I’ll usually spend a few minutes going back and forth before making a request.

For some reason, it just feels like this doesn't work as well with ChatGPT or Gemini. It might be my overuse of o3? The latency can wreck the vibe of a conversation.

This new stillpointlab hacker news account is based on the company name I chose to pursue my Context as a Service idea. My belief is that context is going to be the key differentiator in the future. The shortest description I can give to explain Context as a Service (CaaS) is "ETL for AI".

prompt engineering/context engineering : stringbuilder

Retrieval augmented generation: search+ adding strings to main string

test time compute: running multiple generation and choosing the best

agents: for loop and some ifs

That's when I understood that vibe coding is real and context is the biggest hurdle.

That said, most of the context could not be pulled from the codebase directly but came from me after asking the AI to check/confirm certain things that I suspected could be the problem.

I think vibe coding can be very powerful in the hands of a senior developer because if you're the kind of person who can clearly explain their intuitions with words, it's exactly the missing piece that the AI needs to solve the problem... And you still need to do code review aspect which is also something which senior devs are generally good at. Sometimes it makes mistakes/incorrect assumptions.

I'm feeling positive about LLMs. I was always complaining about other people's ugly code before... I HATE over-modularized, poorly abstracted code where I have to jump across 5+ different files to figure out what a function is doing; with AI, I can just ask it to read all the relevant code across all the files and tell me WTF the spaghetti is doing... Then it generates new code which 'follows' existing 'conventions' (same level of mess). The AI basically automates the most horrible aspect of the work; making sense of the complexity and churning out more complexity that works. I love it.

That said, in the long run, to build sustainable projects, I think it will require following good coding conventions and minimal 'low code' coding... Because the codebase could explode in complexity if not used carefully. Code quality can only drop as the project grows. Poor abstractions tend to stick around and have negative flow-on effects which impact just about everything.

The truly valuable and future-proof skill is "context engineering". This focuses on providing the LLM with the information required to reason through the task at hand. Although current LLMs present a trade-off between the size of the context and the quality of the output, this is a constraint that we can expect to lessen with future advancements.

> > Hey Jim! Tomorrow’s packed on my end, back-to-back all day. Thursday AM free if that works for you? Sent an invite, lmk if it works.

Feel free to send generated AI responses like this if you are a sociopath.

Sure it matters on a technical level - as always garbage in garbage out holds true - but I can't take this "the art of the" stuff seriously.

Assuming that this will be using the totally flawed MCP protocol, I can only see more cases of data exfiltration attacks on these AI systems just like before [0] [1].

Prompt injection + Data exfiltration is the new social engineering in AI Agents.

[0] https://embracethered.com/blog/posts/2025/security-advisory-...

[1] https://www.bleepingcomputer.com/news/security/zero-click-ai...

These agents are just as disappointing as what we had before. Except now I waste more time getting bad results, though I’m really impressed by how these agents manage to fuck things up.

My new way of using them is to just go back to writing all the code myself. It’s less of a headache.

I am leading a small team working on a couple of “hard” problems to put the limits of LLMs to the test.

One is an options trader. Not algo / HFT, but simply doing due diligence, monitoring the news and making safe long-term bets.

Another is an online research and purchasing experience for residential real-estate.

Both these tasks, we’ve realized, you don’t even need a reasoning model. In fact, reasoning models are harder to get consistent results from.

What you need is a knowledge base infrastructure and pub-sub for updates. Amortize the learned knowledge across users and you have collaborative self-learning system that exhibits intelligence beyond any one particular user and is agnostic to the level of prompting skills they have.

Stay tuned for a limited alpha in this space. And DM if you’re interested.