It’s really eye opening to work with these tools on a codebase you know deeply because these problems are everywhere.

However if I opened an unfamiliar project in another language and I wanted to add a little feature with no intention of maintaining it, I’d happily accept the changes and loop until it worked well enough for my temporary needs.

The scary middle is when you’re dealing with coworkers who don’t care about anything other than closing tickets and collecting credit. With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing.

The problem is this. Human cognitive resources are finite, so we inevitably become shallow outside our own expertise. There is no programmer who can do everything well. And as systems grow in scale, they become more modularized and fragmented, making it impossible to understand the whole system. So what should we do about this? That's always the question.

In the end, do I choose not to use AI, finish the project with uneven code outside my domain, and deliver it? Or do I use AI and deliver a program that is uniform and consistent, but not in my own style? I still don't know. I haven't found the answer yet.

I'm more interested right now in what does that abstraction look like for AI generated code. Is there some reasonable solution wherein a sandboxed component in the enterprise architecture has various attributes (e.g. the bytes i stuff into this file store component are always the exact bytes i get back from it) confirmed by methods other than a human reading its code? Those methods, are they cheaper, faster, safer than just having a human do it?

If your enterprise architects have to read every line of code in your system today then i'd claim your architecture practices have room to mature. What can derived from that, and in which scenarios, for the purposes of safely leveraging immutable write-only code? I'm not interested in evolving the code (lines of code spent to solve a business problem was never an asset, it was always a cost) if it wasn't hand crafted by a human, i still have the requirements so i can just regenerate the entire thing with the revised requirement.

I wish it were clearer in these kinds of posts how "I use AI code I don't understand" is so different from "I use libraries written by other people I don't understand", or "I work in a large codebase which was 99% written by other people, and I haven't seen all of it", or even "I use software written by other people I don't understand".

Besides, this post has nothing specific to code produced by an LLM, and placing AI in the stated reasons feels completely arbitrary, or is rather a fallacy of our times:

- I reject [AI] code when I can’t explain the approach in my own words.

- I reject [AI] code when the diff is bigger than the problem.

- I reject [AI] code when it introduces abstractions before proving they’re needed.

- I reject [AI] code when it works locally but makes the system harder to reason about.

- I reject [AI] code when I’m trusting the output more than my understanding.

LLMs are perfect for quick prototypes, speed runs, learning, etc., but if the code really matters its still not clear cut. I think the definition of what "really matters" is very project dependent of course As an extreme example you would want to understand every line of the code for the control system runs an MRI machine or a jet engine since bugs might mean life or death. Depositing money into the wrong account might not kill anyone but could lead to severe economic losses. But, then again, even problems in far less consequential software may be drastically sub-economic (i.e. saving $1000 on the implementation might cost $10000 if customers aren't happy and fails to re new). Pick your scenario I guess.

The problem is, this isn't going to change regardless of how well a new model scores on a benchmark. It seems actually AGI is needed.

Adequate often means done and cheap

I try to make sure the architecture docs of the code base are refreshed regularly based on recent changes, so it's easier for humans and AI agents to make sense of the code.

I also regularly stop all other developments and just focus on auditing the code base with these AI's to make sure they are secure, robust, clean, and well structured and well tested -- some refactoring would be needed most of the time, and it's well worth it.

With this approach, nowadays I often merge code from AI without completely understanding what it's doing, but seems the code has been working so far. :)

Being able to step back and say "this was a failure and we need to discard the day's work and start over" is still hard with LLMs.

What I'm hoping to build ultimately is something that works more like a pair-programming partner than existing harnesses do. I want the user to be an engaged part of the development process all the way through, I don't want the agent disappearing to work on its own. I even want to make it possible for users to swap into the driver role and have the LLM automatically assume the role of navigator when that happens.

There's more info in the readme (actually the readme is all that exists so far, I wanted to get the idea straight in my head first):

https://gitlab.com/philbooth/opair

Even if nobody else uses it, I hope it will be a useful tool for myself and help me find a way to work with LLMs that doesn't harm my mental models, which is what I feel current harnesses do.

What I found myself doing is operating in two modes: 1. For projects that require my attention, I plan and instruct LLM, when needed will draft some code and ask agent to make it better or finish the mundane part (write code and leave gaps with comments asking agent to finish) 2. Full automode where I use spec driven development and TDD - I only ask for changes based on existing PRD, which agent also have to update. Here I do not look at the code at all.

Seems to be working just fine.

And the industry is rushing towards it, whilst failing to train people who are able to fix it

When implementing its often a lot of misses with a few golden hits. The other day it used flex for a table layout while our app uses tables everywhere sigh.

Another typical one is that it tends to prefere frontend aggregation and looping of data instead of letting the database and backend deal with it.

Using mix of claude, cursor composer and codex.

> I reject AI code when I can’t explain the approach in my own words.

I think that's the key problem. LLMs turn code into big, black boxes. Sure, theoretically nothing stops me from reading all that code. I don't, however, because it's wasted effort. The time it takes me to really understand the code is IMO better spent just writing it myself. Once written, I have a very good understanding. Read ten times, not so much.

It reminds me of pen and paper. Journaling the old way remains the best way to learn something, but writing on a computer is much more convenient.

Now we are getting to the point where we are speed-running the deskilling of engineers into comprehension debt and they themselves rapidly losing confidence in reviewing code they did not write.

I think this blog post [0] is the best example of what could go entirely wrong and even worse when you do not know the technology.

If you cannot explain a change even when "the CI is green" or "all tests passing", I will immediately reject it.

Maybe great for vibe coding prototypes, but it all changes when that code is deployed onto mission critical systems. Just ask Amazon with Kiro. [1]

[0] https://sketch.dev/blog/our-first-outage-from-llm-written-co...

[1] https://www.reuters.com/business/retail-consumer/amazons-clo...

TLDR: Keeping your codebase human readable and reason-about-able is not just helping humans to stay relevant. It will save costs for LLMs to maintain it.

How do you verify that it works?

Agents respond really well to feedback! They have no ego and they’ll happily improve code if told where and how. But you need to provide the tools that provide that feedback without your involvement - otherwise you can’t scale.

All the linting and autoformatting you can put in, is a good start. Next, create custom scripts that check for every single dumb AI-ism you can think of, tell the agent about them, tell it to use them to check its work, and put them in hooks so the harness refuses to let the agent stop until all your linters show no errors.

Then, keep iterating basically forever. Any dumb AI-ism you see, make a linter for it, give it to the agent, and enforce it using the harness.

I’ve spent months doing this. When I review a PR - which was built by the agent with TDD so it definitely works - I’m no longer asking if it did dumb stuff or confirming it conformed to the architecture or duplicated code or missed opportunities for reuse. That’s all linted for. I don’t worry about duplication or outdated docstrings/comments because the self review caught all that. I mostly read it to look for opportunities to make the feature even better & more useful.

If this makes no sense or you disagree it’s possible, my contact details are on my profile and I’ll be happy to give a demo.