The really interesting thing to me is that formal verification wouldn't have helped there either -- it would have just written a mathematical proof of the correctness of the backwards feature.

LLM code past a few pages is really unreadable. A workflow of LLM writes code, human reads/checks/corrects is IME very unproductive and frustrating, and you're usually better off writing by hand (YMMV).

If you want to reap some productivity improvements, it can't rely on you reading, grokkong and accepting the LLM code line by line. You just kind of need to suck it in.

So the best you can do is to indeed, constrain it heavily, maybe write the interface, then have the LLM write tests, then have it write the code.

I'm not saying I like the approach, but I agree manually verifying LLM code is very frustrating and unproductive.

I find it plausible that an extra agentic review pass and more testing can bring this number up to the point that one never needs to review code again. AI writes pretty good code nowadays.

(You still need to be diligent and decide the architecture during the planning, and read the gotchas and "things to note" that the agent will spit out at the end of implementation if it had to diverge from the plan.)

Giving up reviewing code is not strategy, but because:

Review AI code line by line is like watch movies frame by frame, and is impossible, very difficult, terribly boring, or abandoned sooner or later.

I think a lot of the "AI Addicts" are masking illiteracy, which is why they cling so passionately to the technology.

Which, is fine, but not what most people are hoping for with these tools.

this is definitely a better outcome since all the human effort now shifts to spec, design, test scenarios tailored to the domain - as it should be.

having seen my share of human slop through decades (mine included), finally it's a relief to not have to deal with devs that don't have high standards, and the endless arguments/politics that ensue.

with llm it's just a text file away from compliance.

Any opinion that starts with such a blatant appeal to authority can safely be ignored.

When you start getting good results from agents you soon realize you are the bottleneck.

Automating the verification of the code is the way to go, otherwise it's just not worth it. It takes longer to read and understand code than to write code, so why bother with agents if you are going to manually review it all anyway?

One thing I miss after ditching Copilot (it got too expensive) was how I could trivially ask for features to be written by one model and verified by another. Have Opus write it and GPT or Gemini verify it.

I figured they were entirely separate models and therefore unlikely to hallucinate in the same way, so it gave me a quick sense of confidence.

Currently I use claude code (different models but all variants of the same) so I have them do planning, review of the plan, implementation, review of the implementation, and unit tests. It's fine, but copilot felt easier.