FRESH

Hacker News

Home

Reviews have become expensive, rewrites have become cheap

79 points by arzh2

by dmitrig01

1 subcomments

Writing blog posts has become cheap, making them sound human has become hard.

by SCdF

1 subcomments

> LLMs aren’t lazy. They don’t cut corners because a simpler solution feels good enough. If they know how to solve something thoroughly, they will.
I don't know why they think this, but no? Perhaps it's badly expressed, but LLMs cut corners all the time. It's sort of their core fault really.
Anyway, I disagree with the core premise[1]. Re-writes are not cheap, because 1) code can be so bad it's unclear how to rewrite it[2], b) code can be so significant it's challenging to rewrite it (architectural choices, schemas, etc), and lastly if you're ever wanting to own your own code and not rent it from LLM companies, having it understandable by a human is still a goal worth working toward.
[1] To be fair, I think OP _might_ be talking about rewriting in the moment of the thing being built, but with some unspoken rule that once they think the change is good enough, then they are reviewing all the code? They don't make it clear..
[2] it's not even that hard. Write a test that exercises an end result and not the rule that causes that end result and 6 months later you've forgotten why the code is like that. I had to maintain a piece of software once where the primary form of tests were a bunch of snapshots of an end report being generated, based on some initial data input, mostly all unlabeled. The code was like "do this SQL query on table A and then take the second result". Why the second result? Your guess is as good as mine! I couldn't even work out why they were querying table A and not table B...

by saidinesh5

1 subcomments

We faced a lot of this this year. Eager new joinees sent 100s of lines of CLs to review to "improve" little things.
Not only did the new changes did not fix what they thought it would fix but it broke other things in unexpected ways.
I brought in two changes after that:
* I'm not reviewing/reading anything that you yourself did not read / test in the target environments properly. If all it takes is an LLM prompt, I could be issuing the same prompt to make my life easier.. and If you're sending a CL, you should be owning the code you send.
* Me being more involved in the design process so review burden itself becomes lower. A bit of pair programming from time to time helped too.
Not sure how things will turn out after this but so far they seem better.

by usernametaken29

3 subcomments

> The shortest path for the model is to implement it completely
Have you worked with LLMs??????????? “I disabled the test so it’s not run so now all the tests pass” is not a hypothetical it’s pretty common. LLMs frequently do shortcut learning. The reason why reviews are expensive is because you still need to do all the steps in order to understand if a shortcut is justified.

by dap

0 subcomment

If your plan is to not review and just have the LLM rewrite if it doesn’t work, it doesn’t sound like the rewrite is gonna be any better.

by geraneum

0 subcomment

> An LLM defaults to building when it should be buying. Not because it doesn’t know about existing libraries, it often mentions them, but because for an LLM, writing two hundred lines of implementation is the same cognitive effort as writing two lines of import.
Or maybe they are trained that way. It’s more tokens used and more money you need to pay.

by drudolph914

0 subcomment

I’ve personally reached a point where if I’m saving time typing, but gaining back that time on reviewing and understanding, I may as well have just written the code most of the time. AI is great for boilerplate and learning, but my team has given up on trying to use the tool for entire implementations

by mpweiher

1 subcomments

Rewrites are only cheap if the rewrite solves the problem.
And the rewrite can only reliably solve the problem if you understand the problem, and even then it's obviously not a guarantee.
If you have a huge blob of code that nobody understands then re-generating a new blob of code that nobody understands is unlikely to solve the problem.
That said: I've always been a fan of optimistic rather than pessimistic merging, at least for human-generated code.

by tabs_or_spaces

0 subcomment

> LLMs aren’t lazy. They don’t cut corners because a simpler solution feels good enough. If they know how to solve something thoroughly, they will.
This is a severe misunderstanding in how LLMs work..
I don't know how this got on my front page....

by eschneider

2 subcomments

Failures in production remain expensive.

by cadamsdotcom

0 subcomment

This is very handy.
Say you discover in review that your agent wrote code before tests. A file that was added, no corresponding test file.
Tell the agent to read in the file, delete it (or roll back), then reimplement using red/green TDD.
Claude can test-drive an identical reimplementation but this time by writing red tests and making them green. Now you have coverage that you KNOW covers the code because you watched those tests fail before they passed.
Going further I added a note to my `/review` command so the agent looks for this mistake for me in its self-review. If it finds uncovered code it makes a plan which includes the delete-reimplement trick. Now all I do is approve that plan - no hunting needed, no explaining, no repeating myself.
You could even put this in a Ralph Wiggum loop and give yourself super high quality tests by going file by file.

by hintymad

0 subcomment

Shouldn't we really consider review and rewrite together? If so, the economics may not change as drastically[1]. It's just like writing code by hand: we spend lots of time organizing our thoughts and examining our code, which is not that different from reviewing.
[1] Unless you're an engineer in Anthropic, so you just spend you time writing "loop".

by bob1029

0 subcomment

LLM generated code does have some advantages at code review time. I find myself paying more attention to the diffs being in the right areas than their actual line-by-line content. It's been a long time since I've seen something like a conditional hallucinated incorrectly. The accuracy in the details is exceptional now. It's mostly a matter of ensuring we are dealing with the correct details.
The compiler does a lot of heavy lifting in the codebases I work with. Strongly typed languages seem mandatory if you want to use an LLM. Worrying about every symbol is too much. Let the CPU get hot for a few seconds on the paranoia and stay focused on the minimap.

by m463

4 subcomments

ai can do some of the reviewing, checking calling and called arguments, even things like crufty shell scripts.
but the higher-level "should you do this?" or "check your design" - could AI do that stuff?

by rob74

0 subcomment

The title sounds a bit like a catch-22: ok, rewrites are now cheap, but rewrites have to be reviewed too?

by dv_dt

0 subcomment

This makes me think of an LLM induced summer of endless rewrites where you get to experience a random reroll of what 80% works on any given rewrite.

0 subcomment

by nnevatie

0 subcomment

As explained by a LLM.

by hluska

2 subcomments

I’m not sure I agree with this or maybe I don’t understand. In my experience, the over engineered code LLMs create have more big problems. Rewriting vast parts of code when I have an outage or need a new feature means the code evolves far faster than my understanding. That gets more and more dangerous. Or maybe I’m not smart enough to follow the new pace?

0 subcomment

by nryoo

0 subcomment

[dead]

by YuriiKholodkov

0 subcomment

[dead]

by simianwords

6 subcomments

Why is reviewing hard? I use LLMs for reviewing. It is dogmatic to review every line written by an LLM.