Around 2 years ago I had to clean up a mess because someone who doesn't really know what they're doing designed an instancing system for a game. They heavily used AI to design every part of it and it was awful. Data corruption, performance problems, lost items, race conditions everything you can think of was an issue. It took me 2 weeks just to get it to an "acceptable" level and it was still awful as the whole design was simply flawed.

Fast forward to today: different company, same person, SAME issues with an AI that is 'allegedly' much better than it was. This time I only heard about these issues and wasn't the one who had to deal with it so I just had a really good laugh.

AI is only as good as the person using it, that's why we have such vast range of what people "claim" AI can do and why everyone has way different opinions of it.

If you need someone to tell you how stupid your ideas are, either learn to ask in a way that invites criticisms, or hire a senior engineer. Don't try to influence LLM makers to make AI less deferential. That's the worst possible direction to go

When one person is able to do too much too quickly, they can create more liability than they can accommodate if something fails.

It is essential that a human is responsible for the utilization of any AI output in the real world, but that is not enough. For our own sakes, we must find ways to minimize the tech-debt bankruptcy blast-radius of those who would utilize (knowingly or unknowingly) AI to create flawed systems upon which others rely.

An example: Jim vibe-codes an extremely popular micropayments app. He hires a few people and sees the company as the WhatsApp of money -- a few engineers and some agentic support staff. It pulls in a few million in VC money -- enough to draw in tens of millions of users. One day, a flaw in the infrastructure causes all of the users' unsalted banking information to be released.

Agentic AI allows that entire list of customers to be exploited rapidly, so the losses for society are in the tens of billions. Jim's company is immediately bankrupt, of course, but there are only a few million dollars to go around.

Today, most of Jim's incentives are to go ahead and build that app. The same is true for his few employees and a small VC contribution. There's not much capital at risk compared with the societal exposure.

How do we ensure that AI users are accountable not just for their actions, but for the size of the risk-exposure that they create?

    Brainstorm N ways to do X. Sort by probability.

Don't outsource all of your thinking.

When left to its own devices with the instructions "make an assembler for the architecture in ISA.md" -- well Claude picked Python as the implementation language. Tokens lifted through a bunch of regex. No expression parser! Oh dear. My first assembler was like that too, to be fair.

However, when I described the desired passes and their types:

    collectDefines :: [SourceLine] -> Either AsmError ([SourceLine], Map Text Text)
    
    runLitPool :: [SourceLine] -> Either AsmError ([SourceLine], [(Text, LitKey)])
    
    evalExpr :: Text -> Map Text Text -> Either AsmError Int

Irony is using Claude to write a beautifully structured, 2,000-word essay warning the industry about the dangers of letting Claude design things. It’s self-awareness by proxy.

> It’s just incapable of the thing that makes a real architect valuable: saying “no.”

From my experience Claude is excellent at saying "no". It won't say "no" if the prompt doesn't call for it (it won't say "no" to your direct request to do something, usually). But it offers good critique and happily pushes back if you make it clear that that's a first class option.

The article kind of lost me here. Agents are way more than that, today. And the author knows it, as later it says stuff like

> Claude will never do this. It’s trained to be helpful.

But the first phrase just tell me author just have a deep dislike for agents and it's looking for rationalizations for that feeling.

Part of the criticism is on point, sure. But if it "being trained to be helpful" is a problem, it's fixable. It can "be trained to be more critical".

Later:

> But it wasn’t designed for your team. (..) It was designed for the median of everything Claude has seen. A generic best practice for a generic problem at a generic company. Which is to say, it was designed for nobody.

That's non-sense. Anybody who understand algorithms know that, sure, on a first instance you have a "good algorithm" that has a good performance on average, or in worst-case. But then, you can design algorithms that are adaptive to the input. Same applies here.

For what it's worth, Opus tells me that I'm wrong and not to do things all of the time. When I reflect on why that is, it's because of the way that I prompt it. You could say that I am subconsciously avoiding setting both me and the LLM to fail in the way the author projects as inevitable.

Specifically, I don't come to it with prompts that resolve cleanly with "tell me how clever I am" replies. I always present myself as a domain expert - because I am a domain expert - and I make it clear when I am open to getting input on the pros and cons of different paths.

With a conclusion that will be unsurprising to any successful LLM daily drivers, this strategy has been remarkably effective.

There is also a mountain of bullshit eg "[architectural|design] [anti]patterns" that are written mostly (I would argue) to sell something (consulting, hardware, etc). This is typically at odds with a good solution.

There is a relative lack of actual documented architectures that work. Not only do you need the details but also the usage of these systems so as to judge what "good" is.

We will probably just go the HTML route with architecture: take a really bad base and just keep throwing compute, memory, and network I/O at the problem until it works.

Note to self: invest in energy ETFs.

Like any tool, it takes effort to master and configure. Before LLMs, I used cookiecutter templates to codify my best practices. Now I invest in custom skills and context management so the tool produces solutions that match my standards and team conventions.

I agree with the author that the craft still matters, and probably now more than ever. I'd add that mastering and configuring your agent is now part of the craft.

Tangentially, the usage of Architect keyword sounds out of place here, I don't like saying it but from what I seen the industry has destroyed the role of architects gradually over the time. There are specialists however you do not have generalists who are good at different parts of the system at scale anymore.

Well, can you prompt it to think about the problem?

> A good architect’s most important skill isn’t designing systems. It’s knowing which systems not to build. It’s pushing back on complexity. It’s asking “why?” five times until the actual requirement emerges from the aspirational nonsense. It’s telling the CTO that their conference-inspired idea is a terrible fit for the team they actually have.

Except for that last one, that all sounds very solvable. Of course, the last one is the most important one. But most humans will struggle there too.

However the good part, what I had planned for 5 years, now looks like doable in 6 months. Looking forward to real use by the end of this year.

Ref: https://github.com/ramshankerji/Vishwakarma

Yes, that's assuming you take time to clean up now and then. If you don't, that's on you.

Just tried this. Claude Opus replied “probably not” and recommended a well structured monolith.

I don’t doubt the problems in this article exist and I’ve seen them, in my experience the vast majority of people are still shipping the same quality or better than before they has Claude. Personally, I feel like I’m probably developing at about 1.5x the speed of not using AI tooling. It’s not a silver bullet, but it can be a great assistant.

Every decision that matters? Some, yes. Is the author only noticing the decisions that go wrong?

So... manually learn architecture and security and then vibe code away?

Does so many people in HN just upvote by title?

It doesn't disappear when you make 1 person do the work of 3. It simply is aggregated

Suppose you had a pod of a PM, designer, and analyst. Leadership lays off the designer and analyst and now the PM can move faster with AI. Hooray!

Well...when the complaints about how it looks on xyz device roll in, who is implementing that? Or you launch the product with much fan fare, adoption is terrible - you double check the numbers and oops, the sizing you had from Claude was actually 10x off.

Who is holding the bag? You are! Not Claude

I'm convinced this is one reason we are seeing slower than expected adoption of AI broadly in tech companies, because it's hard to trust - we know Claude can make mistakes but how do you know what's right vs not? Most people don't want to sense check so they just keep doing the work the way they know best.

I think this could be one thing that pops the AI bubble - execs try to force this, people go along, and results are not any better for this reason. Sure you save some salaries and ship more quickly, but you don't build the right thing and you are fixing more things after launch. Which one is actually better?

The thing that I find Claude incredibly good at when I'm designing architecture is working more like a research assistant on briefing decisions. It has the ability to read the entire code base and draw some conclusions. It can pull from lots of best practices and the millions of blog posts about this or that pretty effortlessly, which would take me a lot more time.

And then if asked, it can do a really good job of laying out the landscape around decisions and walking through the trade-offs. Like the author of this post, I found that if you let it, it will certainly be happy to just come up with some architecture and run with it, often in ways that will paint you quite rapidly into a corner.

But if you ask it to present you with all the trade-offs and let you make the judgment calls, it's great for that too.

That's certainly how I use it. And I think, just like anything else, working with AI is a skill, and similar to working with libraries, SaaS providers, service providers, frameworks, or anything else that's a "helper." You learn how something that could work but will fail silently is a problem, or you learn how depending on a fly-by-night SaaS company for a key framework is different than depending on a well-populated open source project, etc.

In the same way, you learn that relying on Claude's judgment is a bad idea, while relying on Claude's ability to summarize, brief, and research can be incredibly efficient.

We can describe this without talking about technology - so pre-AI.

Imagine the owner of a construction company firing all the architects. After all, he's been the owner for 15 years. He has led the construction of dozens of projects. He's also rich, and being rich seems to be an ego-multiplier.

Why should he waste money on architects? Or more importantly, why should he allow them to constantly annoy him with pushbacks: "This could be a problem if the sustained wind is greater than ... ".

Those engineers obviously don't know the real world. Their elitist education has made them afraid to make bold decisions. Regulations are anti-progress!

Thankfully, that owner now has AI tools. He doesn't need those not-always-yes-people. He now has a perpetual yes-bot.

So where are we now? We're in the same place we always have been. People need to have the humility to recognize that despite their authority, influence, or wealth, they still need other people. And especially, they need other people to challenge their orders or their requests.

But I don't really see this situation self-correcting. There's now so much money concentrated amongst a few who will spray it over exactly the kind of people who do not want to listen to others that most activity in the future will be for naught. Yes, some unicorns will be fabricated, and some people will make a lot of money; but real value will not be created often.

Therefore, I implore the actual thoughtful creators: Do build things, but do not sell out. Look to the past. Create companies where every employee was valued, and every employee had some voice. Yes, use AI. But test and measure where it really helps. And be skeptical, just as you would if someone came to your door promising a black box that would double your profits.

For any nontrivial task I spend 2-8 hours in specification (I spent 3-4 hours on a stateless rust CLI tool design this weekend) and detailed task breakdown in implementation planning.

I use TDD to start with red tests that turn green when acceptance criteria are met.

I write agents to use to check work and they are my enforcers of constraints, as well as fresh eyes. I use these agents for spec review, plan review and code review.

I am actually pretty proud of the projects I create with generative AI. I just apply a lot of discipline so I don’t end up with slop.

So it's the person using the AI that's the problem, not the technology itself?

I ask for multi-turn evaluations and often times parallel sub-agents to get consensus about something, there is plenty of back and forth. Sometimes I have to tell the AI to shush up and that we're doing things the simpler not more correct way cause we need to ship sooner, but generally with enough exploration most ideas are pretty good. As long as you literally don't rubber stamp everything, Opus does an okay job (I also tried out DeepSeek, that one was a bit worse at planning but passable).

Then again, I doubt the CTO in question ever is like: "Okay, after reviewing these other 3 projects that I put in your workspace for comparison against prior work and all of those other documents that provide context, and after writing this detailed plan, would you like to ask me 20-40 additional clarifying questions before we lock in on this design? Anything that is not completely clear or ambiguous."

I have noticed that better results come from throwing more compute at the problem, though. Even in regards to writing code, it will produce something that is sometimes arguably slop, but when there are 3 parallel sub-agents reviewing any changes before commit, often it will surface multiple rounds of fixes in the review loop until none of them find any serious/critical issues.

> Real architecture is full of trade-offs that only make sense in context. You pick Postgres over DynamoDB because your team knows Postgres and you’d rather ship in two weeks than spend a month learning a new data model. You skip the service mesh because you’ve got four services, not forty. You use a monolith because the problem is simple and microservices would be career-driven development.

I also think that all of this should be encapsulated in ADRs or any kind of docs. Then you can point whoever joins the team, or your LLM tools, at the folder and let them get brought up to speed, instead of having to track down whoever wrote a particular piece of the system for questions, or have to do digital archaeology in old Jira issues.

I thought this happened alot. I started using chatgpt to critique my new art hobby and also help me learn unreal engine.

It's basically tearing into me on the art. It's almost ruthless, especially with the verbosity it's like I get it.

Using it for unreal engine, it pushes back on alot of my begginer ideas and how to write code that uses the engine. It corrects me alot. It's called things I wrote becasue I was lazy sloppy or quick hacks that work for now.

In my workflows Claude does pushbacks all the time and justifies why. There is back and forth just like a colleague. It’s not perfect but the results are usually good

This.

If you ask it to be fair and non-biased and provide pros and cons and give possible alternatives - it will. The catch - you might understand the explanation if you don't know the domain good enough.

Overall - a very, VERY good article, thank you!

I then logged in from completely different account, described the problem and asked to design architecture of backend with the same functionality and performance. Suddenly I got standard distributed enterprise monsterware running on amazon. Yes - it could do the task for at least 100-time price for comparable performance and way more complex to manage at even more markup

I then have merged both conversations and started grilling Claude why is it doing such a disservice to a customer who is looking to optimize ROI.

Claude's answer was basically - It runs on large corporate's development methodologies / propaganda that outweighs every rational choice just because of sheer volume

So yes, be careful what you wish AI to do. It can and will set you up.

It’s been quite good for my productivity and the best part for me is that I learn what I’m writing while I’m writing. I can just write things I already understand a lot faster than before. When I work with agentic, I find that I still have to deeply learn the system, but I’ll have to learn it when it falls over instead of at review time.